Time Series Components
Decomposing Time Series
Time series can be decomposed into three components: Trend (long-term direction), Seasonal (repeating patterns at fixed intervals), and Residual (random noise). Decomposition helps understand what drives the series and improves forecasting by modeling each component separately.
Long-term Direction
Trend represents the long-term movement in the series. It can be linear (constant rate of change), exponential (accelerating), or more complex. Identifying trend is crucial for forecasting - extrapolating trend is often the main predictive component for long-horizon forecasts.
Periodic Patterns
Seasonality is regular, predictable variation that repeats at known intervals. Retail peaks in December, energy use peaks in summer/winter, website traffic drops on weekends. Understanding seasonality allows adjustment and better comparison (year-over-year vs month-over-month).
What's Left Over
Residuals are what remains after removing trend and seasonality. Good model residuals look like white noise - random, centered at zero, constant spread, no patterns. Structured residuals indicate the model is missing something important.
Stationarity
What is Stationarity?
A stationary series has constant statistical properties over time: same mean, same variance, same autocorrelation structure. Most time series methods assume stationarity. Non-stationary series (trending, changing variance) must be transformed before analysis.
Importance for Analysis
Non-stationary series cause problems: two unrelated trending series can appear highly correlated (spurious correlation), forecasts deteriorate quickly, and hypothesis tests give wrong answers. Always check stationarity before modeling.
Making Series Stationary
d = 2: Remove quadratic trend
Seasonal diff: Yt - Yt-s removes seasonality
Differencing is the most common way to achieve stationarity. First differencing (d=1) removes linear trends by computing changes between consecutive values. Seasonal differencing removes periodic patterns. Most series need d=1 or d=2 at most.
Augmented Dickey-Fuller Test
The ADF test checks for stationarity by testing for a 'unit root' (indicator of non-stationarity). Low p-value means we can reject non-stationarity. Apply differencing and re-test until stationary. Also check KPSS test for confirmation.
Autocorrelation
Correlation with Past Self
Autocorrelation measures how much a series correlates with lagged versions of itself. High autocorrelation at lag 1 means today's value is similar to yesterday's. This is the key feature distinguishing time series from cross-sectional data and the basis for forecasting.
Autocorrelation Function
The ACF plot shows autocorrelation at each lag. Bars exceeding the significance bounds (±1.96/√n) indicate significant correlation. Slow decay suggests trend (needs differencing). Spikes at regular intervals suggest seasonality.
Partial Autocorrelation
PACF measures correlation at lag k after removing effects of shorter lags. If lag-2 ACF is high just because lag-1 is high, PACF(2) will be near zero. PACF helps identify the true 'memory' of the process - how many past values directly influence the present.
Identifying Models from ACF/PACF
| Model | ACF | PACF |
|---|---|---|
| AR(p) | Gradual decay | Cuts off after lag p |
| MA(q) | Cuts off after lag q | Gradual decay |
| ARMA(p,q) | Gradual decay | Gradual decay |
| White Noise | All near zero | All near zero |
ACF and PACF patterns help identify appropriate models. AR processes show PACF cutoff. MA processes show ACF cutoff. Combined ARMA shows decay in both. White noise (random) has no significant correlations. These patterns guide model selection.
ARIMA Models
ARIMA(p, d, q)
ARIMA combines three components: AR (autoregressive) uses past values to predict current, I (integrated) handles non-stationarity through differencing, MA (moving average) uses past forecast errors. Together they model a wide range of time series patterns.
Autoregressive Component
AR models predict using weighted past values. AR(1) uses one lag, AR(2) uses two lags, etc. The coefficients φ measure how strongly each lag influences the current value. Values revert toward the mean; the rate depends on φ.
Moving Average Component
MA models use past forecast errors (shocks) to predict. Unlike AR where effects persist indefinitely, MA effects are temporary - an MA(q) shock affects only the next q periods. MA captures short-term dependencies and sudden shifts.
Choosing p, d, q
Model selection: (1) Set d by differencing until ADF rejects non-stationarity, (2) Examine PACF for AR order p, (3) Examine ACF for MA order q. Use information criteria (AIC, BIC) to compare models. Lower is better. Always validate with residual diagnostics.
Forecasting
Forecasting Approaches
Simple benchmarks often work surprisingly well. Naive (tomorrow = today) beats many complex models. Seasonal naive uses last year's value for same period. Compare your model against these baselines - if you can't beat naive, your model isn't useful.
Prediction Intervals
Point forecasts are almost always wrong - report prediction intervals. Intervals widen with forecast horizon because errors accumulate. A 95% prediction interval means the true value should fall within those bounds 95% of the time.
Measuring Forecast Accuracy
Evaluate forecasts using holdout data (train on past, test on future). MAE is interpretable in original units. RMSE penalizes large errors more. MAPE gives percentage but fails at zero. MASE compares to naive baseline - below 1 means you're adding value.
Common Forecasting Mistakes
Forecasting pitfalls: Overfit models perform great in-sample but poorly out-of-sample. Structural breaks (policy changes, crises) invalidate historical patterns. Long-horizon forecasts are essentially guessing. Models need regular updating. The future often differs from the past.
Real-World Examples
Apply your knowledge to practical scenarios
Retail Sales Forecasting
A store has 3 years of monthly sales data. They observe: overall upward trend, peaks every December, and dips every January. Total sales: Year 1: $1.2M, Year 2: $1.4M, Year 3: $1.6M.
Key Insights
- Decomposition separates trend, seasonality, and noise
- Forecasts should account for ALL components
- External events can break historical patterns
Are Stock Prices Predictable?
An investor analyzes 5 years of daily stock prices for a major company. They fit a complex ARIMA model and achieve R² = 0.001 on out-of-sample predictions. The naive forecast (tomorrow = today) achieves similar accuracy.
Key Insights
- Random walks are fundamentally unpredictable beyond naive forecasts
- Low R² in forecasting is common—don't expect high accuracy
- Market efficiency limits financial forecasting
The COVID Structural Break
A hotel chain's forecasting model was trained on 2015-2019 data. In March 2020, actual bookings crashed to near zero while the model predicted normal seasonal patterns. The model continued predicting 'recovery' that didn't match reality.
Key Insights
- Structural breaks invalidate historical patterns
- Models fail when the underlying system changes
- Always ask: 'What could break my assumptions?'
Time Series Quiz
Test your understanding • Passing score: 70%