Foundations of Time Series Analysis

Foundations of Time Series Analysis

The ability to understand and predict temporal phenomena stands as one of the most valuable competencies in modern data science. From forecasting stock prices and demand for products to anticipating weather patterns and disease outbreaks, the analysis of time-ordered data permeates virtually every domain where information changes over time. This chapter establishes the mathematical and conceptual foundations that enable such analysis, introducing the vocabulary, concepts, and tools that will recur throughout this book. Our journey begins with a fundamental question: what distinguishes time series data from other forms of data, and why do these distinctions demand specialized analytical techniques?

Time series data consists of observations collected sequentially at regular or irregular time intervals, where the temporal ordering carries substantive information about the phenomenon under study. Consider the daily closing price of a technology stock over five years, monthly unemployment rates reported by a national statistics bureau, or the minute-by-minute electrical load measurements of a power grid. In each case, not only do the values themselves matter, but their position in the temporal sequence fundamentally shapes how we must interpret and analyze them. This contrasts sharply with cross-sectional data, where observations are collected independently at a single point in time—imagine a survey of household incomes on a given date, or measurements of physical characteristics across different individuals. Cross-sectional analysis treats each observation as an independent sample from some underlying population, allowing standard statistical techniques that assume independence among observations.

The independence assumption, so convenient in cross-sectional analysis, fails catastrophically when applied naively to time series data. Successive observations in a time series are rarely independent; today's value typically contains information about tomorrow's value, last month's figure influences this month's, and the entire sequence reflects underlying processes that evolve over time. This phenomenon, known as temporal dependence, represents the central challenge that motivates the specialized techniques developed in this book. Failure to account for temporal dependence leads to models that are misspecified, forecasts that are inefficient, and inferences that are invalid. The statistical framework we develop here provides the tools necessary to respect and exploit the temporal structure inherent in such data.

1.1 The Anatomy of Time Series

Understanding time series requires decomposing the observed sequence into constituent components, each reflecting different aspects of the underlying data-generating process. This decomposition serves both as an analytical framework and as a foundation for subsequent modeling efforts. The classical decomposition approach identifies four primary components: trend, seasonality, cyclical patterns, and residual noise.

Trend represents the long-term movement in the series, the direction in which the data moves over extended periods when short-term fluctuations are averaged out. Trends may be upward or downward, linear or nonlinear, and may shift direction at various points—a phenomenon known as structural change. The global temperature record shows a clear upward trend over the past century, while the sales trajectory of a company launching a new product might display S-curve dynamics as adoption spreads through the market. Trend captures what remains when we abstract away from shorter-term movements, reflecting fundamental shifts in the underlying system being measured.

Seasonality refers to regular, predictable patterns that repeat at fixed intervals, typically corresponding to calendar or institutional cycles. Retail sales exhibit annual seasonality with peaks during holiday seasons; electricity demand shows daily seasonality with morning and evening peaks; hotel bookings display weekly seasonality with weekend premiums. The defining characteristic of seasonal patterns is their periodicity—the predictable repetition at known frequencies. Seasonality arises from human behavior (weekly work schedules, annual holidays), natural phenomena (seasons, lunar cycles), and institutional arrangements (fiscal years, school calendars). These patterns are not mere curiosities; they carry substantial information for forecasting and decision-making.

Cyclical patterns share similarities with seasonality but differ in crucial respects. Cycles are fluctuations that occur at irregular intervals and with variable amplitudes, often associated with economic phenomena such as business cycles. Unlike seasonal patterns, which repeat with clock-like regularity, cyclical patterns lack a fixed period. The duration of an expansion or contraction phase varies from cycle to cycle, making cyclical patterns inherently more difficult to forecast. While seasonality is tied to the calendar, cycles reflect endogenous dynamics of the system—feedback mechanisms, resource constraints, and cumulative effects that generate boom-bust dynamics.

Residual noise, sometimes called the irregular or random component, captures the portion of variation that remains after accounting for systematic components. In an ideal world, residuals would be pure random noise—unpredictable variation that cannot be explained by any knowable factors. In practice, residuals may reveal model misspecification if they contain remaining structure, such as autocorrelation or conditional heteroscedasticity. Careful examination of residuals constitutes a critical diagnostic step in any time series analysis.

The relationship among these components can take two forms. In an additive decomposition, the observed series is expressed as:

$$y_t = T_t + S_t + C_t + \epsilon_t$$

where $T_t$ represents trend, $S_t$ seasonality, $C_t$ cyclical component, and $\epsilon_t$ the residual. This formulation assumes that the magnitude of seasonal and cyclical effects remains constant regardless of the level of the series. An additive decomposition is appropriate when seasonal variations are roughly constant across different levels of the series.

Alternatively, a multiplicative decomposition:

$$y_t = T_t \times S_t \times C_t \times \epsilon_t$$

captures situations where seasonal effects scale with the level of the series. Monthly retail sales might exhibit larger absolute seasonal swings during peak seasons than during off-peak periods, suggesting multiplicative seasonality. The multiplicative form can be converted to additive form through logarithmic transformation, as $\log(y_t) = \log(T_t) + \log(S_t) + \log(C_t) + \log(\epsilon_t)$.

1.2 Classical Decomposition Methods

The classical approach to time series decomposition emerged from the work of statisticians in the mid-twentieth century and remains influential today. These methods provide intuitive, computationally simple techniques for separating the components of a time series, establishing intuition that extends to more sophisticated modern approaches.

Moving averages form the foundation of classical decomposition. A simple moving average of order $m$ computes the average of the most recent $m$ observations:

$$\bar{y}_t = \frac{1}{m} \sum_{i=0}^{m-1} y_{t-i}$$

When applied recursively across the series, moving averages smooth out short-term fluctuations, revealing the underlying trend. The choice of window size $m$ involves a fundamental bias-variance tradeoff: larger windows produce smoother trends but respond slowly to genuine changes in the underlying level, while smaller windows capture more variation but retain noise.

For centered moving averages where $m$ is odd, the smoothed value aligns with the center observation:

$$T_t = \frac{y_{t-2} + 2y_{t-1} + 2y_t + 2y_{t+1} + y_{t+2}}{8}$$

This symmetric weighting recognizes that the central observation should receive less