Foundations of Time Series Analysis

Foundations of Time Series Analysis

Introduction: Why Time Matters

Every second, organizations around the world make decisions based on predictions about the future. A hospital administrator forecasts emergency room visits to ensure adequate staffing. A retailer predicts holiday sales to optimize inventory. A power company anticipates demand to balance supply and prevent blackouts. An investor estimates market volatility to manage portfolio risk. What unites these diverse scenarios is their dependence on temporal data—observations collected sequentially over time—and the critical need to extract meaningful forecasts from patterns that unfold across temporal dimensions.

Time series analysis represents one of the most practically significant branches of statistical science, yet it presents unique challenges that distinguish it from conventional statistical modeling. When we analyze cross-sectional data—say, the relationship between income and education level across a population—we implicitly assume that observations are independent of one another. The income of one individual does not depend on the income of another. Time series data shatters this comfortable assumption. The value of a stock price today is directly connected to its value yesterday. This month's retail sales inherit patterns from last month's performance. This year's flu cases bear the imprint of last season's outbreak.

This interconnectedness—the fact that temporal observations are not independent but rather autocorrelated—fundamentally reshapes how we must approach data analysis, prediction, and decision-making. Traditional regression techniques, while valuable in many contexts, frequently fail when applied naively to time-dependent phenomena. We require specialized methods that acknowledge and exploit the temporal structure of our data.

This opening chapter establishes the conceptual foundation upon which everything else in this book rests. We will explore what constitutes a time series, examine the various ways we classify temporal data, dissect the components that comprise most time series patterns, understand why temporal dependence creates distinctive analytical challenges, trace the historical evolution of forecasting methodology, and survey the practical applications that make this field so vital. By the end, you will possess a robust mental framework for understanding how the diverse forecasting approaches we will encounter in subsequent chapters relate to one another and to the fundamental characteristics of the problems they address.

Defining Time Series: Core Concepts

What Constitutes a Time Series

A time series is a sequence of data points recorded at successive time intervals. These observations are indexed by time, typically at equally spaced intervals such as hourly, daily, monthly, or yearly, though irregularly spaced time series also occur in certain domains. The defining characteristic is that temporal ordering matters—each observation carries information not just about its own value, but about its relationship to observations that came before and will come after.

Mathematically, we denote a time series as $\{Y_t\}$ where $t = 1, 2, 3, ..., T$ represents the time index. Each $Y_t$ is a random variable that has been observed, meaning we work with the realized values $y_1, y_2, ..., y_T$. The observed sequence constitutes a single realization of the underlying stochastic process that generated it—a crucial distinction that has profound implications for how we approach inference and forecasting.

Consider a concrete example. Table 1.1 presents monthly airline passenger numbers (in thousands) from 1949 to 1960—a dataset that has become iconic in time series analysis.

| Year | Jan | Feb | Mar | Apr | May | Jun | Jul | Aug | Sep | Oct | Nov | Dec | |------|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----|-----| | 1949 | 112 | 118 | 132 | 129 | 121 | 135 | 148 | 148 | 136 | 119 | 104 | 118 | | 1950 | 115 | 126 | 141 | 135 | 125 | 149 | 170 | 170 | 158 | 133 | 114 | 140 | | 1951 | 145 | 150 | 178 | 163 | 172 | 178 | 199 | 199 | 184 | 162 | 146 | 166 | | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |

This dataset exemplifies the kind of temporal data that time series methods are designed to analyze. The ordering of observations—January 1949 followed by February 1949, then March 1949—is not incidental but carries substantive information about the underlying process.

The Realization-Process Distinction

A fundamental concept that beginning practitioners sometimes struggle with is the distinction between a realization and a stochastic process. A stochastic process is the underlying data-generating mechanism—the abstract probabilistic system that produces sequences of observations over time. A realization (or sample path) is the specific sequence of values that we actually observe. We never observe the process itself; we observe only one of its many possible manifestations.

This matters enormously for inference. When we fit a model to historical data, we are learning about the underlying process from a single (or at most a few) realization(s) of it. This is like trying to understand the behavior of a fair coin from a single sequence of coin flips—you can estimate the probability of heads, but you can never be absolutely certain that the coin is fair, and any particular sequence might arise even from a biased coin.

This limitation shapes every aspect of time series methodology, from how we estimate parameters to how we construct prediction intervals. The forecasting methods throughout this book are all grounded in this fundamental epistemic constraint—we are trying to infer the properties of an unobservable process from the visible traces it leaves behind.

Classifying Time Series: Essential Distinctions

Understanding the type of time series you are working with determines which methods are appropriate and which are not. There are several critical distinctions every analyst must master.

Univariate Versus Multivariate Time Series

A univariate time series consists of observations at each time point on a single variable of interest. Monthly unemployment rates, daily stock closing prices, hourly temperature readings—these are all univariate series. The analysis focuses on understanding how this single variable evolves over time, typically with the goal of forecasting its future values.

Multivariate time series involve multiple variables observed simultaneously at each time point. Monthly observations on GDP, inflation, and unemployment together constitute a multivariate series. Retail sales, advertising expenditure, and competitor pricing might be tracked together. The key insight is that variables may influence each other—today's advertising spend affects not only today's sales but potentially tomorrow's as well, and sales figures may themselves influence future advertising decisions.

This cross-variable temporal dependence creates both opportunities and complications. We can exploit correlations between variables to improve forecasts—we might predict sales more accurately by incorporating information about advertising spend than by looking at sales history alone. However, we must also account for the feedback relationships between variables, where Y influences Z, which in turn influences Y, creating complex dynamic relationships.

The ARIMA framework we will encounter later was originally developed for univariate series, though extensions exist for multivariate contexts. Modern deep learning approaches often handle multivariate data more naturally, allowing models to learn complex interdependencies across multiple input series simultaneously.

Discrete Versus Continuous Time Series

Discrete time series are observed at distinct, separated time points. The vast majority of practical forecasting problems involve discrete-time data—daily closing prices, monthly sales