Foundations and Taxonomy of Temporal Prediction Problems

Time series forecasting represents a fundamental pillar of statistical learning and data-driven decision making, yet its precise definition and boundaries remain subjects of frequent confusion in both academic literature and industrial practice. At its core, time series forecasting is the task of predicting future values of a variable based exclusively on its historically observed values and potentially other temporally aligned covariates, under the explicit assumption that the temporal ordering of observations carries essential structural information that must be preserved. This distinguishes it fundamentally from cross-sectional regression, where observations are assumed independent and identically distributed, and from classification tasks where the output space is discrete rather than continuous. The temporal dimension imposes unique constraints: future predictions can only depend on past information (causality), the data-generating process may evolve over time (non-stationarity), and the evaluation must respect temporal ordering to avoid look-ahead bias. Formally, given a sequence of observations $y_1, y_2, \dots, y_T$ observed at times $t_1 < t_2 < \dots < t_T$, forecasting aims to produce $\hat{y}_{T+h} = f(y_1, \dots, y_T)$ for some horizon $h \geq 1$, where the function $f$ maps the historical window to a future value or distribution. This seemingly simple formulation belies profound complexity arising from the diverse structures—trend, seasonality, cycles, abrupt regime shifts—that real-world temporal data exhibit, and from the myriad ways one can frame the prediction problem itself.

A rigorous taxonomy of temporal prediction problems is essential not merely for academic classification but because the choice of modeling approach, evaluation protocol, and even the interpretation of results hinges critically on these problem dimensions. Misidentifying a problem's taxonomy leads to inappropriate model selection, invalid evaluation, and ultimately flawed decisions in high-stakes applications ranging from financial risk management to energy grid operations. The taxonomy can be organized along four primary axes: (1) the number and relationship of variables (univariate vs. multivariate, endogenous vs. exogenous), (2) the data structure (cross-sectional vs. panel vs. time series), (3) the forecasting horizon specification (one-step vs. multi-step, direct vs. recursive), and (4) the evaluation design (fixed origin vs. rolling origin). These axes are not orthogonal; their intersections create a rich landscape of problem types, each with distinct methodological implications. For instance, a multivariate time series with exogenous regressors presents different challenges than a univariate series with complex seasonality, and a multi-step direct forecasting problem on a panel dataset requires specialized architectures that respect both temporal and cross-sectional dependencies.

The distinction between univariate and multivariate forecasting is deceptively simple but carries profound modeling consequences. In univariate forecasting, the objective is to predict future values of a single variable $y_t$ using only its own past: $\hat{y}_{T+h} = f(y_{T:h}, y_{T-h:})$ where $y_{T:h}$ denotes the most recent $h$ observations. Classic models like ARIMA (Autoregressive Integrated Moving Average) and exponential smoothing operate exclusively in this paradigm. However, many applications involve multiple interrelated series where predicting $y_t$ may benefit from past and potentially future values of other variables $\mathbf{x}_t = (x_{1,t}, \dots, x_{K,t})^\top$. This gives rise to multivariate time series forecasting, where the model becomes $\hat{y}_{T+h} = f(\mathbf{y}_{1:T}, \mathbf{X}_{1:T})$, with $\mathbf{y}_{1:T}$ being a matrix of multiple target series. Crucially, the variables can be classified as endogenous (part of the target vector $\mathbf{y}$) or exogenous (external covariates $\mathbf{X}$). The inclusion of exogenous variables, often denoted with an "X" in model names (e.g., ARIMAX, VARX), introduces the possibility of leading indicators—variables that change before the target and thus provide predictive power. However, it also raises the critical issue of future values of exogenous variables: for horizons $h > 0$, one must either have access to their future values (through plans, forecasts, or scenarios) or model them jointly with the targets, as in Vector Autoregression (VAR). The boundary blurs in deep learning, where architectures like Temporal Fusion Transformers (TFT) can automatically learn to attend to relevant historical covariates while handling static metadata, effectively creating a continuum between univariate and multivariate formulations.

Closely related is the distinction between cross-sectional, time series, and panel data structures. Cross-sectional data consists of observations on multiple entities (individuals, firms, countries) at a single point in time, with no inherent ordering. Predicting a cross-sectional outcome is a standard regression problem, albeit one that may require accounting for spatial or network dependencies. Time series data, in contrast, tracks a single entity over time, with the temporal order being paramount. Panel data (or longitudinal data) combines both dimensions: multiple entities observed over multiple time periods. Forecasting with panel data—predicting the future of each entity based on its own history and potentially the histories of other entities—requires models that can pool information across entities while respecting individual temporal dynamics. This has led to the development of hierarchical and grouped time series methods, where forecasts are reconciled across aggregation levels (e.g., product → category → total sales), and to deep learning architectures that employ entity-specific embeddings alongside shared temporal layers. The taxonomy matters because a model designed for independent time series (e.g., standard ARIMA) may underperform on panel data where borrowing strength across entities is beneficial, while a pure cross-sectional model (e.g., random forest with entity fixed effects but no temporal structure) would discard valuable autocorrelation information.

The forecasting horizon specification constitutes another critical axis, often misunderstood outside the forecasting community. The most basic distinction is between one-step-ahead forecasting ($h=1$) and multi-step-ahead forecasting ($h>1$). One-step forecasting, where the model predicts $y_{T+1}$ based on $y_{1:T}$, is relatively straightforward and can use recursive strategies (train a single $h=1$ model and iterate predictions). Multi-step forecasting, however, presents a methodological fork: direct forecasting, where a separate model is trained for each horizon $h$ (i.e., $\hat{y}_{T+h} = f_h(y_{1:T})$); recursive forecasting, where a single one-step model is applied iteratively, feeding its own predictions back as inputs; and multi-output forecasting, where a single model simultaneously predicts all $h$ steps as a vector $\hat{\mathbf{y}}_{T+1:T+h} = f(y_{1:T})$. Each strategy has trade-offs: direct forecasting avoids error accumulation but may produce incoherent forecasts (e.g., trend inconsistencies across horizons) and requires training $h$ models; recursive forecasting is simple but suffers from error propagation and potential model misspecification if the one-step model is not stable under repeated application; multi-output forecasting (common in deep learning with sequence-to-sequence architectures) can learn horizon-specific patterns while enforcing consistency but requires more complex architectures and larger datasets. The horizon specification also interacts with the evaluation design: for multi-step forecasts, one must decide whether to evaluate all horizons equally (e.g., average MSE over $h=1,\dots,H$) or weight them differently, reflecting business costs.

The evaluation design—whether forecasts are gener