Foundations of Synthetic Data: Definitions, Taxonomy, and Theoretical Underpinnings

Foundations of Synthetic Data: Definitions, Taxonomy, and Theoretical Underpinnings

The question of what distinguishes synthetic data from other forms of data transformation lies at the heart of contemporary data engineering practice. As organizations increasingly deploy synthetic data for privacy preservation, model training, and system testing, the foundational concepts that define and categorize these approaches become critical for informed decision-making. This chapter establishes the conceptual architecture necessary to navigate the rapidly evolving landscape of synthetic data engineering, addressing the fundamental question: what constitutes synthetic data, how can it be systematically categorized, and what theoretical principles govern its generation and validation?

The significance of these foundational questions extends beyond academic classification. When practitioners select a synthetic data generation approach for a specific application, they implicitly make assumptions about statistical properties, privacy guarantees, and utility preservation. Without a rigorous framework for understanding these assumptions, the risk of misapplication increases substantially. A medical researcher using synthetic patient data for algorithm development, a financial institution employing synthetic transaction records for fraud detection model training, or a software team generating synthetic user behavior for load testing all require clear criteria for evaluating whether their synthetic data adequately serves its intended purpose. This chapter provides that framework.

Defining Synthetic Data: Precision and Boundaries

The Conceptual Boundary

Synthetic data, at its most fundamental level, refers to data that has been algorithmically generated to replicate the statistical properties of an original dataset without containing any actual records from that source. This definition, while seemingly straightforward, requires careful unpacking to distinguish synthetic data from adjacent concepts that often generate confusion in both academic literature and practitioner discourse.

The critical distinguishing characteristic of synthetic data is its generative origin. Unlike augmented data, which modifies existing records through transformations such as rotation, flipping, or noise injection, synthetic data emerges from a generative process that learns and reproduces the underlying data distribution. When a data scientist applies Gaussian noise to numerical features in a dataset, producing what might loosely be termed "synthetic" variations, this represents data augmentation—a fundamentally different operation that preserves direct relationships to the original records. The augmented dataset contains perturbed versions of real data points; the synthetic dataset contains generated data points that conform to the learned distribution.

Consider the distinction through a concrete scenario. A financial institution holds a dataset of 100,000 customer transactions, each containing features such as transaction amount, merchant category, time of day, and geographic location. Data augmentation might produce new records by adding random noise to transaction amounts, swapping merchant categories according to certain rules, or permuting time-location associations. Each augmented record maintains a traceable relationship to some original record in the dataset. In contrast, a synthetic data generator trained on this same dataset produces new records by learning the joint probability distribution over all features and sampling from this learned distribution. The synthetic records bear no direct relationship to any specific original record; they are generated according to the statistical patterns observed in the original data, but they are not derived from or connected to individual source records.

This distinction carries profound implications for privacy analysis. Data augmentation preserves some measure of individual record information in its transformed variants; synthetic data generation, when properly implemented, severs this connection entirely. The privacy implications of these different approaches are fundamentally incommensurable, which is why conflating them leads to significant analytical errors.

Distinguishing Related Concepts

The conceptual boundaries around synthetic data become clearer when examined against several related but distinct data transformation approaches.

Perturbed data maintains direct numerical relationships with source data while introducing controlled noise or randomization. Differential privacy mechanisms, for instance, add calibrated noise to query results or individual records to provide mathematical privacy guarantees. The perturbed value $x'_i$ for a source value $x_i$ might be expressed as $x'_i = x_i + \mathcal{N}(0, \sigma^2)$ for some noise distribution $\mathcal{N}$. While perturbed data shares with synthetic data the goal of obscuring original values, its statistical properties remain anchored to the source distribution; the perturbation is a transformation applied to real data rather than a new sample from an estimated distribution. Synthetic data, in contrast, replaces the original distribution with its estimate.

Simulated data occupies an interesting position relative to synthetic data. Simulation generates data from a known, explicitly specified model—agent-based models of crowd behavior, physics simulations of particle interactions, or economic models of market dynamics. Simulation does not learn from existing data; it implements known relationships to produce data conforming to theoretical expectations. This distinction matters because simulated data verification concerns whether the simulation correctly implements its specified model, while synthetic data verification concerns whether the generated data adequately captures the statistical properties of the source. Synthetic data can incorporate simulation elements (as in simulation-based generation methods), but pure simulation does not qualify as synthetic data generation in the sense this chapter employs.

Federated or pooled data aggregates information from multiple sources without directly sharing raw records. While federated learning can produce model updates that enable synthetic data generation, the aggregated or federated dataset itself is not synthetic—it represents real data from multiple origins, combined through privacy-preserving aggregation mechanisms. Synthetic data generated from federated learning processes is distinct from the federated data that informed its generation.

Anonymized data applies transformation techniques to remove identifying information while preserving statistical utility. Classic anonymization approaches include generalization (replacing specific values with ranges), suppression (removing certain values entirely), and $k$-anonymization (ensuring each record matches at least $k-1$ other records on quasi-identifying attributes). Anonymized data remains fundamentally derived from real records; the transformation aims to prevent re-identification while preserving analytical value. Synthetic data, properly generated, eliminates the connection to original records entirely, achieving what might be termed "derivation privacy"—the impossibility of claiming that any synthetic record corresponds to any specific real record.

The boundaries between these categories are not always crisp in practice. Modern synthetic data generation systems often incorporate elements from multiple approaches: GANs trained with differential privacy guarantees, simulation frameworks that learn parameters from real data, or anonymization pre-processing before synthetic data generation. Such hybrid approaches are legitimate and often valuable, but understanding their component elements is essential for accurate characterization of their properties.

Operational Definition

For the purposes of this book, synthetic data is defined as follows