The Emergence of World Models: From Internal Representations to Embodied Intelligence

The Emergence of World Models: From Internal Representations to Embodied Intelligence

The question of how artificial agents should represent and reason about the world they inhabit sits at the heart of contemporary embodied artificial intelligence. For decades, the dominant paradigm treated perception as a standalone problem: extract features from sensory streams, map them to actions, and iterate. This approach yielded impressive results in constrained settings—controlled factory floors, well-lit image classification benchmarks,干干净净 board games with perfect information. Yet when agents venture into the messy, dynamic, partially observable reality that characterizes genuine embodiment, perception-only architectures falter. They cannot anticipate consequences, reason about causality, or adapt to novel situations because they possess no internal representation of *how the world works*. World models emerged as the conceptual and computational solution to this fundamental limitation. They provide embodied agents with compressed, generative representations of environmental dynamics that enable simulation, prediction, and planning beyond the immediate sensory horizon.

This chapter establishes the theoretical and historical foundation for understanding world models as the cornerstone of embodied intelligence. We examine why rich internal representations prove essential for agents that must act effectively in complex, non-stationary environments. Drawing connections to cognitive science, we explore how the human mind constructs and deploys mental models, and how principles such as predictive coding and the free energy principle formalize these processes in computational terms. We then articulate a precise definition and taxonomy of world models within artificial intelligence, distinguishing them from related but distinct concepts such as state estimators and environment simulators. Through a detailed case study comparing reactive and model-based control in robotic manipulation, we demonstrate concretely how world models transform brittle, situation-specific behaviors into robust, generalizable intelligence. The chapter concludes by previewing the architectural and algorithmic landscape that subsequent chapters will traverse in depth.

Historical Progression: From Reactive Reflexes to Model-Based Intelligence

The evolution of robotic and artificial intelligence systems traces a compelling arc from pure reactivity toward increasingly sophisticated internal representation. Understanding this trajectory illuminates both the motivations for world models and the specific gaps they were designed to fill.

The Reactive Paradigm and Its Limitations

Early robotic systems operated under the reactive paradigm, popularized in the 1980s through Rodney Brooks's subsumption architecture and related behavior-based approaches. The foundational intuition was elegant: biological organisms do not appear to perform extensive internal simulation before acting; their behaviors emerge from direct sensorimotor coupling with the environment. Brooks famously argued against traditional sense-model-plan-act architectures, demonstrating that simple finite-state machines layered in hierarchical structures could produce surprisingly robust locomotion and navigation behaviors without any internal world representation.

The reactive approach excels in structured environments where the mapping from percepts to actions can be directly encoded. A robot vacuum navigating a single floor of an office building encounters a relatively constrained distribution of situations. Obstacles appear, surfaces change, but the fundamental regularities—walls define boundaries, furniture creates obstacles, drop-offs require avoidance—remain stable. In such settings, reactive controllers can achieve acceptable performance by learning or programming appropriate stimulus-response associations.

However, reactive systems exhibit three fundamental brittlenesses when deployed in more demanding contexts. First, they lack temporal extension: a reactive system responds only to immediate sensory input, unable to anticipate consequences that unfold over multiple timesteps. Second, they cannot generalize to novel situations: a reactive controller trained on red balls cannot handle blue balls without explicit retraining, because it has captured only surface statistics rather than underlying causal structure. Third, they cannot represent unobservable state: the true state of the world often exceeds what sensors directly reveal, and reactive mappings from incomplete observations to actions inevitably make errors.

Consider a robot attempting to pour liquid from one container into another. A purely reactive controller might detect the proximity of the two containers and initiate a pouring motion, but without a model of liquid dynamics—gravity, flow rate, container geometry—it cannot regulate pouring speed based on fill level, cannot detect when the source container is empty, and cannot compensate if the target container is tilted. Each failure mode requires explicit perceptual triggers and pre-programmed responses, leading to combinatorial explosion as situations multiply.

The Rise of Model-Based Control

The recognition of these limitations drove renewed interest in model-based control, wherein agents maintain explicit representations of environmental dynamics and use them to reason about action outcomes. Early model-based approaches in robotics focused on physics-based models: rigid body dynamics, contact mechanics, fluid dynamics. Industrial robot arms had been controlled for decades using computed torque control, wherein a detailed mechanical model of the arm enables precise trajectory tracking. Yet these models were hand-crafted, brittle, and required extensive calibration for each specific robot and environment.

The machine learning revolution of the 2000s and 2010s introduced data-driven approaches to model acquisition. Rather than manually specifying dynamics models, researchers developed methods for learning them from observations. System identification techniques learned parametric models of robot dynamics from executed trajectories. Gaussian Process Motion Planning learned continuous-valued dynamics models and used them for real-time trajectory optimization. Model Predictive Control (MPC) frameworks reframed control as online optimization over predicted futures, using learned models to forecast consequences of candidate action sequences.

These developments established the conceptual infrastructure for world models: internal representations that agents could query to simulate outcomes, compare alternatives, and plan courses of action. Yet early model-based systems remained narrowly focused on low-level dynamics—the physics of motion—rather than the broader semantic and causal structure of the environment.

The World Model Renaissance

The contemporary interest in world models extends far beyond dynamics prediction. Inspired by cognitive science, neuroscience, and generative modeling advances in deep learning, researchers now conceive of world models as rich, hierarchical representations encompassing multiple modalities, temporal scales, and abstraction levels.

The seminal "World Models" paper by Ha and Schmidhuber (2018) demonstrated that agents could learn compressed latent representations of game environments and use them for fast policy learning. By training variational autoencoders to compress visual observations and recurrent networks to predict future latent states, they created agents that could simulate and plan within learned representations. This work catalyzed intense research interest, demonstrating that world models were not merely theoretical constructs but practical computational objects amenable to deep learning approaches.

Subsequent advances extended world models in several critical directions. Dreamer and related algorithms showed tha