Why World Models? The Cognitive and Computational Case for Embodied Simulation

Chapter 1 — Why World Models? The Cognitive and Computational Case for Embodied Simulation

1.1 The Central Question and Why It Refuses to Die

Should an embodied agent maintain an explicit, learnable model of its environment, or should it instead refine a reactive policy that maps observations directly to actions? This question has been debated under different vocabularies for more than seventy years. In control theory it appears as the choice between model-predictive and model-free control. In classical AI it is the debate between deliberative planners like STRIPS and reactive architectures like subsumption. In modern reinforcement learning it recurs as the trade-off between model-based and model-free methods. In cognitive science it is the perennial tension between perception as inference and perception as direct pickup.

The fact that the question keeps returning is itself evidence that neither pole has won. Reactive policies win on simplicity and often on wall-clock latency. Explicit models win on transfer, on safety, and on data efficiency. This book is written from the conviction that, for the next generation of embodied systems, the pendulum has decisively swung toward the model side — but only because the models themselves have changed. Generative neural networks have made it possible to learn rich, multimodal world representations directly from high-dimensional sensor streams, something classical symbolic or low-dimensional Kalman-filter approaches could never achieve. The chapter you are reading sets the stage for that claim. It explains where the idea of a world model comes from, why modern incarnations are different from their predecessors, and how to reason about when an explicit model is worth its engineering cost.

We proceed in five steps. Section 1.2 traces the historical lineage from Helmholtz through cybernetics, classical AI planning, and the reactive counter-revolution to the deep generative renaissance. Section 1.3 builds the contemporary case along three lenses — sample efficiency, generalization, and safety — and grounds each lens in concrete results from systems like DreamerV3, GAIA-1, RT-2, and Habitat. Section 1.4 surveys the spectrum of embodiment types and shows why each induces different inductive biases for world-model design. Section 1.5 introduces the granularity taxonomy that will recur throughout the book. Section 1.6 lays out the roadmap and shows, through a single conceptual map, how the remaining fourteen chapters hang together. The chapter closes by motivating the mathematical prerequisites of Chapter 2.

1.2 The Long Lineage: From Unconscious Inference to Generative Worlds

1.2.1 Perception as Construction

Hermann von Helmholtz, writing in the *Handbook of Physiological Optics* (1867), argued that perception is not the passive registration of sensory impressions but an active construction. The retinal image is ambiguous — infinitely many three-dimensional scenes project to the same two-dimensional array — and the visual system resolves this ambiguity by combining current signals with prior expectations about how the world behaves. Helmholtz called this process *unconscious inference*. The cognitive content of the theory is the idea that the brain carries implicit hypotheses about scene geometry, material properties, and motion regularities, and uses them to interpret noisy, partial observations.

A century later, this insight was formalized in the language of Bayesian inference. The brain, on this account, maintains an approximate posterior over latent scene variables $z$ given observations $o$, in the form $p(z \mid o) \propto p(o \mid z)\, p(z)$. The prior $p(z)$ is not arbitrary; it is shaped by evolution and by the agent's lifetime experience of acting in similar scenes. World models in modern robotics inherit this picture almost verbatim: they parameterize a generative model $p_\theta(s' \mid s, a)$ and a recognition or encoder $q_\phi(z \mid o)$, and train them jointly so that the latent state becomes a sufficient statistic for predicting the consequences of action.

Jean Piaget, working in parallel, reached a complementary conclusion from the developmental side. His *sensorimotor schemas* describe the way an infant progressively refines its expectations about how the world responds to its actions. A schema, in Piaget's sense, is a generalization such as *grasping* or *pushing* that bundles a class of actions with their typical outcomes. As the schema is exercised, the child revises its expectation of the outcome distribution. This is exactly the loop that a learned world model implements: actions are executed, observations are gathered, and the predictive model $p_\theta$ is updated to reduce the discrepancy between predicted and observed transitions.

Nikolai Bernstein added a third pillar with his analysis of motor coordination. The *degrees-of-freedom problem*, in his formulation, is that every goal-directed movement admits infinitely many joint trajectories. The central nervous system must select one of these trajectories on the basis of predicted consequences — predicted arm dynamics, predicted contact forces, predicted object motion. Bernstein's *feed-forward model* of motor control, later refined into the *internal model* principle, asserts that accurate movement requires an internal forward simulator that can be queried before committing to an action. This is the same role a world model plays in a model-predictive controller, except that modern learned world models generalize the simulator from rigid-body mechanics to arbitrarily complex visual and physical scenes.

1.2.2 The Good Regulator Theorem

In 1970, Roger Conant and W. Ross Ashby proved a theorem that, despite its simplicity, has shaped decades of work:

> Every good regulator of a system must be a model of that system.

The theorem is technical — it uses perturbation analysis around a fixed operating point — but its conceptual content is unambiguous. If a controller is to regulate a plant to a desired regime, it must, at least implicitly, embody the plant's input–output structure. There is no escape from the model; the only question is whether the model is explicit and inspectable or whether it is smeared across the parameters of a reactive policy.

This theorem is the philosophical backbone of every argument in this book. A model-free policy that succeeds on a benchmark is, in Conant and Ashby's sense, a regulator whose model is hidden inside its weights. The agent of this book is not whether the model exists but whether we want to see it, edit it, reason about it, and transfer it.

1.2.3 Cybernetics, Internal-Model Principle, and Classical Planning

Wiener's cybernetics added the language of feedback and circular causality that made the regulator theorem operational. The control-theoretic literature then refined the picture further. A controller that compensates for a known disturbance must contain a model of how the disturbance was generated; this is the *internal model principle*, formalized by Francis and Wonham in the 1970s and reviewed by Smith in 2006. In its modern form the principle says: for any reference signal or disturbance whose generator lies in a specified class, there exists a regulator that achieves zero steady-state error *if and only if* the regulator contains a replica of that generator.

Robotics inherited this principle in the form of model-predictive control. Given a dynamics model $f(s, a) \to s'$, an MPC controller solves, at each time step,

$$ \arg\max_{a_{t:t+H}} \sum_{k=0}^{H-1} r(s_{t+k}, a_{t+k}) \quad \text{subject to } s_{t+k+1} = f(s_{t+k}, a_{t+k}), $$

executes the first action, observes the new state, and replans. The whole apparatus stands or falls with the quality of $f$.

In AI, the parallel line of work was the *planning* tradition. STRIPS (Fikes and Nilsson, 1971) represented the world as a set of ground literals and actions as operators with preconditions, add lists, a