Foundations of World Models and Embodied Intelligence

Foundations of World Models and Embodied Intelligence

How can an autonomous system acquire a sufficiently rich and computationally tractable understanding of the world it inhabits, and how should that understanding be coupled to the physical body through which the system perceives and acts? The question is not new. It has reappeared under different banners—cybernetics, optimal control, cognitive science, and most recently deep reinforcement learning—yet its persistent recurrence suggests that it identifies a genuine structural feature of intelligence rather than a fashionable research agenda. This chapter establishes the conceptual scaffolding that the remainder of the book builds upon. We trace the historical trajectory that led from early internal-model control to contemporary deep generative architectures, formalize the agent–environment interaction loop in which any world model must operate, defend the embodied cognition thesis that motivates the sensorimotor framing of intelligence, propose a taxonomy that organizes the design space of world models, and illustrate, with a deliberately simple gridworld example, how internal simulation can decisively alter the data efficiency of policy learning. Readers who complete the chapter should be able to define world models precisely, distinguish them from neighboring constructs such as simulators, forward predictors, and digital twins, and articulate the central arguments in favor of embodied approaches to autonomous systems.

1.1 From Internal Models to Generative Worlds: A Historical Trajectory

The notion that an effective controller must contain an internal model of its environment is older than the field of artificial intelligence itself. A brief genealogy clarifies the contemporary usage of the term *world model* and locates it within a longer intellectual lineage.

1.1.1 Cybernetics and the Birth of Internal Models

The mid-twentieth century witnessed the emergence of cybernetics as a unified language for talking about goal-directed systems, whether biological or artificial. Norbert Wiener's *Cybernetics* (1948) argued that feedback, regulation, and internal representation are the defining features of any system that maintains itself against environmental perturbation. Almost simultaneously, William Grey Walter constructed two electromechanical tortoises, Elmer and Elsie (1948–1949), whose behavior was governed by analog circuits that functioned as rudimentary models of the environment: photocells coupled to capacitive integrators produced internal estimates of light intensity and rotational velocity, and these estimates, rather than raw sensor readings alone, drove steering and recharging decisions. W. Ross Ashby's *Design for a Brain* (1952) and his Homeostat (1948) pushed the same intuition further, demonstrating that adaptive stability required the system to maintain an internal regime whose dynamics mirrored those of relevant environmental disturbances.

What unites these early artifacts is a recognition that an agent cannot respond competently to a novel stimulus merely by reacting to it; it must anticipate. Anticipation, in turn, presupposes some encodable structure—a model—that captures the regularities of the world. The cybernetic tradition was thus the first to insist, on engineering grounds, that *the body of the agent and the model of its world are co-constitutive*: neither can be specified without reference to the other.

1.1.2 The Internal Model Principle and Model-Based Control

The cybernetic intuition was formalized within classical control theory by Francis and Wonham's *internal model principle* (1975–1976). The principle states, in its strongest form, that a feedback regulator can perfectly reject a class of disturbances or perfectly track a class of reference signals only if its dynamics embed a faithful copy of the generating process for those signals. In other words, asymptotic regulation requires *structural* knowledge of the disturbance; no purely reactive controller can achieve zero steady-state error against structured disturbances.

Parallel developments in the 1960s and 1970s produced the mathematical apparatus needed to operationalize this principle. The Kalman filter (1960) gave engineers a principled method for maintaining a probabilistic internal state estimate from noisy observations under a known linear-Gaussian transition model. Model predictive control (Richalet et al., 1978; Cutler and Ramaker, 1979) operationalized a different but complementary idea: rather than precomputing a feedback law, repeatedly solve a finite-horizon optimal control problem against an internal model of the plant, execute the first action, observe the new state, and replan. The MPC paradigm is, in essence, *planning against a learned or identified model of the world on every decision step*. By the early 1980s, model-based control had become the dominant paradigm in process control, aerospace, and robotics precisely because explicit models enabled long-horizon reasoning under constraints that purely reactive policies could not satisfy.

1.1.3 The Cognitive Turn: From Symbols to Sensorimotor Loops

The 1980s and 1990s saw two seemingly opposed currents within artificial intelligence. On one side, the symbolic tradition, exemplified by Newell and Simon's physical symbol system hypothesis (1976), held that intelligence was essentially the manipulation of explicit, language-like representations. On the other side, Rodney Brooks' subsumption architecture (1986) argued that intelligent behavior could—and should—emerge from the interaction of many simple, reactive sensorimotor layers without any central world model. The subsumption manifesto was influential precisely because it called into question the necessity of explicit representation.

Yet the same period produced a countercurrent that ultimately rehabilitated the role of internal models by relocating them within embodied, sensorimotor systems. Varela, Thompson, and Rosch's *The Embodied Mind* (1991) argued that cognition is constituted by the dynamic interplay between brain, body, and environment rather than by amodal symbol manipulation. Andy Clark's *Being There* (1997) and later *Supersizing the Mind* (2008) developed a complementary thesis: that biological cognition exploits structural couplings between the agent's body, its tools, and the environment to offload computation, and that these couplings make internal models more effective precisely because they are grounded in sensorimotor experience. In neuroscience, the predictive coding framework (Rao and Ballard, 1999; Friston, 2005) proposed that the cortex maintains generative models of sensory inputs and refines them by minimizing prediction error, providing a neurobiologically grounded account of how brains simulate the world. By the early 2010s, the pendulum had swung decisively: even anti-representationalist thinkers conceded that some form of internal predictive model is indispensable for flexible behavior, and the question had shifted from *whether* to *how*.

1.1.4 Deep Generative World Models

The deep learning revolution of the 2010s supplied the missing representational machinery. Variational autoencoders (Kingma and Welling, 2014) and generative adversarial networks (Goodfellow et al., 2014) demonstrated that high-dimensional sensory streams could be compressed into compact latent codes and reconstructed with high fidelity. Recurrent state-space models, particularly the recurrent world model of Ha and Schmidhuber (2018), showed that an agent could learn, from raw pixel observations alone, a latent dynamics model capable of supporting policy optimization entirely *inside* simulated rollouts: the agent could dream, in the authors' evocative phrasing, and learn faster from synthetic experience than from real interaction alone.

This result opened an avalanche of subsequent work. The Dreamer line of algorithms (Hafner et al., 2020; 2021; 2023) demonstra