Foundations of Embodied Intelligence and the Role of World Models

Foundations of Embodied Intelligence and the Role of World Models

An agent that does not move through, touch, or be physically constrained by the world it reasons about is not embodied in the sense this book cares about. Large language models can describe how to drive a car, recognize that a glass is fragile, or write a navigation algorithm; none of these abilities entails that the model has ever been wronged by gravity, surprised by friction, or forced to commit to a single irreversible action. The central premise of embodied intelligence is that an intelligent system must close the loop between its representations and the consequences of its actions in a real (or physically faithful simulated) environment, learning the structure of that environment by acting on it and observing the results. World models are the abstraction that makes this loop computationally tractable in the modern era: they are learned generative simulators of environment dynamics that an agent can query cheaply, roll out in parallel, and use to plan, optimize a policy, or reason counterfactually. This chapter establishes why that abstraction is now central, and how it sits inside the broader formalism of embodied AI.

1.1 What Is Embodied Intelligence?

A useful working definition runs as follows. An embodied agent is a system equipped with a body (or a simulated surrogate thereof) that mediates all of its interactions with a physical or simulated environment. The body comprises sensors that produce a stream of observations and actuators that produce a stream of actions. The agent is situated: it exists at a particular time, at a particular configuration, in an environment whose state evolves in response to its actions and to dynamics outside its control. The agent is autonomous in the sense that, at every control step, it must commit to an action based on whatever internal state it has constructed from its history. And it is goal-directed: there exists some externally specified or internally generated objective whose expected value the agent's behavior improves over time.

Three properties distinguish embodied intelligence from the purely perceptual or linguistic AI that has dominated the deep learning era of the 2010s and early 2020s.

Closed-loop commitment. A language model receives a prompt and produces a response; even when wrapped in a tool-use framework, it does not need to live with the consequences of its outputs. An embodied agent's actions change the state of the world, and that change feeds back into its next observation. The control loop is mandatory and continuous. This has consequences for sample efficiency (you cannot ask the world a question twice with the same state, except in simulation), for safety (irreversible actions have real costs), and for evaluation (you must run the policy forward, not just inspect its logits). It also forces a particular style of representation: the agent's internal state must be a sufficient statistic for predicting the consequences of future actions, not merely a sufficient statistic for predicting the next token.

Physical or simulated embodiment. The body is not a metaphor. It determines what can be observed (a camera does not see through walls; a force-torque sensor does not feel color), what actions are possible (a four-fingered hand cannot type on a standard keyboard without modification), and what dynamics must be respected (Newtonian mechanics, contact, friction, latency, sensor noise). When the body is a simulated robot in MuJoCo, Isaac Sim, or Habitat, the same requirements apply: the simulator enforces the dynamics, and any model of the environment must respect them. The most consequential design choices in embodied AI are usually about the body, not the algorithm.

Action under partial observability. A real embodied agent never sees the full state of its environment. Sensor noise, occlusion, latency, and limited fields of view are intrinsic, not bugs to be engineered away. The agent must therefore reason about hidden state, maintain a belief, and choose actions that reduce uncertainty when doing so is informative. This is in sharp contrast to most reinforcement learning benchmarks, which are built on fully observed Markov decision processes; in such settings, the "agent" is reduced to a reactive policy and the role of representation is suppressed. The literature on partially observed Markov decision processes (POMDPs), which we develop in Section 1.3, exists precisely because embodied AI cannot ignore partial observability.

These properties are not merely philosophical commitments. They explain why the architectural recipes that produced GPT-4, Stable Diffusion, and CLIP do not, on their own, transfer to embodied control. A pretrained vision-language model may have absorbed a vast amount of human knowledge about manipulation, but it has never had to recover from a dropped object, recover from a perturbation, or plan over a horizon longer than a few seconds. The capabilities that embodied agents require—long-horizon credit assignment, robustness to distribution shift caused by the agent's own actions, causal reasoning about physical dynamics, and the ability to learn from a small number of expensive physical interactions—are precisely the capabilities that world models are designed to supply.

It is worth being precise about what "world model" means in this book, because the term is overloaded. We will use it to mean a learned generative model, parameterized by some function approximator (typically a neural network), that captures the agent's uncertainty about how the environment will evolve in response to actions. Concretely, a world model can answer questions of the form: "If I am in latent state $z_t$ and take action $a_t$, what distribution over observations $o_{t+1}$, latent states $z_{t+1}$, and rewards $r_{t+1}$ should I expect?" The model may operate in observation space, in latent space, or in some hybrid; it may be stochastic or deterministic; it may be a single-step predictor or an autoregressive simulator. What unites these choices is the contract: the world model is the agent's learned surrogate for the environment, and every downstream component—policy learning, planning, representation, reward modeling—consumes this surrogate.

We can already see why world models have become so central. They decouple environment interaction from policy improvement. They turn the agent's learning problem into an in-distribution problem, because the policy can be optimized inside the learned model rather than in the true environment. They are reusable across tasks, because the dynamics of a physical environment do not change when the task changes. And they expose a clean interface for reasoning: counterfactual rollouts, planning, and credit assignment all reduce to queries against the same learned simulator. None of this is possible if the agent's only access to environment dynamics is the environment itself.

1.2 A Brief Intellectual History of Internal Models

The idea that an intelligent agent benefits from an internal model of its environment is older than artificial intelligence as a field. In 1948, Edward Tolman reported a series of maze experiments on rats that, on the surface, were learning a sequence of rewarded turns but, in fact, appeared to be constructing what Tolman called a "cognitive map" of the maze: a layout that the rats could re-use when the goal location or the start location was changed. The cognitive map was not directly observable; Tolman inferred its existence from transfer behavior. The modern term for such an inferred representation is a latent state, and Tolman's observation—that animals seem to learn latent state when they could in principle learn a direct stimulus-response mapping—is one of the foundational observations of cognitive science.

Kenneth Craik, in *The Nature of Explanation* (1943), articulated a more general principle: any system