Welcome to the World of Thinking Machines

Welcome to the World of Thinking Machines

Imagine you are standing in your bedroom, and you want to fetch a glass of water from the kitchen. Before you take a single step, your brain is already busy. It recalls the shape of the hallway, the height of the doorframe, the spot on the floor where you once tripped over a stray shoe. It predicts that if you turn left out of your bedroom door, you will bump into the laundry basket; if you turn right, you will pass the stairs and arrive at the kitchen. None of this requires you to physically test the world. You simply *imagine* the consequences of your actions, choose the best path, and then move. The little movie playing inside your head — the silent rehearsal of what might happen next — is something scientists now call a world model. And the fact that this model lives inside a body that can sense, walk, grasp, and learn from contact with the real world is what researchers call embodiment. This book is about teaching machines to do what you just did, and about how studying these machines can teach you something profound about thinking itself.

1.1 What Is Intelligence, and What Is Artificial Intelligence?

To study machines that think, we must first agree on what "thinking" means. The word intelligence is slippery. Psychologists, neuroscientists, philosophers, and computer scientists have argued about it for more than a century, and they have not yet reached consensus. For the purposes of this book, we will use a working definition that is broad enough to include humans, animals, and machines, yet narrow enough to be useful:

> Intelligence is the ability of an agent to perceive its environment, to form internal representations that help it make sense of what it perceives, to make decisions that move it toward its goals, and to improve those decisions through experience.

Four words deserve attention here. Agent simply means any system that acts — a person, a dog, a thermostat, a robot. Environment means everything outside the agent that can affect it or be affected by it. Internal representations are the agent's private symbols, memories, or mental pictures — the things that let it think about the world without directly touching it. Goals are the desired states the agent tries to reach: a hungry bear seeks food, a student seeks a correct answer, a robot seeks a clean floor.

Artificial intelligence, abbreviated AI, is the branch of computer science devoted to building agents whose behavior, when measured against the definition above, would be called intelligent. The founders of the field, who gathered at a famous summer workshop at Dartmouth College in 1956, framed the goal boldly: every aspect of learning, they wrote, "can in principle be so precisely described that a machine can be made to simulate it." Half a century later, that aspiration has produced systems that recognize faces, translate between languages, predict protein structures, drive cars on highways, and defeat world champions at chess and Go. Yet building a machine that walks into an unfamiliar kitchen and fetches a glass of water — a task a four-year-old child performs without ceremony — remains an open and active challenge. The reason, as we shall see, is that doing so requires more than clever algorithms. It requires a body, and a model of the world inside that body.

AI is not a single subject but a family of related fields. The diagram below sketches the territory we will explore.

```mermaid flowchart TD A[Artificial Intelligence] --> B[Symbolic AI] A --> C[Machine Learning] A --> D[Robotics] A --> E[Embodied AI] C --> F[Deep Learning] C --> G[Reinforcement Learning] D --> E E --> H[World Models] E --> I[Simulated Agents] ```

Let us briefly visit each branch:

• **Symbolic AI**, the oldest tradition, treats thinking as the manipulation of explicit symbols according to formal rules. A symbolic AI for the water-fetching task might contain a rule that reads, "If the kitchen is empty, walk to the cupboard." Such systems are powerful for well-defined puzzles but brittle when faced with the messiness of the real world.
• **Machine learning** flips the picture: instead of being told the rules, the system discovers patterns in data. **Deep learning**, a particularly successful form of machine learning, uses layered networks of artificial neurons loosely inspired by the brain.
• **Reinforcement learning** is the subfield in which an agent learns by trial and error, receiving rewards for actions that bring it closer to its goal. It is the training ground for many of today's game-playing champions.
• **Robotics** is the engineering discipline concerned with building physical machines that sense and act in the world.
• **Embodied AI** is the youngest and most ambitious member of the family. It insists that intelligence cannot be understood apart from the body that houses it, and that the most interesting form of intelligence arises when a learning agent is allowed to act, fail, and recover inside a rich environment.

This book sits at the crossroads of the last three branches. We will use the tools of machine learning, we will build robots in simulation, and we will insist on the central premise of embodied AI: the body is not an accessory to thought; it is a partner in it.

1.2 A Brief History of Thinking Machines

The dream of building an artificial mind is ancient. In Greek mythology, Hephaestus, the god of metalwork, forged golden handmaidens who could speak and think; in Jewish folklore, the golem was a clay figure animated by mystical words; in medieval Europe, clockwork monks and mechanical lions astonished cathedral visitors. These early automata were not intelligent in our modern sense — they were elaborate puppets — but they expressed a longing that still drives the field: that matter, properly arranged, might behave like mind.

The modern history of AI begins in the 1940s, when mathematicians such as Alan Turing, John von Neumann, and Claude Shannon laid the theoretical groundwork for computers that store and process information. Turing's 1950 paper *"Computing Machinery and Intelligence"* opened with the now-famous question, "Can machines think?" and proposed the Turing test, in which a human interrogator, typing messages to two hidden partners (one human, one machine), must decide which is which. A machine that cannot be reliably distinguished from a human, Turing suggested, should be considered to think.

In the 1950s and 1960s, AI researchers were bold and optimistic. Programs such as the Logic Theorist (1956) proved mathematical theorems; ELIZA (1966) mimicked a psychotherapist by parroting user statements back as questions; the Shakey the Robot project (1966–1972) at Stanford Research Institute wheeled around a laboratory, reasoning about which objects to push and which doors to open. Shakey is a particularly important ancestor of the systems we will study, because it was the first machine to combine perception, planning, and action in a single loop.

The optimism of those early decades crashed in the 1970s and again in the late 1980s. Researchers discovered that tasks trivial for humans — recognizing a friend's face, understanding a child's sentence, catching a ball — were enormously difficult for computers. Funds dried up, and the field entered periods known as AI winters.

What pulled AI back to life was a slow, deep lesson: intelligence is not just abstract reasoning, it is interaction with the world. Three threads converged to teach this lesson:

1. Behavior-based robotics, championed by researchers such as Rodney Brooks in the late 1980s, argued that intelligence emerges from the bottom up, from simple sensorimotor loops connecting a robot to its environment, rather than from top-down symbolic planning. 2. Embodied cognition, a movement in cognitive science, showed that even human thinking is deeply shaped