Why Your Robot Needs a Brain (and Imagination)

Why Your Robot Needs a Brain (and Imagination)

Picture this. It's a Tuesday evening in a small apartment in Brooklyn. A thin plastic disc about the size of a dinner plate is gliding across the hardwood floor, methodically sucking up dust, dog hair, and the general entropy of human life. Its name is Roomba, and it is doing exactly what it was told to do.

Then it finds the sock.

The sock is blue, crumpled, lying just to the left of the couch where someone stepped out of it an hour ago. The vacuum approaches at its usual confident angle, the small rubber bumper extended forward like a cautious fingertip. Contact. The bumper compresses. A tiny switch closes. A signal travels down a wire. And in the milliseconds that follow, a processor somewhere inside that little plastic disc executes a routine that was written by an engineer in California sometime around 2004.

The routine is unromantic. It goes something like this:

```python if left_bumper_pressed: stop_motors() reverse(20cm) rotate(45_degrees, clockwise) resume_cleaning() ```

The vacuum backs up. It turns. It moves forward. The sock is still there. The bumper compresses again. Another switch closes. The routine runs again. The vacuum backs up, turns, moves forward, and bumps the sock a third time. Then a fourth. Then a fifth. Each time it executes the same three-line program, with the same vigor, the same optimism, the same complete lack of awareness that it has tried exactly this maneuver five times in a row and the world has not changed.

From the kitchen, the dog's owner watches this happen and sighs. "You'd think it would learn."

That single sentence — *you'd think it would learn* — is the starting point of this book. Because the failure of that little vacuum cleaner is not a bug. It is a window into one of the deepest problems in modern robotics. And the solution to that problem turns out to be one of the most fascinating ideas in all of artificial intelligence: the idea that a machine, like a person, needs an internal model of the world in order to act intelligently. It needs, in a word we will examine carefully, *imagination*.

What you are about to read is the first chapter of a book about that idea. The book is called *Worlds Inside Machines*, and its subject is a relatively young corner of AI research often called world models, or sometimes internal models, or sometimes, more poetically, imagination in silicon. We are going to spend the next several hundred pages understanding what these things are, how they work, how people build them, and why they are quietly becoming the most important idea in robotics.

But before any of that, we need to answer a basic question. The question is almost embarrassingly simple, and yet an enormous amount of research, money, and human cleverness has been spent because people kept trying to skip past it:

Why can't we just program the robot?

If we want a vacuum to avoid socks, why not just write a sock-avoiding rule? If we want a self-driving car to stop at red lights, why not just write a red-light-stopping rule? If we want a robot arm to pick up a cup, why not just measure the cup, measure the arm, compute the angles, and write it all down?

This chapter is the answer to that question. And by the time you finish it, you will not only understand why pure rule-following fails for robots; you will also have a working mental picture of what the alternative looks like. That picture has three parts — perception, action, and prediction — and a fourth part that ties them all together, which we will call the world model. By the end, you will see why that fourth part is not optional. It is the heart of any machine that hopes to survive in a world it did not design.

---

The Vacuum Cleaner That Couldn't Imagine

Let's go back to the sock for a moment, because that little disc on the hardwood floor is going to teach us almost everything we need to know about why robots need brains.

The vacuum's software is a beautiful example of what computer scientists call a rule-based system or, less flatteringly, a hand-coded controller. Somewhere inside the device is a small chip. On that chip lives a list of instructions written by humans. The instructions are simple, declarative, and completely unambiguous. They look something like the pseudocode I showed above, only with more cases and many more lines: rules for what to do when the left bumper is pressed, rules for what to do when the right bumper is pressed, rules for what to do when the wheel is stuck, rules for what to do when the battery is low, rules for what to do when it senses a cliff (which is to say, an absence of floor). Each rule is a little if-then statement. Each if-then statement was thought up by an engineer who was sitting at a desk in California, imagining a sock on a floor, and trying to figure out what the robot should do in that situation.

The problem is not that the engineer was bad at their job. The problem is that they were playing a game they could not win.

Let's count, roughly, the things that can happen on a floor. There is the sock, of course. There are shoes, which are sock-shaped but heavier and wider. There are charging cables, which are long and thin. There are pet toys, which come in a bewildering variety of shapes and textures. There are socks that have been stepped on and are now flat, socks that are bunched up, socks that are hanging off the edge of a table with one toe touching the floor. There are curtains. There are tassels from rugs. There are crumbs. There is dog vomit. There is cat vomit, which is worse. There is the edge of a staircase. There is the underside of a radiator. There are table legs arranged in geometric arrays of two, three, four, six. There are chair legs that swivel. There are LEGO bricks, which are small and hard and ruin the vacuum's brushes. There are puddles, which are not visible to the bumper at all but cause the wheels to slip. There are baseboards, which are not obstacles but feel like obstacles to a confused sensor.

Each of these objects deserves, in principle, its own rule. And each rule has to anticipate a different combination of circumstances: sock on hardwood, sock on carpet, sock in shadow, sock under a couch, sock that the dog has just relocated. If we tried to write down all the rules, we would quickly find ourselves writing more rules than there are atoms in the universe. This is called the combinatorial explosion, and it is the first reason why pure rule-following fails. The world contains too many things, and the number of *combinations* of those things grows faster than any team of programmers can keep up with.

But there is a deeper problem, and the vacuum cleaner makes it visible.

When the robot bumped the sock for the fifth time, it had absolutely no idea that it had bumped the sock four times already. The processor dutifully executed the routine. The bumper pressed. The robot reversed. The robot turned. None of these actions left a trace in memory. None of them informed the next decision. As far as the robot was concerned, each encounter with the sock was a fresh encounter, an entirely new event, requiring the same response. This is not a bug. It is a feature — or rather, a property — of how rule-based systems work. They are stateless in the relevant sense: they do not accumulate experience, and they do not build up any picture of the world that persists across moments.

Now compare this to what a person would do. A person, walking through the same room and seeing the vacuum circling around the same sock, would notice. They would see that the sock has been there for a while. They would form an impression: *that sock keeps being there, the vacuum keeps bumping it, the situation is not changing*. They might also form an explanation: *the sock is too tall for the vacuum to climb over, but too short for the bumper to detect from far away, so the robot keeps approachin