What Lives Inside a Robot's Brain?
What Lives Inside a Robot's Brain?
Pip wakes up on the floor of a child's bedroom. There is no one to say good morning. There is only a sock, two Lego bricks, a skateboard propped against the bed, an open backpack spilling crayons, a sleeping cat named Mochi, and a desk on the other side of the room where Pip is supposed to deliver a small package.
Pip is the size of a soccer ball. Pip has two wheels, a camera that points forward, a small laser scanner that pings the room with tiny invisible light beams, and a tiny computer inside its head. Pip does not have arms. Pip does not have a mouth. Pip only has a job: cross the room, avoid everything, and tap the desk gently with its front bumper.
The bedroom looks like a normal bedroom to you. To Pip, the bedroom is a storm. Every fraction of a second, ten thousand numbers arrive inside Pip's computer: how bright each pixel of the camera is, how far the nearest object is in each laser direction, how fast the wheels are spinning, how much battery is left, whether Mochi is moving. From this storm of numbers, Pip must decide: *should I go forward, turn left, turn right, or stop?*
If you were Pip, what would you do?
This chapter is the beginning of a journey through the strange inner life of a robot. We are going to follow Pip from the moment it wakes up on the carpet all the way to the moment it taps the desk. Along the way, we will open up Pip's "brain" and look at what is really inside it. We will discover something surprising. Pip's brain does not just follow a list of orders. Pip's brain contains a small, private *movie* of the bedroom — a movie it can run forward, rewind, and edit, all in its imagination. That private movie is called a world model, and it is the heart of every modern robot that does anything interesting.
By the end of this chapter, you will be able to answer three big questions:
1. Why can't a robot simply "follow instructions" to be smart? 2. What does a world model actually *do* inside a robot's brain? 3. How can you build a tiny paper version of a world model yourself?
Let's wake Pip up and find out.
---
1. Pip's First Day: A Robot Faces a Messy World
The bedroom is seven meters long and four meters wide. The carpet is dark blue. The bed is in the middle of the right wall. The desk is in the far left corner. The door where Pip entered is in the bottom right. Between Pip and the desk, there are at least eight obstacles: a sock, two Lego bricks, the skateboard, the open backpack, a chair pulled out at an angle, a tall lamp on a tripod, Mochi the cat, and a trail of small paper clips that someone dropped last week.
Pip's computer program was written by human engineers in a clean laboratory far away from this bedroom. The engineers tested Pip on a flat square floor with a single cardboard box in the middle. Pip passed every test. Pip could go around the box. Pip could stop in front of the box. Pip could say, in a small beep, "object detected." The engineers were proud.
Now Pip is in the real bedroom, and the cardboard box is not there. The sock is there. The sock is soft and round and changes shape every time Pip nudges it. Mochi is there. Mochi is alive, and Mochi does not stay in one place like a cardboard box. The Lego bricks are there. They are small, shiny, and almost the same color as the carpet, so the camera sometimes cannot see them at all. The lamp is there. The lamp's tripod legs are so thin that the laser scanner can miss them entirely.
Pip turns on its wheels. Pip rolls forward. Within one second, two things happen almost at the same time:
- The left wheel hits a Lego brick. The right wheel is still spinning. Pip spins in a tiny circle and faces the wrong way.
- Mochi, who was sleeping on the backpack, lifts her head and walks slowly toward Pip, because Pip is new and interesting.
Pip is now confused. Pip's program said: "If you see an object, stop." Pip sees many objects. Pip stops. Then Pip's program said: "If you have stopped for one second and the object has not moved, turn left thirty degrees and go forward." Pip turns left. Pip rolls forward and bumps the skateboard with a hollow *thunk*. The skateboard slides three centimeters. Mochi jumps sideways.
You can already see the problem. Pip's list of instructions was written for a clean room with one box in it. The real world is not like that. The real world has *millions* of possible situations, and no human can write down a separate instruction for every one of them.
This is the central mystery of robot brains: how does a machine turn raw signals from its sensors into plans for action?
To answer that question, we need to know the three main ways that humans have tried to give machines intelligence. Each of these ways is a different recipe for filling the space inside a robot's head.
---
2. Three Ways to Make a Robot Smart
In the seventy years since computers became real machines, three big recipes for "artificial intelligence" have been invented. Each one fills Pip's brain with a different kind of stuff. Let's open the hood and look at all three.
2.1 Classical AI: A Robot That Reads a Rulebook
The oldest recipe is classical AI. The idea is simple. A human engineer writes down every rule the robot might ever need, and the robot follows the rules exactly.
```python
A tiny piece of a classical-AI rulebook for a floor robot
if front_distance < 0.20: stop() elif front_distance < 0.50 and an_object_is_recognized_as_a_cat: say("Hello, cat!") wait(2) elif battery < 0.10: return_to_dock() else: drive_forward(speed=0.30) ```
This looks very neat. It is also very fragile. The rule `an_object_is_recognized_as_a_cat` only fires if the camera system has been *taught* what a cat looks like. If Mochi sits in a shadow, or if a fluffy slipper is on the floor, the rule may never fire. If a brand-new obstacle appears that nobody thought to write a rule for — say, a birthday balloon floating two centimeters off the ground — the robot simply has no instruction for it. The robot will bump the balloon and continue driving.
Classical AI is great when the world is small and unchanging. It runs factories, plays chess at grandmaster level, and solves geometry proofs. It is not enough to drive through a bedroom.
2.2 Machine Learning: A Robot That Studies Examples
The second recipe is machine learning. Instead of writing rules by hand, an engineer collects thousands or millions of *examples*. The computer studies the examples and discovers its own rules.
A classic example is teaching a robot to recognize a Lego brick on the floor. The engineer takes ten thousand photos of Lego bricks on different carpets, in different lights, from different angles. The engineer also takes ten thousand photos of carpet *without* Lego bricks. These twenty thousand pictures are fed into a program called a neural network, which is a soft web of tiny mathematical switches. The network adjusts its switches over and over until, given a new photo, it can answer the question *"Is there a Lego brick here?"* with high accuracy.
This is powerful, because the rules the network learns are *not* written by a human. They are patterns that the network discovered from data. A neural network trained on ten thousand Lego pictures can recognize a Lego brick that the engineer has never personally seen.
But pure machine learning has its own limit. Most machine learning is reactive. It turns an input into an output *right now*, without much thought about the past or the future. A neural network that sees a sock might output "slow down," but it does not think: "If I slow down, I will turn left, and if I turn left, in three seconds I will be near the skateboard." It only sees one frame at a time.
A robot that uses only machine learning is a bit like a person who decides every move by looking at the ground right under their feet. They can walk, but they trip on things two steps ahead.