A Robot Opens Its Eyes

A Robot Opens Its Eyes

Why does a machine need a body? On a winter morning in Zurich, a four-legged robot called ANYmal seems to answer this question without words. It stands on the polished concrete floor of a robotics laboratory at ETH Zurich, its mechanical limbs folded beneath it like a sleeping animal. Then, with a faint electric whine, it unfolds. Its head swivels. Its legs extend. And it walks—across a threshold it has never crossed, into a corridor it has never seen, with the careful deliberation of a creature waking up to a strange room. The corridor is full of obstacles it has never encountered: a stack of wooden pallets, a puddle of spilled oil, a piece of construction debris that someone has propped against the wall as a test. ANYmal approaches each one. It stops. It leans forward, as if peering. Then it steps sideways, leans again, and finds another path. To the untrained eye, the motion looks like curiosity. To the engineers watching from behind a glass panel, it is something else: a negotiation between a body, a brain, and a world that does not care whether the negotiator is made of flesh or metal.

This book is about that negotiation. It is about what happens when a machine opens its eyes and discovers that seeing is not the same as recognizing, that walking is not the same as planning, and that intelligence, of whatever kind, may not be possible without the slow, awkward, embodied work of finding one's way through a world that refuses to be reduced to a spreadsheet. The robot's dream, the one we will trace through these pages, is the dream of becoming a participant in the physical world rather than a spectator of it. It is a dream that has flickered through the history of artificial intelligence for almost a century, has been declared dead more than once, and has, in laboratories like the one in Zurich and others like it in Berkeley, Boston, Pittsburgh, and Tokyo, begun to flicker again.

To understand why this dream matters, we have to begin at the beginning—not with robots, but with an old argument about what the mind actually is. For most of the twentieth century, the dominant answer was simple, almost austere: the mind is a symbol processor. It takes in symbols from the outside world, manipulates them according to rules, and produces other symbols. The body, in this picture, is a peripheral device. It happens to be useful for getting symbols in and out, but the actual thinking happens somewhere else, in a kind of pure realm of reason that is identical whether you implement it in neurons, vacuum tubes, or silicon. A computer playing chess could think as deeply as a grandmaster, because chess is a closed world of symbols; the player's hands, the weight of the pieces, the sweat on the brow—all of this was, in the eyes of the symbolists, biographical noise. The mind was elsewhere.

This view, dignified by the awkward acronym GOFAI—"Good Old-Fashioned Artificial Intelligence"—got us surprisingly far. It produced chess programs that could beat world champions. It produced expert systems that could diagnose diseases and configure computers. It produced planners that could schedule the routing of trucks and the synthesis of chemicals. But it ran into a wall whenever the problem stopped being a closed world of symbols and became the messy, continuous, body-shaped problem of the physical world. A robot that could prove theorems about blocks could not, in the 1970s, pick up a block. A vision system that could identify a "chair" by name could not tell whether a particular chair would support a particular person on a particular floor. Something was missing. Many suspected that the missing piece was the body.

Then, in the 2010s, something happened that nobody quite expected. A new generation of AI systems, built on something called deep learning, began to perform feats that looked like perception. They could recognize faces. They could read handwriting. They could caption photographs. They could, eventually, generate images of people who did not exist, hold conversations in plain language, and translate between dozens of languages. The world changed. Companies were founded; fortunes were made; governments issued strategies. And yet, in the quiet spaces between the press releases, a stubborn question lingered: was this perception, or was it something else? Was a neural network that labeled a million photographs of cats truly *seeing* cats, or was it executing an extraordinarily sophisticated form of pattern matching on a database it would never touch, walk beside, or be scratched by?

This is the question that brings us back to ANYmal, and to a robot called Cassie, and to a researcher named Jitendra Malik who has spent four decades trying to answer it. The answer, as we will see, is not that pattern recognition is wrong, but that it is incomplete. It is a piece of something larger—something that requires a body, an environment, and a continuous, embodied conversation between the two. The name we will give to that larger something is the central concept of this book: the *world model*. A world model is not a database. It is not a static map. It is a living, breathing, continuously updated understanding of how the world behaves, built up through the slow accumulation of sensorimotor experience. To build one, a robot—or, for that matter, a child—must do more than look. It must act. It must push, fall, succeed, fail, and try again. It must, in a very real sense, *live* in the world in order to understand it.

This chapter is the doorway into that story. We will meet three characters along the way. The first is Jitendra Malik, a computer scientist at Berkeley who has watched the field swing from symbols to neural networks and back again, and who believes that the next breakthrough will come from finally taking the body seriously. The second is Cassie, a bipedal robot with the long, gangly legs of a baby bird, whose path from simulation to physical competence is one of the most instructive stories of the last decade. The third is you, the reader—standing in for the curious generalist who has heard about AI in the news, perhaps used it to write an email or generate an image, and is now wondering what all the fuss is really about. We will follow these three figures as they circle the same central question from three different angles, and we will end with you stepping, imaginatively, into the perspective of the robot itself. Because the deepest way to understand what perception is, it turns out, may be to ask what it is like to be the thing that opens its eyes.

The Dream of Disembodied Minds

There is a temptation, when reading the history of artificial intelligence, to assume that the field always knew where it was going. It did not. The founding generation of AI researchers, gathered at places like MIT, Stanford, and Carnegie Mellon in the 1950s and 1960s, made a bet. The bet was that intelligence, properly understood, was a kind of formal manipulation of representations. A mind, on this view, was a kind of software. You could in principle run it on any hardware—wet, dry, biological, electronic—provided the hardware was capable of representing and transforming the right kind of symbols. The body was, in the language of one famous proposal, a "vehicle." The mind was the driver. And the driver, in principle, did not care what kind of vehicle it rode in.

This bet produced some of the most ambitious intellectual projects of the twentieth century. Allen Newell and Herbert Simon, working at what would become Carnegie Mellon, built a program called the General Problem Solver that attempted to formalize the act of reasoning itself. Marvin Minsky at MIT gathered a generation of students who would go on to define the field. John McCarthy, also at MIT and then Stanford, coined the term "artificial intelligence" and organized the 1956 Dartmouth conference that is often cited as the field's founding moment. These were not small thinkers, and they were n