The Spark of Agency: From Behaviorism to the Birth of an Idea

In the dimly lit laboratories of the early twentieth century, a silent drama was unfolding—one that would eventually illuminate the very essence of agency. Rats navigated intricate mazes, pigeons pecked at colored disks, and dogs salivated at the sound of a bell. To the dominant school of thought, behaviorism, these were not stories of thought or intention but of simple stimulus-response chains, forged and strengthened by the hammer blows of reinforcement and punishment. Yet, within this apparent simplicity, a profound mystery lingered: how do organisms learn from trial and error, without a teacher, to achieve goals in complex environments? This question, deceptively simple, would spawn a century-long odyssey, culminating in the artificial agents that now defeat world champions at Go. Our journey begins not with silicon, but with fur and feather, in the laboratories of Edward Tolman and B.F. Skinner, where the spark of agency was first glimpsed through the lens of animal behavior.

The Behaviorist Crucible: Rats, Reinforcement, and the Shadow of the Stimulus

To understand the birth of the idea of agency, one must first step into the intellectual furnace of behaviorism. By the 1910s and 1920s, psychology was striving to become a “hard” science, shedding the introspective, mentalistic baggage of its past. John B. Watson’s 1913 manifesto declared psychology the science of observable behavior, not consciousness. The mind, with its private thoughts and feelings, was a black box to be ignored. Instead, the focus was on the lawful relationships between environmental stimuli and overt responses. This was a radical simplification: the organism was a conduit, a machine that could be understood entirely from the outside.

The experimental paradigm was elegant in its austerity. In Watson’s famous (though ethically dubious) Little Albert experiment, a baby was conditioned to fear a white rat by pairing it with a loud noise. The stimulus (rat) was paired with an unconditioned stimulus (noise), producing a conditioned response (fear). The story was one of association, of bonds forged through contiguity and repetition. Ivan Pavlov’s dogs salivating at a bell were the canonical example: a neutral stimulus became a conditioned stimulus by predicting a biologically potent one. This was classical conditioning, a passive process where the animal was a spectator to its own learning.

Into this world entered B.F. Skinner, who would push behaviorism to its zenith and, in doing so, create the experimental stage upon which the drama of agency would truly play out. Skinner, at Harvard, rejected even the mild cognitive allowances of some of his contemporaries. He coined the term “radical behaviorism” to emphasize that all behavior—including what we call “thinking” and “feeling”—was subject to the same laws of learning as simple reflexes. His genius was in inventing the operant conditioning chamber, now universally known as the Skinner Box.

The Skinner Box was a masterpiece of experimental control. A hungry pigeon or rat was placed in a small chamber. A disk or lever was within reach. A food hopper could deliver a pellet. The key innovation was the precise, automated scheduling of reinforcement. The animal’s behavior—pecks or presses—was the operant, a behavior that operated on the environment to produce consequences. Skinner meticulously varied the *schedule* of reinforcement: continuous reinforcement (every response rewarded), fixed-ratio (every nth response), variable-ratio (on average every nth, but unpredictable), fixed-interval (first response after a fixed time), variable-interval. The resulting patterns of behavior were striking and law-like. Variable-ratio schedules, for example, produced the highest, most persistent response rates—the very schedule that underpins the addictive pull of gambling machines.

Skinner’s system was beautifully parsimonious. The history of reinforcement—the sequence of rewards and non-rewards—was the sole determinant of future behavior. There was no need to invoke internal states, goals, or representations. The organism was a “blank slate” shaped by its reinforcement history. In his 1938 magnum opus, *The Behavior of Organisms*, Skinner laid out his taxonomy of reflexes and operants, and his experimental methods promised a science of behavior as precise as physics. For a time, it seemed the mystery of learning had been solved: trial and error was simply the accidental production of behaviors, some of which were fortuitously followed by reward, thereby increasing their probability. The agent was nothing more than the sum of its reinforced stimulus-response connections.

But a crack was already forming in this edifice. It came from an unlikely source: a rat in a maze.

Tolman's Rebellion: The Cognitive Map and the Ghost in the Machine

Edward C. Tolman, a psychology professor at the University of California, Berkeley, was no naive mentalist. He was a behaviorist in the sense that he believed in an objective, scientific study of behavior. But he was a “purposive” behaviorist, one who insisted that behavior must be understood in terms of its goal-directedness. To Tolman, the stimulus-response (S-R) bond was an insufficient explanation for the rich, flexible, and apparently insightful behavior of animals.

His primary tool was the maze, a much messier environment than the Skinner Box. A maze presented a spatial problem: find the food at the goal. The classic S-R interpretation would be that the rat learned a chain of responses (turn left at the first intersection, go straight at the second, etc.) through reinforcement at the goal. But Tolman designed experiments that systematically challenged this view.

The most famous is the latent learning experiment, conducted with C.L. Honzik and published in 1930. They had three groups of rats run a complex maze:

• **Group 1**: Received food reward at the goal every trial.
• **Group 2**: Ran the maze for 10 days *without any reward*.
• **Group 3**: Ran for 10 days without reward, then began receiving food on day 11.

The results were telling. Group 1 showed the classic learning curve, rapidly decreasing their errors (wrong turns) over trials. Group 2, during the unrewarded phase, showed no improvement—their error rate remained high and constant. This seemed to support Skinner: no reinforcement, no learning. But the bombshell came on day 11 for Group 3. Upon suddenly receiving food, these rats’ performance *instantly* improved to the level of Group 1. Their error rate plummeted in a single trial. They had, it seemed, learned the maze during the unrewarded phase, but this learning was “latent”—it only became manifest when there was a reason to demonstrate it.

The S-R theory could not easily explain this. If learning required reinforcement, Group 3 should have started from scratch on day 11. Instead, they had a knowledge of the maze waiting to be mobilized. Tolman’s interpretation was revolutionary: the rats had formed a cognitive map—a mental representation of the spatial layout of the maze. During the unrewarded exploration, they were not just milling about; they were acquiring a survey-like understanding of the environment, a “map” in their heads that encoded the relationships between locations. When food was introduced, this map allowed them to quickly determine the shortest path to the goal. They were not executing a memorized sequence of turns; they were using an internal model of the world to plan a route.

Tolman pushed this idea further. He proposed that behavior was not a simple response to a stimulus, but was mediated by intervening variables—internal processes such as motives, expectations, and beliefs. In his 1932 book *Purposive Behavior in Animals and Men*, he argued for a “molar” approach to behavior, looking at the overall pattern of goal-directed action rather than the “molecular” tally of individual responses. The cognitive map was his flagship intervening variable.