Foundations of AI Agents: Definitions, History, and Taxonomy

Why "Agent" Is a Hard Word

The word "agent" appears in software marketing materials, academic papers, conference talks, and casual conversations with startling frequency—and with startlingly different meanings. A consumer might call a chatbot an "AI agent." A researcher might reserve the term for systems that exhibit autonomous goal-directed behavior in an environment. A product manager might describe a workflow automation tool as an "agentic" platform. These overlapping usages are not merely semantic noise; they obscure engineering decisions. If a team cannot agree on what an agent is, they cannot agree on what to build, how to evaluate it, or how to integrate it with existing systems.

This chapter confronts the definition problem head-on. We begin by constructing a formal definition rooted in the perception-action loop, a formulation that has survived more than seven decades of AI research. We then trace the intellectual history that produced our current understanding: from the cybernetic feedback loops of the 1940s, through the symbolic reasoning systems of the 1970s, to the reactive and hybrid architectures of the 1990s, and finally to the LLM-driven agents that dominate today's discourse. With this foundation in place, we introduce a five-level taxonomy—reflex, model-based, goal-based, utility-based, and learning agents—adapted from the canonical treatment by Russell and Norvig and reworked for the engineering audience. We then clarify terminological boundaries that have become practically important: how an "agent" differs from an "assistant," a "copilot," and a "multi-agent system."

The chapter closes with two forward-looking pieces. First, we preview the four architectural pillars—task planning, memory, tool use, and workflow orchestration—that structure the remainder of this book. Second, we present a small case study: a personal scheduling assistant, walked through from problem statement to architectural sketch, which illustrates how the conceptual vocabulary translates into design choices. By the end of the chapter, readers should be able to articulate a precise definition of an AI agent, classify any given system within the established taxonomy, and identify which architectural pillars are most relevant to a particular application domain.

A Formal Definition: Perception, State, and Action

The most enduring definition of an intelligent agent comes from the tradition of cybernetics and control theory and was formalized for AI by Russell and Norvig in their textbook *Artificial Intelligence: A Modern Approach*. Stripped of its pedagogical embellishments, the definition is concise.

An agent is anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators. It does so on behalf of an owner, principal, or stakeholder, mapping a history of percepts to actions through some decision function.

Let us unpack this definition into its constituent parts, because each part constrains how we will design and evaluate agents in the rest of this book.

Perception

Perception is the process by which an agent acquires information about the state of its environment. In a physical robot, perception might consist of camera frames, lidar returns, joint positions, and tactile sensor readings. In a software agent, perception is typically text—the contents of a chat message, the response to an API call, the contents of a file on disk, or a row of a database query. Importantly, perception is partial: the agent does not have direct access to the ground-truth state of its environment. It receives only a stream of observations, each of which is filtered, sampled, and potentially noisy.

The sequence of all percepts an agent has received up to time $t$ is called its percept history, often denoted $h_t$. In mathematical notation, if $p_1, p_2, \ldots, p_t$ are the percepts observed at each time step, then:

$$h_t = (p_1, p_2, \ldots, p_t)$$

The percept history is the agent's only epistemic window into the world. Any internal model the agent builds, any memory it retains, any belief it forms—all are derived from this history.

Action

Action is the means by which the agent influences its environment. For a robot, actions are motor commands that move joints, wheels, or grippers. For a software agent, actions include tool invocations, structured outputs, text generation, database writes, and HTTP requests to external services. The action set $\mathcal{A}$ is the discrete or continuous space of all possible actions the agent can take at a given moment.

It is useful to distinguish between primitive actions, which are atomic at the level of the agent's interface to the world (e.g., "append 'X' to file Y," "send POST to URL U with payload P"), and composite actions, which are sequences of primitive actions organized under a higher-level intent. Much of the architectural complexity of modern LLM agents arises from the need to plan composite actions from primitive capabilities.

The Agent Function

The agent function $f$ is the mapping from percept histories to actions:

$$f: \mathcal{H} \rightarrow \mathcal{A}$$

where $\mathcal{H}$ is the space of all possible percept histories and $\mathcal{A}$ is the action set. The agent function is the abstract specification of behavior. It is the answer to the question: *given what the agent has perceived so far, what should it do next?*

The agent function is a mathematical object, not a software artifact. In practice, the agent function is realized by an agent program, which runs on some physical or virtual platform and produces actions in response to percepts. The distinction matters because different programs can implement the same agent function, and conversely, the same program can be considered the implementation of different agent functions under different environmental assumptions.

For an agent whose environment is fully observable, deterministic, and known, the optimal agent function reduces to a lookup in a precomputed policy or plan. For an agent in a partial-information, stochastic environment, the agent function is typically a probabilistic policy $\pi(a \mid h)$ that gives the probability of each action given the percept history. Modern LLM agents often implement their agent function as a conditional sampling procedure: given the percept history (formatted as a prompt), the language model produces a distribution over next tokens, which is then decoded into an action.

Internal State

In most realistic environments, the percept history grows without bound. An agent cannot simply tabulate $h_t$ and look up an action; it must summarize the history into an internal state $s_t$ that supports decision-making. The internal state is encoded in any persistent mechanism available to the agent program: variables, data structures, a vector database, a working-memory buffer, a learned neural representation. The state-update function is:

$$s_t = \text{update}(s_{t-1}, p_t)$$

where $p_t$ is the percept at time $t$. The agent function can then be written in terms of the internal state rather than the full history:

$$a_t = \pi(s_t)$$

The design of the state-update function is one of the central engineering challenges in agent architecture. It determines what the agent remembers, what it forgets, and how it integrates new information with prior beliefs. We will devote considerable attention to this question in the chapter on memory systems.

Environment and Performance Measure

The definition of an agent is incomplete without a description of the environment in which it operates and a performance measure that evaluates how well it is doing. The environment is characterized along several axes:

• **Observability**: full versus partial. In a fully observable environment, the agent's percepts at each time step contain all information relevant to choosing the next action. In a partially observable envi