Foundations of AI Agent Systems

Foundations of AI Agent Systems

The question at the heart of this book—*what constitutes an intelligent agent and how does it produce autonomous, goal-directed behavior*—has occupied artificial intelligence researchers for over half a century. Yet the question has never been more urgent or more tractable than it is today. The emergence of Large Language Models (LLMs) as general-purpose reasoning engines has fundamentally transformed the landscape of agent design, enabling systems that can perceive diverse information streams, reason through multi-step problems, invoke external tools, and coordinate complex workflows—all while operating with minimal human intervention. Understanding how these capabilities interlock, where current systems succeed and falter, and what architectural choices determine performance across different task categories: these are the concerns that animate the pages ahead.

This chapter establishes the conceptual and historical foundation upon which everything else in this book builds. We begin by tracing the intellectual lineage from early reactive systems to the goal-directed agents of today, identifying the pivotal moments where researchers made architectural decisions that still echo in contemporary frameworks. From there, we articulate the core properties that distinguish intelligent agents—autonomy, reactivity, proactivity, and social ability—and show how these properties manifest in concrete system behaviors. The Perception-Reasoning-Action (PRA) loop emerges as a unifying architectural framework that helps us reason about agent systems at an appropriate level of abstraction. We then classify agent architectures along dimensions of complexity and capability, from simple reflex agents to sophisticated learning-based systems, grounding each category in representative examples. The chapter concludes with a comparative analysis of pioneering systems—SHRDLU from the symbolic AI era and modern frameworks like AutoGPT and LangChain—to illuminate how architectural decisions shape agent behavior, and offers a preview of the four pillars (task planning, memory, tools, and workflow orchestration) that will receive detailed treatment in subsequent chapters.

The Evolution of Agent Architectures

Understanding where AI agents stand today requires appreciating how the field arrived here. The history of agent-oriented AI is not a smooth progression but rather a series of paradigm shifts, each introducing new capabilities while exposing limitations of its predecessors. Tracing this evolution reveals the architectural choices that define contemporary systems and clarifies why certain design patterns have proven more robust than others.

From Symbolic Reasoning to Reactive Control

The earliest AI systems operated on a principle of explicit symbolic manipulation. Researchers in the 1960s and 1970s believed that intelligence could be achieved by representing knowledge as formal symbols and employing logical inference to derive new knowledge or select actions. This approach yielded remarkable achievements in constrained domains—the logic theorist programs that proved mathematical theorems, the general problem solver (GPS) that attempted universal problem-solving through means-ends analysis, and the expert systems that codified human expertise in fields like medical diagnosis and chemical analysis.

The limitations of purely symbolic systems became increasingly apparent as researchers pushed toward more realistic tasks. The frame problem—the difficulty of specifying which aspects of the world remain relevant as situations change—proved stubbornly resistant to solution within purely logical frameworks. Knowledge representation proved far more difficult than anticipated; encoding the casual, temporal, and default reasoning that humans perform effortlessly required increasingly elaborate formalisms that scaled poorly to real-world complexity. Most critically, purely symbolic systems struggled with the gap between knowledge and action. A program might know that a goal is achievable but have no mechanism to actually execute the steps required.

The reactive systems movement of the 1980s emerged as a response to these limitations. Researchers like Rodney Brooks argued that intelligence did not require centralized symbolic reasoning but could instead emerge from direct coupling between perception and action. Brooks's subsumption architecture, which organized behavior as layered finite state machines, demonstrated that complex navigation and obstacle avoidance could be achieved through simple, reactive behaviors without any explicit world model or planning. The "intelligence without representation" and "intelligence without reasoning" slogans captured the spirit of this movement.

What the reactive approach gained in robustness and real-time performance, however, it sacrificed in flexibility. Reactive systems excelled at specific, well-defined tasks in structured environments but struggled when tasks required deliberation, learning from past experience, or adaptation to novel situations. The pendulum swung back toward hybrid approaches that combined reactive competence with deliberative planning, giving rise to the three-layer architectures that became standard in mobile robotics: a reactive layer for immediate response, an executive or sequencing layer for short-term planning, and a deliberative layer for long-term planning and optimization.

The Planning Revolution: From Means-Ends to Hierarchical Task Networks

Parallel to these architectural debates, the field of automated planning evolved through its own series of revolutions. Early planning systems like STRIPS (Stanford Research Institute Problem Solver) represented actions as preconditions and effects, and searched the space of action sequences to find plans that achieved specified goals. The planning problem—given an initial state, a goal state, and a set of available actions, find a sequence of actions that transforms the initial state into the goal—proved computationally intractable in the general case, driving decades of research into more efficient planning algorithms, heuristic search techniques, and expressive action representation languages.

The shift from flat, linear planning to hierarchical task networks (HTN) represented a major advance in practical planning capability. Rather than decomposing goals directly into primitive actions, HTN planning decomposed tasks into subtasks according to predefined task reduction schemas. This hierarchical structure captured the way humans naturally approach complex problems—breaking them into manageable pieces, solving each piece, and composing the solutions. HTN planning remained computationally tractable for larger problems because it constrained the search space while exploiting domain-specific problem-solving knowledge encoded in the task reduction hierarchies.

More recent advances in planning have been driven by the integration of probabilistic reasoning, temporal planning that handles concurrent actions and resource constraints, and most recently, the application of deep learning to planning. Neural planners can learn planning heuristics from experience, generate candidate action sequences that can be refined by classical planning algorithms, or even learn to plan end-to-end from demonstrations. The relationship between neural approaches and classical planning remains an active research frontier; current consensus favors hybrid systems that combine the flexibility of learned components with the reliability guarantees of classical planning where available.

The LLM Inflection Point

The introduction of Large Language Models fundamentally altered the trajectory of agent research. Transformer-based language models trained on massive text corpora demonstrated an unprecedented ability to understand and generate natural language, reason about complex problems presented in text, and transfer capabilities learned during training