Evolution of AI Agent Paradigms: From Symbolic Systems to LLM-Native Architectures

The core essence of an AI Agent, across all historical paradigms, lies in its closed-loop capability of *perception-decision-action*: it perceives the state of its environment, makes decisions based on internal models, and executes actions to modify the environment, ultimately achieving predefined or self-generated goals. This closed-loop definition, first formalized by Wooldridge and Jennings (1995) in the foundational work on agent-oriented software engineering[^3], remains the north star for evaluating the evolution of Agent architectures. Over the past seven decades, the design of AI Agents has undergone three paradigm shifts, each driven by breakthroughs in underlying computational models and responses to the limitations of prior approaches: from the symbolic logic-based agents of the mid-20th century, to reinforcement learning (RL)-driven autonomous agents of the late 20th century, to the current generation of LLM-native generative agents. This chapter traces this evolutionary trajectory, formalizes the core bottlenecks of legacy paradigms in open-world, ambiguous task scenarios, establishes the foundational design principles for modern modular Agent architectures, and introduces a capability taxonomy that frames the technical scope of subsequent chapters.

---

1.1 The Symbolic Agent Era: Logic-Driven Closed-World Reasoning (1950s–1990s)

The first wave of AI Agent research emerged directly from the *Physical Symbol System Hypothesis* proposed by Newell and Simon (1976)[^4], which posits that any system capable of manipulating physical symbols to represent problems and generate solutions is sufficient for general intelligence. This assumption became the foundational bedrock of symbolic Agents, which treat environmental perception, reasoning, and action as processes of symbol manipulation.

1.1.1 Core Architecture and Representative Systems

Symbolic Agents follow a strict pipeline of *symbol grounding → logical reasoning → action execution*. The most iconic system of this era is Shakey the Robot, developed by the Stanford Research Institute between 1966 and 1972, widely recognized as the first general-purpose mobile AI Agent[^1]. Shakey’s architecture consists of three tightly coupled modules: 1. Perception Module: Converts raw sensor data from laser rangefinders and monochrome cameras into discrete symbolic representations of the environment, e.g., `(room lab) (room hallway) (door d1 connects lab and hallway) (box b1 in hallway)`. 2. Reasoning Module: Uses the STRIPS (Stanford Research Institute Problem Solver) planning algorithm, based on first-order logic, to generate action sequences that satisfy task goals. For a task like "fetch the box from the hallway and push it to the target location", STRIPS decomposes the goal into preconditions and effects, then generates a valid sequence of actions: `(move to hallway) → (grasp b1) → (push b1 to target)`. 3. Action Module: Converts symbolic action commands into low-level control signals for wheels and robotic arms to execute the plan.

The symbolic paradigm also gave rise to the influential BDI (Belief-Desire-Intention) architecture proposed by Bratman (1987)[^2], which formalizes Agent decision-making as a process of maintaining beliefs about the environment, generating desires (potential goals), and committing to intentions (executable plans). BDI-based frameworks like JADE (Java Agent Development Framework) remained the mainstream of Agent engineering through the 2000s, and are still used in closed industrial scenarios like manufacturing control and telecom network management.

1.1.2 Core Limitations: The Symbol Grounding and Combinatorial Explosion Bottlenecks

While symbolic Agents demonstrated rigorous logical reasoning capabilities in constrained environments, they face two fatal bottlenecks when applied to open-world, ambiguous tasks: 1. Symbol Grounding Problem: The process of converting continuous, fuzzy real-world perceptual signals (e.g., visual appearance of a "red cup", the concept of "delicious food") into discrete, precise symbols is fundamentally ill-posed. For example, the spectral range of "red" spans hundreds of nanometers, and the shape and size of "cups" vary wildly across contexts; manual definition of all possible symbol instances is impossible. A 1988 study of expert systems (the most successful application of symbolic AI) found that 90% of projects failed due to the prohibitive cost of knowledge engineering: maintaining a rule base for a single medical diagnosis task required over 100 person-years of expert labeling, and the rule base could not generalize to unseen cases[^5]. 2. Combinatorial Explosion in Open Worlds: The reasoning complexity of first-order logic grows exponentially with the number of entities and rules in the knowledge base. In a closed lab environment like Shakey’s, the knowledge base contained only a few hundred entities and rules, so planning could be completed in seconds. But in an open environment like a city, with millions of entities (buildings, people, vehicles) and dynamic rules (traffic regulations, store opening hours), the number of possible action sequences exceeds $10^{100}$, making exhaustive planning computationally infeasible.

```mermaid graph LR A[Raw Sensor Data] --> B[Symbol Grounding Module] B --> C[First-Order Logic Knowledge Base] C --> D[STRIPS Planner] D --> E[Symbolic Action Sequence] E --> F[Low-Level Actuator Control] style C fill:#f9f,stroke:#333,stroke-width:2px ``` *Figure 1.1: Pipeline of a typical symbolic Agent (e.g., Shakey the Robot)*

---

1.2 The RL-Driven Autonomous Agent Era: Reward Maximization in Closed Environments (1990s–2010s)

The limitations of symbolic Agents led researchers to question the necessity of explicit symbolic representations, giving rise to the behaviorist paradigm of AI: intelligence emerges from the interaction between an Agent and its environment, with the sole objective of maximizing cumulative reward. This paradigm is formalized by the Markov Decision Process (MDP) framework, which defines an Agent’s behavior as a policy $\pi(a|s)$ that maps environmental states $s \in S$ to actions $a \in A$, with the goal of maximizing the expected cumulative discounted reward:

$$J(\pi) = \mathbb{E}_{\pi} \left[ \sum_{t=0}^{T} \gamma^t R(s_t, a_t) \right]$$

Where $R(s_t, a_t)$ is the immediate reward received after taking action $a_t$ in state $s_t$, $\gamma \in [0,1]$ is the discount factor, and $T$ is the task horizon.

1.2.1 Core Architecture and Representative Systems

RL-driven Agents do not require pre-defined symbolic rules or knowledge bases; instead, they learn optimal policies through trial-and-error interaction with the environment. The breakthrough of deep reinforcement learning (DRL) in the 2010s enabled these Agents to handle high-dimensional perceptual inputs (e.g., images, sensor data) directly.

• **Game Agents**: DeepMind’s DQN (Deep Q-Network) achieved superhuman performance on 49 Atari 2600 games in 2015, using only raw pixel inputs and game scores as reward signals[^6]. Later, AlphaGo and AlphaZero used policy gradient and Monte Carlo Tree Search (MCTS) to defeat world champions in Go, chess, and shogi, without any human domain knowledge.
• **Physical Control Agents**: Boston Dynamics’ Atlas robot uses model-based RL to learn walking, running, and parkour skills in simulation, then transfers the policy to real-world hardware with minimal fine-tuning[^7].
• **Early Autonomous Driving Agents**: Waymo’s early decision-making modules used RL to learn lane changing, merging, and intersection handling in closed simulation environments.

1.2.2 Core Limitations: Sample Inefficiency and Open-World Generalization Failure

While RL-driven Agents achieved remarkable performance in closed, well-defined environments, they face three critical bottlenecks that prevent deployment in open-world real-world scenarios: 1.