What Even Is an AI Agent?

What Even Is an AI Agent?

If you've spent any time in tech Twitter, LinkedIn, or any conference hallway in 2024–2026, you've heard the word "agent" thrown around like confetti. AI agents are going to replace knowledge workers. AI agents are going to take over the world. AI agents will definitely, absolutely, for sure book your next dentist appointment—probably.

But here's the thing: most of those claims are marketing. The actual definition of an "AI agent" is much more humble, much more practical, and—importantly for you, someone who probably wants to build one—much more useful. Once you understand what an agent actually is, you can stop being confused by the hype and start building systems that genuinely do things in the world.

So let's start with a deceptively simple question: what makes an AI an "agent" rather than just a chatbot? This isn't a philosophical puzzle. It's a practical distinction that determines what you can build, what tools you need, and what your users can expect. Get this wrong, and you'll spend months building a chatbot and wondering why it can't do anything useful. Get it right, and the rest of this book will feel like obvious extensions of a single, clean idea.

A Working Definition (That You Can Actually Use)

Here's the working definition we'll use throughout this book, and it's deliberately unglamorous:

> An AI agent is an LLM running in a control loop, equipped with tools and memory, pursuing a goal.

That's it. Five ingredients, each doing real work:

1. An LLM as the reasoning engine 2. A control loop that lets it act more than once 3. Tools that let it affect the world beyond text 4. Memory of what it has already done 5. A goal that the loop is trying to satisfy

Strip any of these away and you don't have an agent—you have something else, something less interesting. A chatbot that can't call functions is just a chatbot. A function caller that doesn't loop is just an API wrapper. A loop that has no memory is just an expensive random number generator. A goal-less loop is just a script that runs forever.

Let's unpack each ingredient, because each one is doing more work than it looks like.

The LLM: The Brain, Not the Whole Body

The LLM is the part that does the thinking. It reads the situation, makes a judgment, decides what to do next. But—and this is critical—the LLM is *not* the agent. The LLM is a *component* of the agent, like an engine is a component of a car. The engine can produce torque, but a torque producer sitting on a pallet in your garage is not a car. It's a confusing pile of metal that doesn't know where the wheels are.

This distinction matters because it changes how you think about reliability, debugging, and architecture. When your agent does something weird, the question "what did the LLM do?" is the *first* question, not the *only* question. The loop, the tool definitions, the memory, the prompts, the error handling—those all shape the agent's behavior just as much as the model choice. In fact, in production systems, the *non-LLM* code often determines success or failure far more than the model does. We'll return to this insight many times.

The LLM also has a specific job: it produces a structured decision at each step of the loop. That decision might be "call the `get_weather` function with these arguments," or "write this final answer to the user," or "ask the user a clarifying question." The LLM isn't doing the actual work; it's deciding what work to do. The actual work happens elsewhere—in your tools, your APIs, your database. This division of labor is what makes agents tractable to build and reason about.

The Control Loop: Where Agents Stop Being Toys

This is the secret sauce, and it's almost embarrassing how simple it is. A chatbot answers one question and stops. An agent can take action, observe the result, and decide what to do next. That's the loop.

```python while not done: action = llm.decide(state) result = execute(action) state = update(state, result) ```

This is genuinely the whole idea. You take the LLM's output, do something with it, feed the result back in, and ask again. Repeat until the LLM says "I'm done" or you hit some limit (token budget, time limit, step count, sanity).

Why does this matter? Because the real world is full of tasks that can't be solved in a single LLM call. To book a flight, you might need to:

1. Check the user's calendar for available dates 2. Search for flights on those dates 3. Compare prices and layovers 4. Check the user's frequent flyer status 5. Pick a flight 6. Verify the user's credit card on file 7. Confirm and book

That's seven steps, with branching logic and external state. A single LLM call can't do it, no matter how smart the model is. The LLM would have to hallucinate all seven steps, with no way to check any of them. But a *loop* can. The loop is what turns "a very smart text generator" into "a system that can do useful work in the real world."

The loop also introduces something subtle: the agent's *actions have consequences that affect its future decisions*. When the agent searches for flights and gets a list of results, those results become part of the context for the next step. The agent's past constrains its present, which is a fancy way of saying it has a kind of memory, even without a database. This is the foundation of any kind of reasoning, planning, or strategy in agents.

Tools: Hands and Eyes in the Real World

LLMs can only generate text. They can't check the weather, query a database, send an email, or click a button. To do any of those things, they need tools.

A tool, in agent parlance, is just a function that the LLM can choose to invoke. You give the LLM a list of available tools, with descriptions of what they do and what arguments they take. The LLM's job is to output a structured request—"I want to call `get_weather` with `location='San Francisco'`"—and your code handles the actual execution.

This is the part that bridges the gap between "talking about doing" and "actually doing." Without tools, even the smartest LLM in the world is just a very eloquent prisoner, locked in a box that only generates text. With tools, it can do things that affect the world—read data, write data, take actions, observe outcomes. The box now has hands, eyes, and a way to call the outside world.

The design of your tools is one of the most consequential decisions you'll make as an agent builder. A tool with a confusing name or unclear description will be misused. A tool that returns too much data will overwhelm the LLM's context window. A tool that fails silently will lead the agent down dead ends. We'll spend significant time on tool design later in the book, but for now, the key insight is this: tools are your agent's interface to reality, and good interfaces make everything else easier.

Memory: The Difference Between Amnesia and Strategy

Memory in an agent comes in two flavors, and you almost always need both:

Short-term memory is the conversation history within a single run. The agent remembers what it has done, what tools it has called, what results it got. Without this, every step of the loop would be the first step—the agent would have no idea what it just tried or what happened. Short-term memory is what lets the agent say "I just searched for flights, got ten results, and now I need to filter them by price." It lives in the messages list, the context window, the call stack of the run.

Long-term memory is persistent storage across runs. This might be a vector database of past interactions, a user profile, a knowledge base, a JSON file, or even a SQL database. Long-term memory is what lets an agent get better over time, or at least remember what it learned yesterday. It's what makes an agent *yours* rather than a generic instance that resets every time you call it.

Most production agents start with just short-term memory and add long-term memory as