Chapter 1: What Even Is a Prompt? The Secret Life of Large Language Models

Last week, I asked a popular LLM to write a sick day excuse note for me, because I’d spent the entire previous night debugging a broken data pipeline and couldn’t bear to type out a single coherent sentence to my manager. What I got back was a formal letter citing “acute quantum computing-induced cognitive fatigue” as the root cause, complete with a fake citation to a 2022 MIT study that doesn’t exist. My manager didn’t buy it, and I had to spend the next hour explaining that I hadn’t actually been abducted by aliens to work on a top-secret government supercomputer.

If you’ve ever used an LLM, you’ve probably had a similar experience: one minute it’s writing you a perfect birthday card for your mom, the next it’s telling you that the capital of France is London, or that the inventor of the telephone was a 19th-century circus clown named “Bubbles McFluffernutter.” Most people write this off as “AI being dumb,” but the gap between the magical, useful outputs you see online and the garbage you get when you first open ChatGPT isn’t a flaw in the model. It’s a flaw in the prompt.

That’s right: the difference between an LLM that feels like a superpowered assistant and one that feels like a broken parrot comes down to the text you type into the chat box. But if you ask most people what a prompt even is, you’ll get answers ranging from “a magic spell” to “just typing what you want clearly.” Neither is quite right. To master prompt engineering—the skill that lets you unlock capabilities the model never explicitly showed you before—you first need to understand what’s going on under the hood of that chat window. No PhD in machine learning required, I promise. We’re going to use silly analogies, zero jargon (or at least, jargon we’ll explain immediately), and zero math beyond basic addition to break down how LLMs actually work, why prompts matter so much, and what you can do right now to stop getting quantum fatigue excuses from your AI assistant.

The Three-Stage Life of an LLM: From Internet Bookworm to Your Personal Assistant

To understand why prompts work, you first need to understand how LLMs are built. Almost every modern LLM goes through three core stages of development, each of which changes what the model can and can’t do. Let’s use a silly but surprisingly accurate analogy to walk through each: imagine you’re hiring a super-smart intern who has never held a job before, but has read literally every piece of English text ever published on the internet.

Stage 1: Pre-Training (The Intern Reads Everything)

First, you take your intern and lock them in a room with 10 terabytes of text—everything from Shakespeare’s complete works to every Reddit comment ever posted, from peer-reviewed medical journals to the 2012 viral blog post “I Ate 50 Tacos In One Sitting And Here’s What Happened.” Their only job for the next 6 months is to play a game of “guess the next word”: you show them a sentence fragment like “The cat sat on the ”, and they have to guess what word comes next. If they guess “mat,” they get a point; if they guess “toaster,” they lose a point. Over billions of these guesses, they learn the patterns of human language: how grammar works, what facts are commonly associated with which terms, how logical arguments are structured, even how internet slang and sarcasm work.

This is pre-training. The model that comes out of this stage is called a base LLM, and it’s a very smart, very knowledgeable bookworm that can’t actually do any useful work for you. If you ask it “How do I fix a leaky faucet?” it will probably just autocomplete the sentence with the first 10 steps of a WikiHow article it saw during training, even if you’re a renter who can’t touch the pipes. It knows a lot of facts, but it has no idea what you want from it.

> 📌 Sidebar (Digital Edition: hover for expanded explanation; Print Edition: see page 42; Markdown: tagged `#sidebar #pretraining`) > Pre-Training 101 > Modern LLMs are trained on trillions of tokens (individual words or word fragments) of text and code. For example, Meta’s Llama 3 was trained on 15 trillion tokens, while the exact training data size for OpenAI’s GPT-4 is not publicly disclosed. The cost of pre-training a state-of-the-art LLM can exceed $100 million, requiring thousands of specialized GPUs running for weeks or months at a time. That’s why most teams don’t train their own base LLMs: they use pre-trained open-source models like Llama 3 or Mistral, or pay for access to closed-source models like GPT-4 or Claude 3.

Stage 2: Fine-Tuning (The Intern Gets On-the-Job Training)

Next, you give your intern on-the-job training for the specific role you want them to fill. If you want them to be a customer service agent, you feed them 1 million examples of good customer service interactions, and reward them when they respond politely to complaints and give useful solutions, and penalize them when they tell angry customers to “go read the terms of service.” If you want them to be a legal research assistant, you feed them 1 million examples of legal briefs and court rulings, and reward them when they cite relevant case law and structure arguments logically. This process is called fine-tuning, and it aligns the base LLM to follow instructions and perform specific tasks.

The model that comes out of fine-tuning is called an instruction-tuned LLM, and this is the version of the model most people interact with when they open ChatGPT or Claude. It can follow basic instructions, chat politely, and perform general tasks like writing emails or summarizing articles. But it still has big limitations: its knowledge is frozen at the time it was fine-tuned (so it won’t know about events that happened after its training cutoff, unless you fine-tune it again with new data), and it can’t adapt to new, niche tasks without a full, expensive fine-tuning run. If you want your fine-tuned customer service intern to suddenly start writing Python code, you’re out of luck—you’ll have to hire a new intern and train them from scratch.

Stage 3: In-Context Learning (The Intern Learns On the Fly, No Training Required)

This is where prompts come in, and it’s the single most important concept to understand if you want to master prompt engineering. Let’s go back to our intern analogy: what if, instead of spending months and millions of dollars training them for every possible task, you could just show them a few examples of what you want right before you ask them to do it? If you want them to classify customer support tickets as “urgent” or “non-urgent,” you don’t need to fine-tune them: you just show them two examples before you give them a new ticket: “Example 1: ‘My house is flooding and my credit card info was stolen’ = Urgent. Example 2: ‘Where’s my refund for the shirt I returned last week?’ = Non-urgent.” Your intern will immediately be able to classify any new ticket you give them, even if they’ve never seen a customer support ticket before in their life.

This ability to learn new tasks on the fly, without any updates to the model’s internal parameters, is called in-context learning (ICL for short), and it’s the core superpower that makes prompt engineering possible. The “prompt” you type into the chat box is just the set of instructions, context, and examples you give the model to enable this on-the-fly learning. The model doesn’t need to be retrained, fine-tuned, or hacked: it just needs enough clear information to understand what you want.

To make this concrete, let’s map the full LLM development and usage workflow to a simple diagram, which works for all three editions of this book:

```mermaid graph TD A[Raw Text Corpus<br>10TB+ of internet text<br>(books, articles, social media, code)] --> B[Pre-Training<br>Self-supervised next-token prediction<br>~1 trillion parameters trained for months] B --> C[Base LLM<br>Knows language, facts, reasoning<br>Can only autocomplete, no task alignment] C --> D[Fine-Tuning<br>Ta