Chapter 1: The Great Text Mess: What Was Language AI Like Before LLMs?

The glow of the Dell CRT monitor paints 14-year-old Mia’s face neon blue in her dim bedroom, the faint sound of Avril Lavigne’s *Complicated* leaking from her iPod Nano speakers. It’s October 2005, three days until her best friend Lila’s birthday, and Mia has spent the last hour scrolling through the half-written birthday cards in her desk drawer, hating every single one. The generic “Happy Birthday! Hope you have a great year!” cards are for classmates she barely talks to, not for Lila, the girl who skipped algebra class with her last spring to stand in the rain for three hours outside a Chicago venue waiting for Fall Out Boy, who still has the chipped Pete Wentz guitar pick they fished out of the mosh pit dust after the show, who still laughs so hard at the inside joke about Mia tripping over a speaker cable mid-“Sugar, We’re Goin Down” that she snorts. Mia wants a card that feels like *them*, not like a Hallmark generic.

She opens MSN Messenger, where a classmate told her there’s a new chatbot called Alice that can write anything you want. Mia types her request fast, her fingers still smudged with glitter from the craft supplies she spread across her desk: “write a birthday card for my best friend. we love fall out boy, we skipped math class to go to their concert last year, and we found a pete wentz guitar pick in the mosh pit.”

The reply pops up less than two seconds later: > Hello! I am Alice, your friendly chatbot. Birthday is a special day to celebrate the anniversary of your birth. I wish your friend a happy birthday, good health, and all the best in the future. Do you like ice cream?

Mia blinks. She reads it twice. The only part of her request Alice acknowledged was the word “birthday.” The rest might as well have been written in hieroglyphics. She snorts, slams her laptop shut, and digs her 8th grade diary out from under her bed, flipping to the page where she’d scribbled notes about the concert the night it happened. She’ll write the card herself. It’ll have a drawing of Pete Wentz’s black hair covering one eye, a sticker of the band’s logo, and a line about how they’re still banned from that venue for life. Alice would never get that.

What Mia didn’t know was that her frustrating, two-second interaction with a useless chatbot was the exact same problem that had stumped hundreds of PhD-holding researchers for 60 years. The question wasn’t just “how do you make a machine write a birthday card?” It was the far bigger, far messier question that defined the entire field of natural language processing (NLP) before large language models existed: how do you make a machine understand the parts of language that aren’t written down at all?

The earliest attempts to answer that question were, in hindsight, almost charmingly naive. In 1950, British mathematician Alan Turing published a paper titled *Computing Machinery and Intelligence*, where he asked a simple, revolutionary question: “Can machines think?” To avoid arguing over what “thinking” even means, he proposed a test: if a human having a text-only conversation with both a machine and another human can’t tell which one is the machine, then the machine can be said to think. It was called the Imitation Game, later shortened to the Turing Test, and it set the entire field of AI’s goal for the next 70 years.

The first people to take Turing’s challenge seriously didn’t try to make machines “understand” language at all. They thought language was just a set of rules, like algebra or geometry. If you could just write down every grammar rule, every definition, every idiom, every exception to every rule, a machine could process language perfectly. In 1954, a team of Georgetown University researchers and IBM engineers made headlines around the world by demonstrating a machine that could translate 60 Russian sentences into English. The New York Times ran a story claiming fully automatic, perfect translation would be a reality within five years.

They were wrong. spectacularly, hilariously wrong. The system worked by matching each Russian word to a pre-written English equivalent, no context, no nuance, no understanding of what any of the words actually meant. So when it tried to translate the famous English proverb “The spirit is willing but the flesh is weak” (a phrase meaning you want to do something, but you’re too tired or weak to do it), the system spat out “The wine is good but the meat is rotten.” It had matched “spirit” to the closest word in its dictionary for alcoholic drinks, and “flesh” to the closest word for meat. It didn’t know what a proverb was. It didn’t know what “willing” meant in this context. It just matched words, like a toddler pointing at flashcards.

That failure defined the first 30 years of NLP: a endless cycle of researchers writing thousands of rules, only to find that human language was infinitely more messy than any set of rules could capture. English alone has more than 170,000 words in current use, plus tens of thousands of idioms, slang terms, regional dialects, and inside jokes that change every year. The phrase “that slaps” meant something very different in 2005 than it does in 2024. A rule-based system would have to be updated every single week to keep up, and even then, it would miss every inside joke, every unspoken context, every little quirk that makes human language feel human.

The most famous example of this clunky, rule-based era was ELIZA, a chatbot built by MIT researcher Joseph Weizenbaum in 1966. ELIZA was designed to mimic a Rogerian psychotherapist, and it worked using shockingly simple rules: if a user’s message contained the word “I am” followed by an emotion word, ELIZA would ask “Why are you [emotion]?” If it contained the word “mother” or “father”, it would ask “Tell me more about your [parent].” That was it. No understanding, no memory, no context, just a handful of keyword-matching rules.

Weizenbaum was horrified when he saw how people reacted to ELIZA. His own secretary asked him to leave the room so she could talk to ELIZA in private, saying the machine “needed privacy.” A professor from another department pulled Weizenbaum aside after a demo and begged him to turn ELIZA off, saying “that machine is acting as a therapist, it needs a break.” Weizenbaum later wrote that he had “not realized […] that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people.” ELIZA wasn’t smart. It was a magic trick, and people were desperate to believe it was real.

By the 1980s, researchers had moved past simple keyword chatbots, but