The Great Information Hunt: Why Keyword Search Stopped Working for Us

Maya had just finished a brutal 60-hour work week, and all she wanted was to lose herself in a fluffy seaside romance novel—the kind where the lead works at a tiny coastal bank, gets caught in a rainstorm on her way to work, and bumps into a brooding travel writer who’s in town to research “forgotten European fishing villages.” She pulled out her laptop, typed *seaside bank romance novel* into the search bar of her favorite book retailer, and hit enter.

The first result was *The Coastal Ledger: A True Crime Story of Bank Fraud in a British Seaside Town*. The second was a 47-page PDF from the Federal Reserve titled “Regulatory Framework for Coastal Branch Operations.” The third was a 10-second TikTok of a Gen Z influencer showing off her “aesthetic seaside bank job” (she worked as a teller in a Cape Cod credit union, and the video had 2.3 million likes). The fourth result was a romance novel, but the lead character worked as a cleaner at the seaside bank, not a teller, and the plot revolved around solving a cold case, not falling in love. Maya scrolled through three pages of results, sighed, and closed her laptop. “It’s not that hard to understand what I want,” she muttered.

If you’ve ever tried to search for something specific only to get pages of irrelevant results, you’ve run headfirst into the fundamental flaw of keyword search: it can only match the exact words you type, not the meaning behind them. This flaw is not a recent bug, not a problem with lazy engineers, not a temporary glitch in an otherwise perfect system. It is a structural limitation baked into the way we’ve organized and searched for information for 5,000 years. To understand why we need the new technologies we’ll explore in this book, we have to start at the very beginning of the human quest to store and retrieve information.

The First Search Engines: Clay Tablets and Papyrus Scrolls

The earliest known system for organized information retrieval dates back to 1800 BCE, in the kingdom of Babylon under King Hammurabi. The Babylonian state kept tens of thousands of clay tablets recording everything from crop yields to loan agreements to legal rulings, stored in labeled clay jars in the royal archive. A scribe was in charge of the collection, and to find a specific tablet, you had to describe the content to him using the exact keywords written on the jar’s label.

Consider the case of Nabu, a barley merchant who needed to find the record of a loan he’d made to a trader from Ur three years prior. He told the scribe: “I lent barley to the potter from Ur before the flood season three years ago, and I need to collect the principal plus interest.” The scribe had no way to connect “potter” to the trader’s name, “barley” to the loan type, or “three years ago” to the 17th year of Hammurabi’s reign, which was the only time reference on the jar label. He could only look for jars marked with the exact words *barley loan, Year 17, Ur, Etana*—the exact keywords the archive used. If Nabu had used even slightly different wording, the scribe would never have found the tablet. The information would have been as good as lost.

Two centuries later, the Great Library of Alexandria built a far more sophisticated system: a catalog of papyrus scrolls sorted by author and subject. But even this marvel of the ancient world suffered from the same keyword flaw. A scholar looking for a lost fragment of Pythagoras’s philosophical writings on the nature of numbers would find the scroll listed only under “Pythagoras, Mathematics, Scroll A-472.” If he asked the librarian for “any writings by Pythagoras about how numbers are the foundation of the universe,” the librarian would have no way to look it up. The catalog had no summaries, no semantic tags, no way to connect the scholar’s description of the content to the scroll’s actual subject. You had to know the exact classification the library used to find anything.

These early systems were not failures of design—they were the only possible design for an era when every piece of information had to be physically labeled and sorted by hand. But they established a pattern that would repeat for thousands of years: the ability to find information depended entirely on your ability to guess the exact, standardized keywords the system used to label it. If you didn’t speak the system’s language, the information was inaccessible to you.

The Digital Keyword Revolution (and Its Flaws)

The invention of the printing press in the 15th century made information far more widely available, but it did nothing to solve the retrieval problem. If you wanted to find a specific passage in a 500-page book, you still had to know the exact words to look for, or flip through every page one by one. The first real breakthrough came in the 1940s, when researchers began