Introduction to Vector Databases and Semantic Search

Introduction to Vector Databases and Semantic Search

The way we retrieve information has undergone dramatic transformations over the past several decades, yet the fundamental challenge remains unchanged: connecting people with the information they need. When you type a query into a search engine, you expect relevant results to appear within milliseconds. Behind this seemingly simple interaction lies decades of research in information retrieval, each advancement building upon—and sometimes revolutionizing—what came before. We now stand at another inflection point in this evolution, where the traditional approach of matching keywords is giving way to something fundamentally different: semantic understanding through vector representations.

Consider the following scenario: You run a research organization and need to find all documents related to "climate change mitigation strategies for urban environments." A traditional keyword search might return documents containing those exact words, but it would miss highly relevant content discussing "green infrastructure in metropolitan areas," "city-level carbon reduction approaches," or "sustainable urban planning for emissions control." These concepts are semantically equivalent to your query, yet they share virtually no vocabulary. This vocabulary mismatch problem represents one of the fundamental limitations of keyword-based search, and it has motivated the development of the semantic search technologies that form the core of this book.

The paradigm shift toward semantic search is not merely an incremental improvement—it represents a fundamentally different way of thinking about information retrieval. Rather than treating documents and queries as collections of words to be matched, we now represent them as vectors in high-dimensional space, where semantic similarity corresponds to geometric proximity. This seemingly simple change has profound implications: it enables systems to understand that "car" and "automobile" are related despite having different spellings, that "bank" might refer to a financial institution or a river edge depending on context, and that a query about "neural network architectures" should probably return documents discussing "deep learning models" even though those exact words never appear.

This chapter establishes the conceptual foundation for everything that follows. We begin by tracing the historical evolution of information retrieval from its early Boolean systems through the dominant tf-idf and BM25 approaches, examining why these methods succeeded and where they ultimately fall short. We then introduce the core concepts that underpin modern vector search: embeddings, vector spaces, and similarity measures. With this foundation in place, we explore the diverse applications that depend on semantic search, with particular attention to retrieval-augmented generation (RAG) for large language models, which has emerged as one of the most consequential applications of vector database technology. We conclude with an overview of the current vector database ecosystem and a practical exercise that guides you through setting up your first semantic search environment.

The Evolution of Information Retrieval

The Boolean Search Era

The earliest computer-based information retrieval systems, emerging in the 1950s and 1960s, relied almost exclusively on Boolean logic. These systems represented documents as sets of terms and queries as Boolean expressions combining those terms with operators like AND, OR, and NOT. A user searching for information about "machine learning" would construct a query like:

``` machine AND learning ```

The system would then return only documents containing both terms, perhaps further filtered by excluding documents containing the term "learning" in the context of "classical learning" or "educational learning" using additional NOT operators.

Boolean search has the virtue of precision—it returns exactly what it is asked for, nothing more and nothing less. However, this precision is often illusory. The user must know precisely which terms appear in relevant documents, must understand the Boolean operators, and must be able to construct queries that filter out irrelevant results without accidentally excluding relevant ones. In practice, Boolean search suffers from several critical limitations:

Overly restrictive queries: Requiring all query terms to appear in documents can exclude relevant documents that use different vocabulary. A document about "neural networks" would not be returned by a query for "deep learning" even though these topics are closely related.

Overly broad queries: Using OR operators to capture synonyms can return enormous result sets that are difficult to sift through. Searching for "car OR automobile OR vehicle OR auto" might return millions of documents covering everything from automotive manufacturing to vehicle insurance to public transit systems.

Zero results problem: When no document satisfies a complex Boolean expression, users receive no results at all, leaving them with no path forward except to reformulate their query, often without understanding why it failed.

Despite these limitations, Boolean search remains influential today. Many enterprise search systems, library catalogs, and database query interfaces still rely on Boolean logic, and the basic operators (AND, OR, NOT) form the foundation of query languages like those used in SQL databases.

The Rise of tf-idf

The 1970s and 1980s saw the development of statistical approaches to information retrieval that addressed some of Boolean search's limitations. The most influential of these was tf-idf (term frequency-inverse document frequency), a weighting scheme that ranks documents by estimating their relevance to a query based on how frequently query terms appear in each document and how rare those terms are across the entire collection.

The intuition behind tf-idf is elegant: a term that appears frequently in a document is likely important to that document's meaning (high term frequency), but if that same term appears in almost every document in the collection, it is not useful for distinguishing relevant from irrelevant documents (high document frequency reduces importance). Mathematically, the tf-idf score for a term $t$ in document $d$ with respect to a corpus $D$ is:

$$\text{tf-idf}(t, d, D) = \text{tf}(t, d) \times \text{idf}(t, D)$$

Where term frequency $\text{tf}(t, d)$ measures how often term $t$ appears in document $d$, and inverse document frequency $\text{idf}(t, D)$ is typically calculated as:

$$\text{idf}(t, D) = \log\left(\frac{|D|}{|\{d \in D : t \in d\}|}\right)$$

This formulation gives high scores to terms that are frequent in the target document but rare across the corpus. The word "the" would receive a low score because it appears in almost every document, while a distinctive technical term might receive a high score if it appears frequently in one document but rarely elsewhere.

The tf-idf framework enabled several important advances:

Ranking by relevance: Rather than returning a binary set of matching documents, tf-idf allows documents to be ranked by their estimated relevance scores, presenting the most promising results first.

Partial matching: A document containing three of four query terms would receive a higher score than a document containing only one, allowing the system to find partially relevant documents that Boolean search would miss.

Automatic query expansion: The framework naturally supports techniques for expanding queries with related terms, based on their statistical co-occurrence patterns.

However, tf-idf retains fundamental limitations that would become increasingly apparent as information needs grew more sophisticated. The approach treats each document and query as a "bag of words"—an unordered collection of terms—and computes similarity based on exact term overlap. It cannot recognize that "buy" and