Introduction to Database Systems

Introduction to Database Systems

The modern world runs on data. Every time you check your bank balance, search for a product online, post a status on social media, or book a flight, you are interacting with a database. Databases quietly underpin virtually every significant application in our digital lives, from the most simple mobile applications to the most complex enterprise systems handling millions of transactions per second. Yet despite their omnipresence, the inner workings of database systems remain mysterious to many software developers and engineers who use them daily. This book aims to demystify these fascinating systems, taking you from the fundamental concepts all the way to the cutting-edge technologies that power today's most sophisticated data management solutions.

Understanding database systems is not merely an academic exercise. As a software engineer, database knowledge directly impacts your ability to design scalable applications, optimize performance, troubleshoot issues, and make informed architectural decisions. Whether you are building a small web application or architecting a distributed system that serves billions of users, the principles you will learn in this book will provide the foundation for making those decisions wisely. The database is often the heart of any information system, and understanding how it functions will make you a more effective and well-rounded engineer.

This opening chapter establishes the conceptual foundation upon which everything else in this book will build. We begin by exploring why database systems exist in the first place—a question that might seem obvious until you truly consider the problems they solve. We trace the historical evolution from simple file-based storage to the sophisticated database management systems of today, understanding not just what exists but why it evolved that way. We then examine the crucial concept of data abstraction, particularly the three-level architecture that forms the theoretical backbone of virtually all modern database systems. Finally, we survey the core components that make up a database system and understand how they work together to provide the services we depend on.

The Evolution of Data Storage

To appreciate modern database systems, we must first understand the journey that brought us here. The story of data storage is one of gradual evolution, driven by the ever-increasing demands of applications and users. Each step in this evolution was motivated by the limitations of previous approaches, and understanding these motivations helps us grasp why database systems are designed the way they are.

The Era of File-Based Storage

In the early days of computing, data was stored in flat files on magnetic tapes and later on disk drives. A file, in this context, is a named collection of related records stored on secondary storage. These files were typically managed by application programs directly, with each application responsible for defining its own file structures, accessing methods, and data organization. This approach, known as file-based processing, dominated early computing from the 1950s through the 1960s.

Consider a simple example of file-based storage for a business application managing customer information. The application might store customer data in a file called `CUSTOMERS.DAT`, with each record containing fields for customer name, address, phone number, and account balance. The application program would contain code to read records from this file, write new records, update existing ones, and delete records when necessary. The file format—how records were organized, how fields were delimited, and how the file was structured—was entirely determined by the application program.

```c // Example of file-based data access in legacy code #include <stdio.h> #include <string.h>

#define MAX_NAME_LEN 50 #define MAX_ADDR_LEN 100

struct Customer { int customer_id; char name[MAX_NAME_LEN]; char address[MAX_ADDR_ADDR_LEN]; char phone[20]; float balance; };

// Function to add a new customer to the file int add_customer(FILE *fp, struct Customer *cust) { fseek(fp, 0, SEEK_END); return fwrite(cust, sizeof(struct Customer), 1, fp); }

// Function to find a customer by ID int find_customer(FILE *fp, int customer_id, struct Customer *cust) { rewind(fp); while (fread(cust, sizeof(struct Customer), 1, fp) == 1) { if (cust->customer_id == customer_id) { return 1; // Found } } return 0; // Not found } ```

This file-based approach worked adequately for simple applications with limited data and few users. However, as computing needs grew more complex, several fundamental problems emerged that file-based systems could not adequately address.

The Problems with File-Based Systems

The limitations of file-based storage became increasingly severe as applications became more sophisticated. These problems, which drove the development of database technology, fall into several interconnected categories.

Data Redundancy and Inconsistency represent one of the most significant issues with file-based systems. In an organization with multiple applications, each application typically maintained its own files containing overlapping data. A customer might appear in the customer master file, the accounts receivable file, the sales order file, and the shipping file. This redundancy meant that the same data was stored multiple times, wasting storage space. More critically, when a customer's address changed, updating it in all relevant files was error-prone and often incomplete, leading to data inconsistency where different files contained contradictory information about the same entity.

Difficulty in Accessing Data plagued file-based systems because accessing data required writing custom programs for each specific query. There was no standardized way to ask questions about the data. If a business analyst needed to know all customers in a particular zip code who had made a purchase in the last month, a programmer would have to write a special program to extract this information. The rigid structure of sequential or indexed files made ad-hoc queries slow and impractical. There was no query language—only the fixed operations that application programmers had anticipated and implemented.

Data Isolation occurred because data was scattered across so many files in various formats. Writing programs to access multiple files simultaneously was complex and error-prone. The programmer had to understand the physical format and location of each file, as well as how to navigate through different access methods. This made it nearly impossible to write applications that could flexibly adapt to new requirements.

Integrity Problems emerged because data integrity constraints were embedded in application programs rather than being enforced at the data level. Consider the constraint that a customer's account balance should never be negative. In a file-based system, this constraint would be enforced only in the application programs that updated the balance field. If a programmer forgot to include the check, or if another program was written that could modify the balance without the check, the constraint could be violated. Similarly, ensuring that referenced entities existed—for example, that an order record referenced a valid customer—was difficult to enforce consistently across all applications.

Atomicity Issues became critical for business transactions that spanned multiple files. A bank transfer involves debiting one account and crediting another—both operations must succeed or both must be rolled back. In file-based systems, if the system failed after completing the debit but before completing the credit, the data would be left in an inconsistent state. There was no general mechanism to ensure that a sequence of file operations either all completed or all were undone.

**Concurrent Access Anom