Foundations of Explainable AI: Context, Taxonomy, and Imperatives

The black box problem is further exacerbated by the scale of modern foundation models, which often contain hundreds of billions of parameters trained on trillions of tokens of unstructured data. Unlike early machine learning systems such as logistic regression or decision trees, where decision logic could be directly traced to feature weights or split rules, deep learning models rely on stacked non-linear transformations that obscure the mapping between input features and output predictions. For instance, a 7-billion-parameter large language model (LLM) like LLaMA 2 processes a single input prompt through 32 transformer layers, with each layer containing billions of interacting parameters; even the model’s developers cannot manually enumerate the causal chain that leads to a specific output, such as a summary of a research paper or a diagnosis of a medical condition. This opacity is not merely a theoretical inconvenience: it has tangible real-world consequences when black box systems are deployed in high-stakes domains.

Historical Evolution of Interpretability Requirements

The demand for model interpretability is not new, but its urgency has evolved alongside advances in AI system complexity. To contextualize the current XAI landscape, it is useful to trace the historical trajectory of interpretability requirements across three distinct eras of AI development.

Era 1: Symbolic AI and Expert Systems (1950s–1980s)

The first generation of AI systems were built on symbolic reasoning, where knowledge was encoded as explicit if-then rules and logical propositions. These systems were inherently interpretable by design: for example, MYCIN, a 1970s expert system developed at Stanford to diagnose bacterial infections, would output a recommendation for a specific antibiotic alongside a full trace of the rules used to reach that conclusion, such as “If the patient has a fever >38°C and a positive gram stain for gram-negative rods, then the probability of sepsis is 85%.”[^1] Clinicians could verify, challenge, or modify these rules directly, making MYCIN’s decision process fully transparent. The core limitation of this era was not interpretability, but scalability: symbolic systems required manual knowledge engineering from domain experts, making them prohibitively expensive to build for complex, high-dimensional tasks like image recognition or natural language processing.

Era 2: Traditional Machine Learning (1990s–2010s)

The rise of statistical machine learning shifted AI development from rule-based systems to data-driven models, including support vector machines (SVMs), random forests, and gradient boosting machines. These models offered improved performance on many tasks, but their interpretability varied widely. Linear models such as logistic regression retained full intrinsic interpretability: each feature’s coefficient directly quantified its contribution to the output, and global decision boundaries could be visualized easily. Tree-based ensembles like random forests provided global feature importance scores, but their internal decision logic was far less transparent than a single decision tree, as the aggregated output of hundreds of independent trees could not be easily summarized in human-readable rules.

By the early 2010s, the machine learning community had begun to recognize the limitations of opaque models in high-stakes domains. For example, a 2011 study of a commercial credit scoring model found that 30% of rejected applicants could not be given a clear explanation for their denial, leading to regulatory scrutiny and consumer complaints. However, the problem was still manageable for most traditional models, as their relatively small parameter counts and simple decision boundaries allowed for post-hoc analysis using techniques like partial dependence plots or rule extraction.

Era 3: Deep Learning and Foundation Models (2012–Present)

The 2012 ImageNet competition, where AlexNet achieved a top-5 error rate of 15.3%—10.9 percentage points lower than the previous state-of-the-art—marked the beginning of the deep learning revolution. As model architectures grew in complexity—from convolutional neural networks (CNNs) for computer vision to recurrent neural networks (RNNs) for sequence modeling, and finally to transformer-based foundation models—the interpretability gap widened dramatically. By 2023, state-of-the-art LLMs contained up to 1 trillion parameters, and multimodal foundation models could process text, image, audio, and video inputs simultaneously, making their decision logic virtually impossible to audit manually.

This opacity has led to a series of high-profile failures that have galvanized regulatory and public demand for XAI. The most well-documented case is Amazon’s internal hiring algorithm, which was abandoned in 2017 after Reuters revealed that it systematically discriminated against female candidates.[^2] The algorithm