Theoretical Foundations: Linguistics, Philosophy, and the Computational Turn

The multidisciplinary bedrock upon which contemporary Natural Language Processing (NLP) is constructed is not merely a historical curiosity but a living, breathing framework that continues to inform and constrain our most advanced models. To treat NLP as a purely engineering discipline, divorced from its linguistic and philosophical ancestry, is to risk building on sand. The central argument of this chapter is that every major paradigm shift in NLP—from symbolic rule-based systems to statistical methods and now to deep learning—has been a response to, and a reinterpretation of, foundational questions about the nature of language, meaning, and rational communication. The "architecture of meaning" in our title is therefore not a metaphor for neural network layers alone; it refers to the enduring intellectual scaffolding provided by formal linguistics, analytical philosophy, and the computational lens that has focused them into an engineering science.

**1.1 The Linguistic Substrate: From Descriptive Rules to Formal Grammars**

Language, at its core, is a combinatorial system. A finite set of phonemes combines into a finite set of morphemes, which combine according to syntactic rules to generate a theoretically infinite array of utterances. This combinatorial property is the first and most critical insight for any computational treatment of language.

Phonology and Morphology: The Atomic Level. While modern NLP often begins at the word or subword level (e.g., Byte-Pair Encoding), the foundational problems are phonological and morphological. The phoneme is the smallest unit capable of distinguishing meaning (*minimal pair*), as in /p/ vs. /b/ in *pat* vs. *bat*. Computational phonology deals with finite-state transducers that model rules like pluralization (/s/ vs. /z/ vs. /ɪz/ after voiced, voiceless, sibilant contexts) or past tense formation (the famous English *go/went* irregularity). Morphology bridges sound and syntax. The distinction between *derivational* morphology (changing word class or core meaning, e.g., *happy* → *unhappiness*) and *inflectional* morphology (encoding grammatical relations like number, tense, case, e.g., *dog* → *dogs*, *walk* → *walked*) is crucial for tasks like stemming and lemmatization. A formal model is the Finite-State Morphology (Koskenniemi, 1983), where a single lexicon and two-level rules map surface forms to lexical representations. For example, the rule that final *e* is deleted before *-ing* (*make* → *making*) can be captured by constraints operating over a finite-state network.

Syntax: The Architecture of Combination. Syntax governs how words are ordered and grouped into phrases. The descriptive goal is to distinguish grammatical from ungrammatical strings (e.g., *The cat chased the dog* vs. *\*Chased the cat the dog*). The explanatory goal is to understand the mental representation of this knowledge. Phrase Structure Grammar (Harris, 1950s) introduced constituency, the idea that words group into nested, hierarchical structures. A context-free grammar (CFG) formalizes this with rules like $S \rightarrow NP\ VP$, $VP \rightarrow V\ NP$. This provides a clean, recursive engine for generation and parsing. However, CFGs are inadequate for cross-serial dependencies (e.g., Dutch or Swiss German verb clusters) and for capturing long-distance relationships like subject-verb agreement across intervening phrases (*The cats [that the dog chased] were fast*). This led to mildly context-sensitive grammars (e.g., Tree-Adjoining Grammar, Linear Context-Free Rewriting Systems), which have just enough power to handle natural language while remaining computationally tractable ($O(n^3)$ parsing).

The most influential formal framework, however, is Generative Grammar (Chomsky, 1957, 1965). Its core principles are: 1. The Generativity of Language: A finite set of rules and principles can generate an infinite set of sentences. 2. The Competence/Performance Distinction: We model an idealized speaker-hearer's *competence* (knowledge of the language), not *performance* (actual use with errors, memory limits). 3. The Universal Grammar (UG) Hypothesis: The initial state of the language faculty consists of a set of universal principles and parameters that are fixed by exposure to primary linguistic data (the "poverty of the stimulus" argument). 4. The Autonomy of Syntax: Syntactic structures are generated by a dedicated, formal module, independent of (though later interfacing with) semantics and phonology.

The successive models—Standard Theory (1965), Government and Binding Theory (1981), and the Minimalist Program (1995)—refined the machinery (e.g., X-bar theory, Move-α, feature checking, Merge), but the core commitment to a formal, rule-based, and innately constrained system remained. This framework directly inspired the first generation of NLP parsers (e.g., the SHRDLU world, early ATN parsers), which attempted to encode these explicit, hand-crafted rules.

In contrast, Construction Grammar (Fillmore & Kay, 1990s; Goldberg, 1995) rejects the strict generative dichotomy between lexicon and grammar. It posits that the basic units of language are constructions—form-meaning pairings at all levels of generality and abstraction, from morphemes (*-ed* for past tense) to idioms (*kick the bucket*) to fully abstract argument structures (*Ditransitive Construction*: *X gave Y Z*). Meaning is not solely derived from the compositional assembly of verb-specific entries but is stored directly with the syntactic pattern. This perspective has profound implications for distributional semantics and modern neural models, which inherently learn form-meaning associations from corpus data without explicit rule separation.

Semantics and Pragmatics: The Interface to Meaning. Semantics is the study of meaning in isolation; pragmatics is meaning in context.

• **Formal Semantics**, rooted in Montague Grammar (1970s), uses **model theory** from mathematical logic to define sentence meaning. The meaning of a sentence is its **truth conditions** in a possible world. Key tools are **λ-calculus** for function-argument composition and **generalized quantifiers** (e.g., *every* as a function from sets to truth values: $\forall x (Student(x) \rightarrow Brilliant(x))$). The **Principle of Compositionality** (Frege's principle) states that the meaning of a complex expression is a function of the meanings of its parts and their syntactic mode of combination. This provides a precise, algebraic framework for semantic interpretation that directly maps onto syntactic trees. For instance, the VP *chased the dog* can be represented as $\lambda x. \exists y (Dog(y) \land Chase(x, y))$, which, when applied to the subject NP *the cat* ($\iota x. Cat(x)$), yields a complete proposition.
• **Lexical Semantics** deals with word meaning. **Predicate-Argument Structure** (Levin, 1993) classifies verbs into **semantic classes** (e.g., *Break* verbs: *break, snap, fracture*; *Send* verbs: *send, mail, ship*) that share syntactic behavior (causative/inchoative alternation: *The window broke* vs. *John broke the window*). **Word Sense Disambiguation (WSD)** is the computational task of selecting the correct sense from a resource like **WordNet** (Fellbaum, 1998), a manually curated semantic network grouping synonyms into **synsets** linked by relations like *hypernymy* (is-a), *meronymy* (part-of), and *antonymy*.
• **Pragmatics** studies how context contributes to meaning. **Speech Act Theory** (Austin, 1962; Searle, 1969) distinguishes **locutionary** (the act of saying), **illocutionary** (the intended force: asserting, questioning, commanding), and **perlocutionary** (the effect on the listener) acts. *"It's cold in here"* may be a locution stating a fact, an illocutionary *request* to close the window, and a perlocutionary act that causes someone to get up. **Gricean Pragmatics** (G