The Autonomous Vehicle Stack: System Architecture and Design Principles

At the heart of every autonomous vehicle lies a complex symphony of hardware and software, orchestrated through a carefully designed system architecture. This architecture determines not only what the vehicle can do, but how reliably, safely, and efficiently it performs. Understanding this blueprint is the first step toward mastering autonomous driving technology. The autonomous vehicle stack is not a single monolithic program but a distributed, real-time system that must perceive an uncertain world, make split-second decisions, and control actuators with millimeter precision—all while maintaining functional safety and handling inevitable hardware failures. This chapter establishes the foundational blueprint, exploring the historical evolution from driver-assistance to full autonomy, dissecting modern layered architectures, and examining the critical hardware and software choices that shape system capabilities. We will delve into core design principles—modularity, redundancy, and real-time constraints—that separate research prototypes from deployable systems. By the end, you will possess a framework to critique existing architectures, such as those from Waymo or Tesla, and propose reasoned improvements for specific operational design domains.

Historical Evolution: From ADAS to Full Autonomy

The architectural journey of autonomous vehicles mirrors the broader evolution of computing: from centralized batch processing to distributed, networked systems. In the early 1990s, projects like Carnegie Mellon's Navlab and EUREKA's Prometheus treated autonomy as a monolithic, tightly coupled problem. These systems often ran on single, powerful workstations with custom hardware interfaces. Perception, planning, and control were monolithic processes sharing memory space, a design that limited scalability and robustness. The primary goal was proof-of-concept, not operational reliability.

The 2000s saw the rise of Advanced Driver-Assistance Systems (ADAS) in production vehicles. Features like adaptive cruise control (ACC) and lane-keeping assist (LKA) introduced a modular separation: a sensor layer (radar, camera), a perception/decision layer (often a dedicated Electronic Control Unit or ECU), and an actuator interface. These systems were *designed for driver supervision*, not full autonomy. Their architectures were *functionally isolated*: the ACC module did not need to understand lane markings, and the LKA module ignored radar targets. This siloed approach simplified certification under standards like ISO 26262 but created a *fragmented landscape* where cooperation between ADAS features was limited to basic arbitration logic.

The DARPA Grand Challenges (2004-2007) catalyzed a paradigm shift. Teams like Stanford's Stanley and CMU's Boss moved toward a *sense-plan-act* paradigm, but with a crucial innovation: *middleware*. To integrate multiple sensors (LIDAR, radar, cameras), multiple planning algorithms, and complex control loops, teams adopted robotics middleware like the Robot Operating System (ROS), which provided hardware abstraction, inter-process communication, and tooling. This era established the *layered architecture* as the de facto standard for autonomous driving research.

The modern era, marked by companies like Waymo, Cruise, and Tesla, sees this layered architecture refined for scale and safety. Waymo's "Chauffeur" system exemplifies a *conservative, sensor-heavy* approach with explicit redundancy at every layer. Tesla, in contrast, pursues a *vision-only* architecture, leveraging massive data fleets to compensate for sensor limitations. Both, however, adhere to a core principle: *separation of concerns* across perception, prediction, planning, and control. This separation allows independent development, testing, and certification of modules but introduces challenges in interface definition and timing guarantees.

Key Milestones in Architectural Thinking

| Era | Primary Goal | Architectural Style | Example | |-----|--------------|---------------------|---------| | 1990s | Proof-of-Concept | Monolithic, Centralized | Navlab | | 2000s | Driver Assistance | Functional Isolation, Siloed ECUs | ACC/LKA Systems | | 2004-2007 | DARPA Challenges | Layered Sense-Plan-Act with Middleware | Stanley, Boss | | 2010s-Present | Commercial Deployment | Redundant, Distributed, Safety-Critical | Waymo, Tesla, Cruise |

This evolution reveals a clear trajectory: from *tightly coupled* monolithic systems toward *loosely coupled*, *redundant* distributed systems. The driver is no longer a fallback but a *supervisory agent* whose role is being systematically engineered out of the loop. This shift demands architectures that are not just capable, but *provably safe* and *scalable*.

The Modern Layered Architecture: Sense-Plan-Act and Its Variants

The canonical architecture for autonomous driving is the *sense-plan-act* loop, borrowed from robotics. It decomposes the driving task into three sequential stages:

1. Sense (Perception & Localization): Convert raw sensor data into a structured representation of the world—a *scene graph* or *occupancy grid*—and determine the vehicle's precise pose within a map. 2. Plan (Prediction & Motion Planning): Predict the future behavior of dynamic agents (other vehicles, pedestrians) and generate a safe, comfortable, and efficient trajectory for the ego vehicle. 3. Act (Control): Execute the trajectory by sending low-level commands to the steering, throttle, and brake actuators.

This decomposition is powerful because it mirrors human cognition: *see* → *think* → *act*. However, its sequential nature introduces *latency accumulation*: an error or delay in perception propagates to planning and control. Modern systems mitigate this with *parallelization* and *feedback loops*.

```mermaid graph TD A[Sensors<br/>LIDAR, Camera, Radar, IMU, GNSS] --> B[Perception Module] A --> C[Localization Module] B --> D[World Model<br/>Object List, Occupancy Grid] C --> D D --> E[Prediction Module] E --> F[Planning Module] F --> G[Trajectory] G --> H[Control Module] H --> I[Actuators<br/>Steering, Throttle, Brake] I --> J[Vehicle Dynamics] J --> A style A fill:#e1f5fe style B fill:#f3e5f5 style C fill:#f3e5f5 style D fill:#fff3e0 style E fill:#fff3e0 style F fill:#e8f5e8 style G fill:#e8f5e8 style H fill:#fce4ec style I fill:#fce4ec style J fill:#e1f5fe ```

Figure 1.1: The canonical sense-plan-act loop with feedback. Note the parallel perception and localization, and the feedback from vehicle dynamics to sensors (for motion compensation). Colors indicate conceptual layers: sensing (blue), world modeling (orange), decision (green), control (pink).

Variants and Hybrid Architectures

Pure sequentiality is rare. Real-world systems employ several variations:

• **Parallel Perception:** Multiple perception algorithms (e.g., a LIDAR-based 3D detector and a camera-based 2D detector) run concurrently, and their outputs are *fused* before world modeling. This reduces single-point failure risk.
• **Behavioral Planning Hierarchies:** The "Plan" stage is often split into a high-level *behavioral planner* (e.g., "change lane now") and a low-level *motion planner* (e.g., generate a specific trajectory). This allows strategic decisions to be made with less frequent updates (e.g., 10 Hz) than trajectory generation (e.g., 100 Hz).
• **Predictive Planning:** Some architectures integrate prediction and planning tightly, using *game-theoretic* models where the planner anticipates how other agents will react to the ego vehicle's trajectory. This is computationally intensive but can handle interactive scenarios better.
• **End-to-End Learning:** A controversial variant where a single neural network maps raw sensor data directly to control commands (steering, acceleration). While elegant, these systems are *black boxes*, making safety certification