Autonomous Driving System Taxonomy and Core Design Principles

The historical evolution of core module architectures can be traced back to the late 20th century, with early prototypes relying on hand-engineered rules and limited sensing capabilities: Carnegie Mellon University’s Navlab 1 (1986) used five monocular cameras and a shallow neural network for road following, achieving a top speed of 1 km/h with a rule-based control stack. By 1995, Navlab 5 completed the first coast-to-coast autonomous drive in the U.S., with 98% of the 4,800 km journey completed autonomously, using a hierarchical rule-based pipeline that combined GPS-based localization, laser-based obstacle detection, and finite state machine (FSM) based decision logic. The 2004–2007 DARPA Grand Challenge series marked the first large-scale validation of autonomous driving architectures: Stanford University’s Stanley, the winner of the 2005 challenge, combined LiDAR-based obstacle detection, GPS/IMU localization, and a hybrid rule-learning planning stack to achieve an average speed of 30.7 km/h across the 212 km desert course, while CMU’s Boss, the 2007 Urban Challenge winner, added structured rule sets for urban traffic compliance (traffic light recognition, right-of-way negotiation, pedestrian avoidance) but remained fundamentally reliant on pre-defined logical rules that failed in unmodeled scenarios such as temporary construction zones.

The 2010s saw the emergence of the modular learning-augmented pipeline that remains the dominant industrial architecture today, pioneered by Google X’s self-driving car project (later Waymo). This architecture decomposes the end-to-end driving task into discrete, loosely coupled subsystems, each optimized for a specific function, with machine learning models replacing hand-engineered rules for perception and prediction tasks. The 2020s have introduced two competing architectural paradigms: the modular transformer-based pipeline with emerging large model integration, and the fully end-to-end neural network architecture that eliminates explicit modular decomposition. To formalize the functional boundaries and interactions of these subsystems, we first define the standard autonomous driving functional stack as a six-tuple $AS = (S, A, P, R, O, \pi)$, where:

• $S \in \mathbb{R}^{n_s}$ is the system state space, encompassing vehicle dynamic states (position, velocity, acceleration, heading), environmental states (road geometry, traffic signals, weather), and traffic participant states (position, velocity, intent of surrounding vehicles, pedestrians, and non-motorized road users);
• $A \in \mathbb{R}^{n_a}$ is the action space, consisting of low-level control commands (steering angle, throttle opening, brake pressure);
• $P: S \times A \rightarrow S$ is the stochastic state transition function, modeling the dynamic evolution of the environment and vehicle response to control inputs;
• $R: S \times A \rightarrow \mathbb{R}$ is the reward function, quantifying the performance of a given action in terms of safety, comfort, and efficiency, typically formalized as a weighted sum $R = w_1 R_{safe} + w_2 R_{comfort} + w_3 R_{efficiency}$, where $w_1 \gg w_2, w_3$ to prioritize safety as the primary design objective;
• $O \in \mathbb{R}^{n_o}$ is the observation space, comprising raw and processed data from on-board sensors (cameras, LiDAR, radar, GNSS, IMU);
• $\pi: O \rightarrow A$ is the system policy, the core mapping from sensor observations to control actions that is the primary target of autonomous driving system design.

The standard decomposition of this functional stack into discrete subsystems, aligned with industrial practice and the SAE J3016 taxonomy, is illustrated in the following diagram: ```mermaid graph TD A[Autonomous Driving System<br/>(AS = (S,A,P,R,O,π))] --> B[Perception Subsystem<br/>(O → Object/Scene State)] A --> C[Localization Subsystem<br/>(O → Vehicle Pose in S)] A --> D[Prediction Subsystem<br/>(S_t → S_{t+1:T} for surrounding actors)] A --> E[Decision-Making Subsystem<br/>(S → High-level behavior intent)] A --> F[Motion Planning Subsystem<br/>(Intent, S → Safe, comfortable trajectory)] A --> G[Control Subsystem<br/>(Trajectory, S → Low-level control commands A)] A --> H[Vehicle Interface Layer<br/>(A → Actuator actuation, HMI feedback)] %% Perception submodules B --> B1[Camera Processing<br/>(2D/3D detection, semantic segmentation)] B --> B2[LiDAR Processing<br/>(3D point cloud detection, SLAM)] B --> B3[Radar Processing<br/>(Velocity measurement, long-range obstacle detection)] B --> B4[Multi-Sensor Fusion<br/>(Cross-modal feature alignment, state estimation)] %% Localization submodules C --> C1[GNSS/RTK<br/>(Global position, <10cm accuracy)] C --> C2[IMU<br/>(High-frequency pose, velocity estimation)] C --> C3[LiDAR SLAM<br/>(Localization in GPS-denied environments)] C --> C4[Visual SLAM<br/>(Camera-based pose estimation)] %% Prediction submodules D --> D1[Vehicle Trajectory Prediction<br/>(Interaction-aware forecasting)] D --> D2[Pedestrian Trajectory Prediction<br/>(Intent-aware motion forecasting)] D --> D3[Non-Motorized Vehicle Prediction<br/>(Cyclist, scooter trajectory forecasting)] %% Decision submodules E --> E1[Behavior Planning<br/>(FSM/LLM-based high-level intent selection)] E --> E2[Route Planning<br/>(Global path optimization from origin to destination)] E --> E3[Rule Compliance Checking<br/>(Traffic law, ODD constraint verification)] %% Planning submodules F --> F1[Global Path Planning<br/>(A*, Dijkstra, or learning-based routing)] F --> F2[Local Trajectory Planning<br/>(Hybrid A*, lattice planner, sampling-based planner)] F --> F3[Trajectory Optimization<br/>(Smoothness, safety, efficiency optimization)] %% Control submodules G --> G1[Longitudinal Control<br/>(Speed tracking, adaptive cruise control)] G --> G2[Lateral Control<br/>(Lane keeping, path tracking, Stanley/MPC control)] G --> G3[Stability Assurance<br/>(Anti-lock braking, electronic stability control integration)] %% Interface submodules H --> H1[Actuator Control<br/>(CAN/ETH communication with brake, steering, powertrain)] H --> H2[Vehicle State Feedback<br/>(Wheel speed, steering angle, actuator state monitoring)] H --> H3[HMI Interface<br/>(Driver/passenger alerts, takeover requests)] ```

This decomposition is not arbitrary: it reflects both the physical separation of sensing, computation, and actuation in modern vehicle platforms, and the distinct technical requirements of each driving subtask. For example, the perception subsystem operates at 10–30 Hz to process raw sensor data into structured environmental states, while the control subsystem operates at 100–1000 Hz to ensure stable vehicle dynamic response, a latency requirement that cannot be met by higher-level planning or decision modules. The loose coupling between subsystems, enforced by standardized interfaces (e.g., the perception subsystem outputs a standardized list of obstacles with associated position, velocity, class, and confidence scores, regardless of the underlying sensor suite or detection algorithm), enables independent upgrades of individual modules without full system revalidation, a critical requirement for over-the-air (OTA) update capabilities in production systems.

The core design of any autonomous driving system is governed by four interdependent, often conflicting, performance dimensions: safety, latency, computational cost, and operational performance. These dimensions form a Pareto frontier, where improvement in one dimension inevitably incurs a tradeoff in one or more others, and no single design can achieve optimal performance across all four simultaneously. To formalize these tradeoffs, we first define quantitative metrics for each dimension: 1. Safety: Quantified as the per-hour collision probability $P_{coll} \in [0,1]$, with regulatory requirements for Level 3/4 systems m