Foundations of AI Ethics: Theories and Frameworks
The development of artificial intelligence systems that make consequential decisions about human lives has created an urgent need for rigorous ethical frameworks. Unlike traditional software that merely executes instructions, AI systems increasingly operate in domains requiring nuanced judgment—approving loans, diagnosing diseases, controlling vehicles, and allocating scarce resources. These applications demand that we confront fundamental questions about how machines should behave when human values are at stake. Before we can design ethical AI systems, however, we must understand the philosophical foundations upon which such systems must be built.
This chapter establishes the conceptual groundwork necessary for the technical work that follows. We examine major ethical theories—utilitarianism, deontology, virtue ethics, and care ethics—exploring not only their core principles but also how each framework generates specific recommendations for AI decision-making scenarios. More importantly, we analyze the limitations and tensions that emerge when these frameworks encounter the unique challenges posed by computational systems: the problem of moral status for AI entities, the attribution of agency and responsibility in distributed systems, the paradox of incommensurable values, and the profound difficulty of encoding human values into formal computational processes. Through detailed case studies in algorithmic hiring, medical diagnosis, and resource allocation, we demonstrate how different ethical frameworks lead to divergent recommendations, and why understanding these frameworks is essential for anyone building or deploying AI systems.
The Philosophical Landscape of AI Ethics
The intersection of artificial intelligence and ethics is not merely an academic exercise. Every AI system that makes or influences decisions affecting human welfare implicitly embodies certain ethical assumptions, whether those assumptions were made explicitly by designers or emerged implicitly from training data and optimization objectives. Understanding these underlying ethical frameworks is crucial because different frameworks not only provide different justifications for AI behavior but can also generate directly contradictory recommendations for the same scenario.
The field of AI ethics draws heavily from moral philosophy, but AI systems introduce novel complications that strain traditional frameworks. When a deep learning system decides whether to approve a mortgage application, who bears moral responsibility for the outcome? When an autonomous vehicle must choose between two harmful actions, what principle should govern the decision? When a recommendation algorithm shapes public opinion, what obligations does it have to the people it influences? These questions require us to both apply established ethical theories and extend them to address situations that philosophers have not previously contemplated.
Consequentialism and Utilitarian Approaches
Utilitarianism, the most prominent form of consequentialism, holds that the moral rightness of an action is determined solely by its outcomes—in particular, by the extent to which it promotes happiness or reduces suffering. The classical utilitarian tradition, associated with Jeremy Bentham and John Stuart Mill,主张 that we should maximize aggregate welfare, typically understood as the greatest good for the greatest number.
For AI systems, utilitarian reasoning has proven particularly influential in the development of decision-making frameworks. Consider the now-classic scenario of autonomous vehicles facing unavoidable accidents. A utilitarian approach would evaluate potential actions based on their expected consequences: minimizing total harm, potentially even including harm to the vehicle's occupants. This reasoning underlies proposed algorithms that would, in certain extreme circumstances, calculate the action resulting in the fewest total casualties.
The mathematical formalization of utilitarianism in AI often takes the form of utility functions—mathematical representations of preferences or welfare that can be optimized. Reinforcement learning systems, for instance, learn policies that maximize cumulative reward, which is itself a utility function. The challenge, as we shall see, lies in specifying utility functions that adequately capture the full range of human values.
$$U_{total} = \sum_{t=0}^{\infty} \gamma^t \cdot u(s_t, a_t)$$
This formulation, common in reinforcement learning, represents cumulative discounted utility as the sum of immediate utilities $u$ at each time step $t$, weighted by discount factor $\gamma$. While mathematically elegant, such formulations reveal a fundamental tension: utility functions can represent preference orderings, but the translation from complex human values to numerical utility is neither straightforward nor neutral.
Research by Hadfield-Menell et al. (2016) on cooperative inverse reinforcement learning highlighted another limitation of purely utilitarian approaches to AI: the difficulty of specifying correct utility functions without full knowledge of human preferences. When humans themselves disagree about values, or when individuals' preferences conflict with each other, aggregating these into a single utility function involves necessarily normative choices that utilitarian frameworks alone cannot resolve.
Deontological Ethics and Moral Constraints
Deontological ethics, associated primarily with Immanuel Kant, emphasizes duties, rights, and moral constraints independent of consequences. For Kant, the fundamental principle was the categorical imperative: act only according to maxims that you could will to become universal laws, and treat humanity never merely as a means but always as an end in itself.
Deontological approaches to AI ethics focus on what AI systems should not do rather than solely on outcomes to be maximized. This perspective has proven influential in debates about lethal autonomous weapons systems, where the concern is not merely that such systems might cause excessive harm, but that certain actions—particularly the deliberate killing of humans—might be categorically prohibited regardless of their consequences.
The Kantian emphasis on treating persons as ends rather than means has direct implications for AI system design. Recommendation systems that manipulate users' emotions or exploit psychological vulnerabilities for profit treat users as means to corporate ends. Hiring algorithms that score candidates based on irrelevant characteristics or that function as proxies for discriminatory factors treat applicants as means to organizational efficiency rather than as autonomous persons deserving fair consideration.
Deontological frameworks also introduce the concept of moral constraints—actions that are absolutely prohibited regardless of their beneficial consequences. For AI systems, this suggests the need for what technical researchers have termed "ethical governors" or "hard constraints" that cannot be violated even under instructions to maximize some objective. The Asilomar AI Principles, developed through broad consultation with AI researchers, include the principle that "Highly autonomous AI systems should be designed so that their goals and behaviors can be assured to align with human values throughout their operation," which reflects a deontological concern with absolute alignment rather than mere probabilistic alignment.
However, deontological approaches face their own challenges in the AI context. Pure deontology struggles with situations involving unavoidable moral tradeoffs—scenarios where any action will violate some duty. The classic case of the ticking time bomb, where torture might be justified to save many lives, illustrates the tension between absolute prohibitions and desperate circumstances. For AI systems operating in real-world environments, such conflicts are not theoretical but practical.