Foundations of AI Ethics and Technical Prerequisites

In 2018, Amazon’s internal AI recruiting team discovered a fatal flaw in their automated hiring system: the algorithm systematically downgraded resumes containing the word “women’s,” such as “women’s chess club captain,” and penalized graduates of two all-women’s colleges. Trained on a decade of historical hiring data from a tech industry that had long been dominated by men, the model had learned to associate male candidates with positive outcomes and female candidates with negative ones. When the team tried to correct the bias by removing the gender feature from the input data, the model simply learned to infer gender from other proxy features, such as participation in women’s organizations. Amazon ultimately scrapped the system in 2017, but the incident laid bare an uncomfortable truth: AI ethics is not an abstract philosophical exercise, but a set of concrete technical decisions embedded in every stage of the machine learning lifecycle. This chapter establishes the foundational conceptual and technical ground for the rest of the text, bridging normative ethical theory with the technical primitives that drive AI system design, and highlighting the unique ethical risks posed by AI relative to traditional software.

<!-- tags: ai-ethics, foundational-frameworks, ml-lifecycle -->

Normative Ethical Frameworks as Direct Mappings to AI Design Decisions

Core normative ethical frameworks provide the conceptual scaffolding for evaluating AI system design and deployment choices. Unlike generic ethical guidelines, these frameworks map directly to concrete technical decisions, from utility function formulation to constraint design in model training. We review four dominant frameworks and their practical implications for AI engineering.

Utilitarianism: Consequence-Oriented AI Design

Utilitarianism, formalized by Jeremy Bentham