The Day Maya Met the Algorithm

The Day Maya Met the Algorithm

The email arrived on a Tuesday afternoon, right between second and third period. Maya Chen had just settled into her usual seat in AP Computer Science—front row, left side, next to the window where she could watch the oak tree sway when she needed to think—when her phone buzzed. She glanced at the notification: SummerTech Academy: Application Status Update.

Maya's heart did that little skip it always did when something might be about to change her life. She'd been hearing about SummerTech since eighth grade—three weeks at MIT, working with actual engineers, building projects that mattered. Her older brother Marcus had attended two years ago and come back with stories about debugging neural networks and pitching ideas to venture capitalists. He'd gotten into Stanford's computer science program that fall. Maya wanted Stanford too. SummerTech was the first step.

She opened the email.

*Dear Ms. Chen,*

*Thank you for your interest in SummerTech Academy. After careful review of all applications by our admissions committee, we regret to inform you that we are unable to offer you a place in this year's program.*

*We received a record number of qualified applications this cycle, and the competition was exceptionally strong. This decision reflects the difficulty of our selection process rather than any shortcoming in your qualifications.*

*We encourage you to reapply next year and wish you the best in your academic pursuits.*

*Warm regards,* *The SummerTech Admissions Team*

Maya read it twice. Then a third time. The words didn't change.

*But I didn't apply.*

She sat very still. Around her, students were shuffling papers, charging laptops, asking each other about the weekend. The teacher, Mr. Patterson, was writing something on the board about recursion. Maya stared at her phone as if it might suddenly reveal some hidden message, some explanation for why she was being rejected from something she'd never even started.

Her friend Priya leaned over. "You okay? You look like you saw a ghost."

"I didn't apply," Maya whispered.

"What?"

"I never submitted an application to SummerTech. I was going to—I've been planning it for months—but I never finished the essay. I never sent it."

Priya's face scrunched up in confusion. "Then why did they email you?"

That was exactly the question Maya couldn't answer. She stared at her phone, at the rejection that had arrived from a future that had never existed for her, and felt something cold settle in her stomach. Someone—something—had decided she wasn't good enough before she'd even tried.

---

The Invisible Gatekeepers

What happened to Maya isn't rare. It's not even unusual. Across the country, and across the world, algorithms are making decisions about people's lives every single day—decisions about who gets into programs, who gets hired, who gets loans, who gets parole, who gets flagged for extra security screening at airports. Most of the time, nobody sees these decisions happen. They just see the results: the acceptance letter, the job offer, the approval notice—or the rejection.

This is what we mean when we talk about AI decision-making: the use of machine learning systems and automated processes to evaluate, categorize, and sort human beings. These systems are designed to be efficient, to handle millions of applications and requests and transactions that would overwhelm any human workforce. They're also designed to be consistent, to apply the same rules to everyone without the messy complications of human bias or fatigue or bad moods.

Sounds good, right?

The problem is that these systems don't operate in a vacuum. They're built by people—people with their own assumptions, their own blind spots, their own data. And the data they learn from? That data comes from the world as it actually is: a world with centuries of discrimination baked into it, encoded in historical patterns of who got opportunities and who didn't, who lived where and went to which schools and worked which jobs.

When an algorithm learns from historical data, it learns history. And history isn't neutral.

Let's look at Maya's case more closely. How did SummerTech's system know to reject her when she'd never applied?

The answer lies in a web of data-sharing agreements and automated systems that most people never see. SummerTech Academy partners with a company called EdMetrics that helps schools and programs identify promising candidates early—before they even apply. EdMetrics collects data from public school records, standardized test scores, extracurricular databases, and demographic information. Their algorithm then generates "predicted applicant profiles" for students who might be strong candidates, flagging them for outreach.

Except when the algorithm flags students for outreach, it also flags them for rejection.

Here's why: SummerTech has a limited number of slots and an overwhelming number of qualified applicants. To manage this, they've built a system that automatically filters out candidates based on a "risk score" before any human admissions officer even sees the file. The risk score is calculated using dozens of factors—some obvious, like academic performance, and some much less obvious, like neighborhood crime statistics, average credit scores in the student's zip code, and whether their family's address appears in certain public records databases.

Maya's address triggered a flag.

She lived in the Lakewood neighborhood on the east side of the city—a working-class area with a complicated history. In the 1980s, Lakewood had been redlined by banks, meaning that generations of families had been denied mortgages and home improvement loans based purely on where they lived, not who they were. The neighborhood had higher poverty rates than suburban areas, higher crime rates (in part because of underinvestment in police and community resources), and lower property values. It also had some of the most dedicated, hardworking people Maya had ever known.

But the algorithm didn't know any of that. It just saw the data: Lakewood address, elevated risk profile, automatic rejection. Maya's 4.0 GPA, her coding competitions, her robotics team leadership—none of it mattered, because her application was filtered out before a human ever laid eyes on it.

She wasn't rejected because she wasn't qualified. She was rejected because a machine decided her zip code made her too risky.

---

How Algorithms Learn to Discriminate

To understand what happened to Maya, we need to understand a bit about how machine learning systems actually work. This isn't going to be a computer science lecture—I promise. But a basic understanding of what these systems do and how they do it is essential if we're going to figure out how to make them fairer.

Machine learning is a type of artificial intelligence that allows computers to learn from data rather than being explicitly programmed with rules. Instead of telling a computer "reject anyone from zip codes with poverty rates above 15%," you show the computer thousands of examples of past decisions—applications, outcomes, profiles—and let it find patterns on its own.

The problem is that "letting it find patterns" means the algorithm figures out which factors correlate with outcomes in the training data. And if the training data reflects historical discrimination, the algorithm will learn to discriminate too.

Here's a simplified example. Imagine you're building an algorithm to help a company screen job applicants. You train it on the company's past hiring decisions—ten years of resumes, interviews, and hires. Sounds reasonable, right? The algorithm will learn what "a good hire" looks like based on who the company actually hired.

But what if, for the past ten years, the company has been unconsciously biased against hiring women for technical roles? What if women who applied were held to higher standards, received less friendly interview questions, and were passed o