Lulu's World

June 30, 2026 — by Lulu Liu in Uncategorized

I. The False Hope for a Code Patch

When a biased algorithm makes the news, the public response follows a predictable script: outrage, a press statement, and the assumption that somewhere, engineers are already fixing it. It sounds simple enough, right? Just patch the code, swap out the bad idea, and boom. But what surface-level CS leaves out is that there is no patch. Not because engineers don’t care enough to try, but because the main problem is not just a bug but a theorem. In machine learning, “fairness” isn’t a single setting you can toggle on and off — it’s a collection of mathematical definitions that are incompatible with each other. As Harvard’s Berkman Klein Center explains, this creates an inherent structural tension that no amount of clever engineering can completely resolve.

II. A Statistical Reality Check

Consider a university admissions algorithm that assigns each applicant a “success probability score.” Intuitively, we would want it to be fair in such ways:

Calibration: A score of 80% should mean the same thing regardless of who the applicant is. (e.g., two students from different backgrounds with the same score should have the same likelihood of graduating)

Error-Rate Balance: The algorithm shouldn’t make mistakes unevenly, rejecting qualified applicants from one group at double the rate it rejects qualified applicants from another.

Both sound reasonable, but both are mathematically incompatible. The Fairness Impossibility Theorem shows that an algorithm cannot satisfy both definitions simultaneously when baseline rates differ across groups. In a world shaped by historical inequity, those base rates almost always do, meaning that optimizing for one inherently breaks the other.

III. The Real-World Clash and The Case of COMPAS

This isn’t just theoretical. In 2016, ProPublica published an investigation into COMPAS, a recidivism risk algorithm used across the U.S. to inform bail, sentencing, and parole decisions. Their finding says that the algorithm was nearly twice as likely to falsely flag Black defendants as future criminals compared to white defendants. The company opposed, arguing that a risk score of “7” carried the same meaning for both racial groups. The calibration was intact.

Both sides were mathematically correct. ProPublica measured error-rate balance, and COMPAS measured calibration. Two different, equally valid definitions of fairness resulted in opposite conclusions from the same data. Neither side lied. It was the impossibility theorem playing a role in real people’s lives in the courtroom.

IV. The Human Burden

Because the math cannot resolve its own contradictions, humans are the ones to step in and make a choice: which type of unfairness are we willing to live with? That is beyond a mere technical decision. And right now, most people affected by these systems don’t even know the choice is being made on their behalf.

Harvard’s Data-Smart City Solutions Framework documents describe this struggle, showing how local governments deploying automated tools for housing, public safety, and resource allocations are silently navigating these trade-offs without any public input or awareness. When a city chooses the fairness metric, it is choosing who bears the cost of the algorithm’s unavoidable errors, and that choice deserves public transparency beyond technical specification.

V. Rejecting the Automated Absolution

The math is limited, and that limitation carries a high cost: we can no longer use algorithms as a form of “absolution,” outsourcing complex social choices to a machine while pretending the results are fully objective. If no model can be perfectly fair, then no automated output should ever be treated as the final verdict. Working with these equations on a daily basis has taught me a lesson. The most important skill in such fields isn’t just knowing how to build a “perfect model”; it is knowing when to question one.