Documented cases of algorithmic bias include a tutoring company's hiring software, a student lender's AI underwriting, a criminal risk score and facial recognition used by police and a retailer. The outcomes differ: some ended in settlements, some in policy changes, one is still in court and one regulator cleared the company. Each case below says which it is and links the primary source.
01How to read these cases
Three labels matter. A finding is a conclusion by a court or regulator. A settlement ends a dispute and is not always a finding. An allegation is a claim that has not been proven. We say which applies to each case. This page is general information, not legal advice. The cases are all from the United States, where the primary documents are public, so they do not show the whole global picture.
02Hiring: age cut-offs and a pending class case
The US Equal Employment Opportunity Commission (EEOC) alleged that software used by iTutorGroup automatically rejected women aged 55 or older and men aged 60 or older, affecting more than 200 qualified US applicants. The company agreed to pay $365,000, adopt a new anti-discrimination policy and train hiring staff, with the EEOC monitoring for at least five years. The EEOC also said the company has stopped hiring tutors in the United States. The release is dated 11 September 2023 (EEOC). This is a settlement, not a court finding.
In Mobley v. Workday (US District Court, Northern District of California), the plaintiffs allege that Workday's AI recommendation system discriminated against applicants by race, age and disability. On 16 May 2025 the judge let the age claim go forward as a preliminary collective action, reasoning that the plaintiffs had substantially alleged a single policy of using the system to score, sort, rank or screen applicants (court order). That ruling only lets similar applicants be notified.
A 22 June 2026 order left several claims in place, including California-law claims. The ruling did not decide whether the tools discriminated, and the allegations remain unproven (Duane Morris summary).
03Lending: a settlement and a clearance
On 10 July 2025 the Massachusetts Attorney General announced a $2.5 million settlement with student lender Earnest Operations. The Attorney General alleged the company failed to address the risk that its AI underwriting models, used for eligibility, terms and pricing, harmed Black, Hispanic and non-citizen applicants. The deal is an Assurance of Discontinuance, a formal agreement with the state rather than a court order, and it requires written policies, risk assessments, testing, model inventories and an oversight team (DLA Piper summary). It is a settlement of allegations, not a court finding.
The Apple Card is a counter-example. After public complaints in November 2019 that women got lower limits, New York's financial regulator analysed about 400,000 applicants. Its 23 March 2021 report found no unlawful discrimination and said the decisions were explainable. It also said poor customer service and a lack of transparency damaged trust, and urged modernised credit scoring and updated laws (NYDFS).
04Policing and security: a risk score and face matches
In 2016 ProPublica analysed the COMPAS risk tool for more than 7,000 people arrested in Broward County, Florida. It reported that 44.9% of Black defendants were labelled higher risk but did not reoffend, against 23.5% of white defendants. It also reported that white defendants were more often labelled lower risk and then did reoffend, 47.7% against 28.0% for Black defendants (ProPublica). The maker, Northpointe, disputed the analysis, and both sides argued over the statistics (ProPublica's response). The dispute is unresolved in these sources.
In January 2020 Detroit police arrested a man at his home after a facial recognition search of a poor-quality surveillance image returned a false match. The parties settled the lawsuit on 28 June 2024. Police now need independent evidence before arrests based on a match, with training and an audit of cases since 2017 (ACLU case page).
On 19 December 2023 the Federal Trade Commission (FTC) alleged that Rite Aid's facial recognition system, used from 2012 to 2020, produced thousands of false matches, with more false positives in stores in mostly Black and Asian communities, and that the company did not test its accuracy. The FTC's proposed order bans the use of facial recognition for surveillance for five years (FTC).
05What the cases have in common
Some patterns repeat:
- Regulators used existing laws on age discrimination, fair lending and consumer protection, not AI-specific ones.
- Missing testing appears often. The FTC said Rite Aid did not test accuracy, and the Massachusetts settlement requires testing.
- A person acted on the machine's output in the Detroit arrest, so human checks matter.
- Investigations can also clear a company, as New York's did.
?What is not known
- Mobley v. Workday is unresolved. The June 2026 ruling decided what claims could proceed, not whether discrimination occurred.
- We did not confirm whether the FTC's proposed Rite Aid order has since been finalised.
- Researchers and companies still disagree about which fairness statistic fits tools like COMPAS.
QQuestions people ask
Is a settlement an admission that the algorithm was biased?
Not necessarily. A settlement ends a dispute, and the agreement itself says what was admitted. Read the primary document, and ask a lawyer about any legal question.
Why include a case where the company was cleared?
A fair picture needs both. The New York regulator tested the complaint against data and found no unlawful discrimination, while still criticising transparency.
Last checked October 24, 2026. Rules and settings change, so open the source to see the current version.
- iTutorGroup to pay $365,000 to settle EEOC discriminatory hiring suit US Equal Employment Opportunity Commission The 11 September 2023 press release; we used the allegation, the settlement amount and the decree terms.
- Mobley v. Workday, Order Granting Preliminary Collective Certification US District Court, Northern District of California (copy hosted by Akin Gump) The 16 May 2025 order; we used its description of the claims and the reasoning on a single policy.
- California federal court grants in part and denies in part Workday's motion to dismiss Duane Morris LLP A law firm summary of the 22 June 2026 order; we used it for the case status.
- State action targets use of biased AI underwriting models DLA Piper A law firm summary of the Massachusetts settlement; we used the amount, date, allegations and terms.
- DFS Issues Findings on the Apple Card and Its Underwriter Goldman Sachs Bank New York State Department of Financial Services The 23 March 2021 press release; we used its findings and recommendations.
- Machine Bias ProPublica The May 2016 analysis of the COMPAS risk tool; we used its figures.
- Technical Response to Northpointe ProPublica ProPublica's reply to the tool maker; we used it for the outline of the dispute.
- Williams v. City of Detroit case page American Civil Liberties Union The ACLU's account of the false facial recognition match and the 28 June 2024 settlement; we used both.
- Rite Aid banned from using AI facial recognition US Federal Trade Commission The 19 December 2023 press release; we used the allegations and the proposed order.
General information, not legal, medical, financial or investment advice. We are not affiliated with any company, agency or standards body named here. See our editorial policy and corrections, or tell us about a mistake at [email protected].