Unit 8 · AI & the Impact of Computing · Lesson 8.4
Computing Bias
Your 8.3 program is 90% accurate and fails most of the people it was supposed to serve. Nobody meant for that to happen. That’s exactly why this lesson exists.
Big Idea 5 — Impact of ComputingCED 5.3
On this page: AP-TESTABLE sections marked with a BI5 tag · CONTEXT — NOT ON THE AP EXAM anything with this label. Exam-tagged questions never test the context material.
Before this lesson
No reading — this page is the reading. Retrieve 4.8: a heuristic decides what “good enough” means. Somebody chose that.
Impact. Bias isn’t a bug in one line of code. It can enter at every step, from the data to the decision.
2
The Hook
“Your program is 90% accurate.”
“…and it doesn’t work for a quarter of the people it will be used on.” The 8.3 result,
still on your screen. Who should have caught it — and how?
3
The Idea
AP-TESTABLE
Computing bias: computing innovations can reflect existing human biases, through the data they
use or the way they’re designed. Bias can enter at every level of building software. Three places to look:
Source
What it means
You’ve seen it
The data
training or input data under-represents a group or carries past unfairness
8.2’s lighting, 8.3’s mangoes, 5.10’s potholes
The design
a choice about what to measure or optimize favors some people
4.8: somebody decided what “best” means
The use
results are used for a purpose, or on people, they weren’t built for
a model trained on one city, deployed in another
Bias is often unintentional — so it has to be looked for on purpose. It doesn’t
announce itself; the overall accuracy looked great. Programmers should take action to reduce bias: test with
representative data, check results group by group, and ask who the system might fail.
Real cases, briefly (your teacher has the details):
Hiring: a résumé screener trained on a company’s past hires was reported to
learn the company’s past preferences — including downgrading résumés that mentioned
women’s organizations. Source: the data.
Face analysis: researchers testing commercial systems found much higher error rates for
darker-skinned women than for lighter-skinned men. Source: the data used to build and test them.
Health care: a widely used risk score used past medical spending to stand in for
medical need. Groups who had less access to care had spent less, so their needs were underestimated.
Source: a design choice about what to measure.
Every answer names a mechanism. “The AI is biased” earns nothing.
“It was trained on past hires, who were mostly one group, so it learned to prefer that group” is the
answer.
4
Try It — Name the Source
Each system has a bias problem. Is it coming from the data, the design, or the use?
5
Vocabulary
computing bias Impact
When a computing innovation reflects or amplifies human biases, through its data, its design, or how it is used.
unintentional bias Impact
Bias no one meant to build in — which is why it has to be tested for deliberately.
algorithmic decision-making Impact
Using a program’s output to make or guide decisions about people — hiring, loans, school placement.
6
Check
Five questions. Every correct answer names a mechanism.
0 of 5 answered
7
Impact Check
Who checks?
Your 8.3 model looked great until someone split the results by group. In a real
company, whose job is that — the programmer, a tester, the company buying the product, a government
regulator, the people affected? Pick one and defend it in two sentences.
8
Connections
Came from 8.3 — the demonstration. Today it gets a name and three sources.
Came from 4.8 — heuristics: a design choice about what “good enough” means.
Next in 8.5 — you audit a real system with a structured worksheet.
Returns on the exam — “identify a source of bias” — with a mechanism, every time.