Skip to content

Bias

A systematic skew in what a system produces, inherited from the data it learned on.

Bias in an AI system is a pattern in its outputs that reflects something about its training data rather than about the world. Ask for a picture of a doctor and it will produce one that matches the demographics of the images it saw, not the ones its users are.

The sources are ordinary: the data was collected unevenly, the labels encode the annotators' assumptions, and the objective rewarded the most common pattern rather than the correct one. None of these require anyone to be prejudiced.

Bias is not fully removable and pretending otherwise is its own failure. What is achievable is measuring it on your own use case with your own data, documenting what you found, and deciding which skews you can accept for a given decision. A ranking system for loan applications and a system that summarises meeting notes do not deserve the same standard.

Related terms

Want this working inside your team?

It starts with one conversation about your workflow.

Ask on WhatsApp