Skip to content

Retrieval

The step that finds the material a model will answer from.

Retrieval is the search step in front of a model: given a question, find the passages that contain the answer. It exists because a model cannot know what it has not been given, and because what it was given may be out of date.

Retrieval has three stages that are usually collapsed into one word. Recall: gather every candidate that might be relevant. Rerank: order the candidates by how well they actually answer the question. Generate: let the model write using the top of that list. Quality problems at the first stage are invisible at the last, which is why "the model made it up" is often really "the right document never made it into the context".

The most useful diagnostic is to look at what retrieval returned before looking at the answer. If the answer is wrong and the source is missing, the problem is retrieval. If the source was there and the answer ignored it, the problem is attention.

Related terms

Want this working inside your team?

It starts with one conversation about your workflow.

Ask on WhatsApp