Evaluation
Evaluasi
Measuring whether the system does the job, on a fixed set of cases, repeatedly.
Evaluation is running a fixed set of cases through the system and scoring the output against expectations. Without it, every change to a prompt, a model, or a retrieval setting is a guess.
The useful hierarchy runs from cheap and frequent to expensive and rare. Unit tests check a function. A golden dataset of real user questions, scored by a program, catches regressions in seconds. An LLM judge scores open-ended answers against a rubric. A human panel reads a sample and decides whether the product is actually good.
Two rules keep evaluation honest. The set must be held out from the cases you iterate on, or you will measure memorisation. And the score must come from the real use case: an answer that is factually perfect but written for a different audience is still wrong.
Related terms
Benchmark
A shared test that lets you compare systems. Useful, and easy to over-trust.
LLM Judge
Using a model to score another model's output against a rubric.
Golden Dataset
A fixed, reviewed set of real cases the system is measured against.
Regression
A change that made something worse. The reason to keep tests at all.
Guardrails
Checks that stop or constrain an action before it happens.
Want this working inside your team?
It starts with one conversation about your workflow.