Non-determinism
Non-determinisme
The same input can produce a different output each time. Normal, and worth designing around.
Non-determinism means the same request does not always give the same answer. It comes from sampling during generation: the model picks among several plausible next tokens, and a small numeric difference in the logits can flip a choice.
This is normal, not a defect, and any system with a temperature above zero has it. It becomes a problem only when something downstream assumes exactness: a test that expects a fixed string, a diff that must be empty, a parser that requires an exact format.
The fix is rarely to force the model to be deterministic. It is to make the output robust: ask for structured output, validate it, and treat the free-text part as prose while the structure carries the meaning. Where exactness truly matters, move the logic out of the model and into code.
Related terms
Temperature
The dial for randomness. Low is repeatable, high is surprising.
Evaluation
Measuring whether the system does the job, on a fixed set of cases, repeatedly.
Structured Output
Making the model answer in a fixed shape a program can read.
Regression
A change that made something worse. The reason to keep tests at all.
Want this working inside your team?
It starts with one conversation about your workflow.