Next-token Prediction
Prediksi Token Berikutnya
What the model actually does. Samples one next token from the context, appends it, and runs again.
Next-token prediction is what the model actually does. Given a context, it samples one next token, appends it, and runs again. Every output, a sentence, a tool call, a thousand-line file, is built one token at a time. The model has no other mode of operation.
Each step works the same way. The tokens in the context window run through the parameters, producing a probability for every token in the vocabulary. One token is sampled from those probabilities, appended, and the loop runs again with a slightly longer context. That sampling step is why the same prompt produces different output on different runs. Non-determinism is built into the mechanism, not a bug layered on top.
Holding onto this explains behaviour that otherwise looks strange. The model never checks whether a token is true before emitting it, only whether it is likely, which is the root of hallucination. It commits to each token as it goes, so a confident opening sentence can steer the rest of the answer wrong. And because output is produced strictly one token at a time, generation speed puts a floor on how fast any agent can work.
Related terms
Token
The unit a model reads and writes. Roughly word-sized, but not exactly.
LLM
A model trained on a huge amount of text to predict the next token. The engine behind most AI products.
Hallucination
Output that is fluent and wrong. Two different problems hide under one word.
Non-determinism
The same input can produce a different output each time. Normal, and worth designing around.
Inference
Running a trained model to produce an answer. What happens on every request.
Want this working inside your team?
It starts with one conversation about your workflow.