Attention
The mechanism that lets each token weigh the relevance of every other token.
Attention computes, for each token, a set of weights over all other tokens: how relevant is each one to me right now. The transformer turns those weights into a weighted combination of values, so each token's representation is rewritten to include the context it attended to.
Because the mechanism is a competition over a fixed capacity, later tokens can crowd out earlier ones. This is why the same instruction given at the top of a long context has less effect than the same instruction given at the bottom, and why recency is such a strong effect in chat models.
Attention is also the part most often misread as reasoning. It is a routing mechanism over context, not a search over facts.
Related terms
Transformer
The neural network design behind nearly every modern language model.
Context Window
Everything the model sees on one request. Finite, and it decides what fits.
Attention Degradation
Each token has a limited influence to spend, and later tokens crowd out earlier ones.
LLM
A model trained on a huge amount of text to predict the next token. The engine behind most AI products.
Want this working inside your team?
It starts with one conversation about your workflow.