Attention Budget
Anggaran Atensi
Each token has a finite amount of influence to distribute across the rest of the context.
The attention budget is the finite amount of influence each token has to distribute across the rest of the context. It is per token: it does not grow when the context grows. Every token in the window competes for the same limited capacity, and what one token gains, others lose.
This is the mechanism behind attention degradation. When the context holds a thousand tokens, each of them has a thousandth of the influence it would have had alone, so the instruction you wrote at the top of the file is competing with everything that came after it.
It also explains why adding context is not free even when it fits. The window can have room to spare and the context still get worse, because the budget is about share, not about space. The practical reading is a budget you spend, not a bucket you fill: every token you add takes a slice from the ones that matter, so the question is never whether the context is big enough, but whether every token in it earns its slice.
Related terms
Attention
The mechanism that lets each token weigh the relevance of every other token.
Attention Degradation
Each token has a limited influence to spend, and later tokens crowd out earlier ones.
Context Window
Everything the model sees on one request. Finite, and it decides what fits.
Attention Relationship
The pairing between two tokens. Meaningful pairs influence each other more than unrelated ones.
Context
The relevant information the agent has access to right now. A measure of quality, not quantity.
Want this working inside your team?
It starts with one conversation about your workflow.