Cache Tokens
Token Cache
Input tokens the provider cached from a previous request, billed at a much lower rate.
Cache tokens are input tokens the provider has already processed and stored, so it does not have to process them again. When consecutive requests share a prefix, the provider reuses the work and bills that portion as cache tokens, often at a tenth of the input rate or less.
The reason this matters is how sessions are billed. The model is stateless, so every request resends the entire conversation. By turn fifty, each request carries fifty turns of history, and you would pay full rate on all of it every time. The cache changes the arithmetic.
The cache is fragile in one specific way: it matches exact prefixes. If anything changes earlier in the conversation, the harness reorders content, a timestamp updates, a file representation shifts, the cache misses from that point onward and everything after it is billed at full input rate. Caches also expire after a few minutes of inactivity, so a session resumed after a long pause re-pays its history once. When a session's cost jumps without an obvious cause, compare cache tokens to input tokens in the usage report.
Related terms
Prefix Cache
The provider-side store that lets consecutive requests skip re-processing a shared prefix.
Input Tokens
Tokens the harness sends on each request. Billed at a lower rate than output tokens.
Cost per Token
What you pay for each unit of text the model reads or writes.
Model Provider Request
One round-trip from the harness to the model provider. The unit that tokens are billed on.
Caching
Storing results so the same work is not repeated, and the cost drops with it.
Want this working inside your team?
It starts with one conversation about your workflow.