iCode Agent · Engineering

The context window is a budget.

A coding agent is mostly a loop: explore, change, run, verify. The loop is cheap to describe and expensive to run, because every step lands in the context window and is paid for again on every request that follows.

Where the tokens go

An agent reading a file, searching the tree, running a test and reading a stack trace produces a lot of output — most of it useful exactly once. Left alone, that history grows until the request no longer fits the window. At that point an agent does one of two bad things: it truncates and loses the thread, or it stalls outright.

Neither is necessary. The output of a tool call is not equally valuable forever. A file you read twenty steps ago is usually summarizable; the result you got one step ago is what the current decision rests on. A context system should reflect that.

Three layers, each doing the cheapest thing

iCode Agent manages context at every boundary, from cheapest to most expensive:

1. Cap at the source. Oversized tool output is capped as it is produced — spilled to a file with a short preview — so a single huge result cannot flood the window in the first place.
2. Reduce at request time. Only when a request would exceed its input budget, older tool results are replaced by compact markers, oldest first, and only as many as it takes to fit. The current turn is protected: it is what the agent is working on.
3. Summarize durably. When history truly overflows, older turns are summarized into one message with a recent tail preserved, so the session continues instead of ending.

Reduction must be reversible

Truncation loses information. The interesting design question is how to reduce without losing the thread. iCode Agent's answer is that a withheld result is not deleted — it is replaced by a structured pointer back:

• A route to the full output. The marker names where the content still lives — the spilled file, or the file a read came from — so the agent can go back for detail instead of guessing.
• Failure evidence is never withheld. Errors and failed commands are exactly the facts a long task depends on, so they survive even when everything around them is reduced.
• Your history is never rewritten by reduction. Reduction is a request-time view only. The saved session keeps every result.

Why this matters for cost

Cost in a coding agent is not one number — it is the size of every request, multiplied by the number of steps. A smaller, well-managed context is not only cheaper; it is also what keeps a long task coherent, because the model is not drowning in stale output.

The goal is not the smallest possible context. It is the smallest context that still answers the question — with a way back to the rest.

The mechanics described here are implemented in iCode Agent's context layer; see the project repository.