Context Engineering Is the New Bottleneck

Somewhere in the last eighteen months, the interesting problem in building with language models quietly moved. It used to be phrasing: how do you word the instruction so the model does the right thing. That problem still exists, but it is no longer where projects fail.

Projects fail now on context. The model is capable. The prompt is fine. The agent still does something stupid on step nine of a twelve-step task, because by step nine it is reasoning over a window stuffed with forty tool results, three stale file versions, and a conversation history that no longer reflects what is true.

This has a name now — context engineering — and it is worth taking seriously, because the failure it describes is not a model problem. It is an architecture problem, and architecture problems are ours.

The window is a budget, not a container

The first mental shift is refusing to treat the context window as storage.

A large window invites a lazy habit: throw everything in, let the model sort it out. This feels reasonable, and it works in demos, where tasks are short. It stops working when tasks get long, for reasons that are structural rather than mystical.

Every token you add competes with every other token for the model's attention. Add the entire file when the function was enough, and you have not been generous — you have diluted the part that mattered. Add the result of a tool call that has since been superseded, and you have not preserved history, you have planted a contradiction. The model now has two versions of the truth and no principled way to prefer the newer one.

So the window is a budget. Every token in it should be there because it earns its place for the next decision, not because it was relevant at some earlier point. That reframing does most of the work.

Four things that actually fill the window

When I look at an agent trace that has gone wrong, the bloat is almost always one of four kinds.

Raw tool output. A database query returns two hundred rows; the agent needed the count and three examples. A file read returns nine hundred lines; the agent needed one function. The fix is not cleverness, it is a boring transformation layer between the tool and the context: summarize, truncate, project. The tool returns what the model needs to decide, not everything the tool knows.

Dead conversation. Turns three through eleven were about an approach that got abandoned in turn twelve. They are still sitting there, arguing quietly for a path the agent already rejected. Long-running agents need compaction — periodically replacing a stretch of history with a short statement of what was concluded and what remains open.

Tool definitions nobody will use. Twenty tool schemas, of which the current task can plausibly touch four. Each unused definition is a permanent tax on every single turn. Scoping the toolset to the task — even crudely, by phase — often buys back more room than any prompt tightening.

Retrieved documents that were never checked. A retrieval step returns five chunks ranked by similarity. Similarity is not relevance. If nothing filters them, the agent reasons over two useful chunks and three plausible distractors, and distractors are worse than nothing because they look like evidence.

What I do about it

The pattern that has held up for me is to stop thinking of context as a transcript and start thinking of it as a working set — the smallest complete description of the current state of the task.

Concretely, that means an agent's context gets assembled fresh for each phase rather than accumulated. There is a stable part: the instruction, the constraints, the definition of done. There is a rotating part: the three or four facts that matter right now. When a phase ends, its raw material gets discarded and only the conclusion survives.

This costs something. You have to decide what a conclusion is, which means you have to actually understand the task decomposition rather than delegating that understanding to the model's attention mechanism. That is the work. Context engineering is mostly the discipline of making that decision explicitly rather than by default.

A concrete version

Take a task I actually care about: an agent that helps diagnose a failing payment webhook. Naive version — give it database access, log access, the codebase, and the incident description, then let it explore.

What happens is predictable. It reads a log file, which is enormous. It queries the payments table, which returns a wall of rows. By the time it forms a hypothesis, the evidence that would confirm or kill that hypothesis is buried under the noise it generated getting there.

The engineered version splits it. Phase one: the agent gets the incident description and a single tool that returns aggregate failure counts grouped by provider and error code — a table with maybe six rows. From that it forms a hypothesis. Phase two starts with a fresh window containing the hypothesis, not the exploration, plus a tool scoped to fetching individual event records for one provider. Phase three gets the hypothesis and the confirming records, and writes the summary.

Same model, same underlying data. The difference is that at no point is the model asked to hold three hundred rows in its head while reasoning about causality.

The part I hold loosely

I am wary of turning this into an orthodoxy, because a lot of it is a response to current limitations rather than a permanent truth. Windows are getting bigger and attention is getting better at long ranges. Some of what I describe as discipline will look like superstition in three years, the way manual memory management looks to someone who has only written Python.

But some of it will not, and I think the durable part is this: deciding what information a decision requires is a design activity, and design activities do not get automated by making the substrate faster. A bigger window means you can afford to be sloppier. It does not mean sloppiness stops costing you — it means the cost shows up later, at a scale where it is harder to debug.

What I would say with more confidence: if your agent works on short tasks and falls apart on long ones, do not start by changing the model or rewriting the prompt. Print the context at the step where it broke and read it as if you were the model. Most of the time the answer is embarrassingly visible — the thing it needed was there, buried under nine things it did not.

Further reading: Anthropic 2026 Agentic Coding Trends Report; Sourcegraph, "Context Engineering: A Practical Guide for AI Agents." Neither is cited above — the argument here is my own, and these are where I would go next.