Loop, graph, or neither

Start here, because skipping this step is the single most expensive mistake in the category. Before choosing between a loop and a graph, establish that you need either one. The honest test is narrow: does the work genuinely exceed what one agent can hold in one context window, end to end?

For most tasks the answer is no, and a single session is not a compromise. It is the correct design. One session keeps the entire transcript available, so the model can see its own reasoning from twenty turns ago. It keeps a stable prompt prefix, so caching works without any effort on your part. It has no handoff, which means no place for state to get lost. Every orchestration pattern in this guide is a way of buying past the context limit, and each one pays for that with a loss of continuity.

The community mood has shifted toward this position. The dominant sentiment in practitioner forums through 2026 has been some version of when not to build an agent, and it is not contrarian posturing. Carnegie Mellon's agent benchmark work found the best-performing agent succeeded at roughly 30% of realistic office tasks, and chaining three agents together produced an end-to-end success rate near 34%.¹ Those numbers are not an argument against agents. They are an argument against adding coordination to a component that is not yet reliable on its own.

A loop runs the agent repeatedly, giving it a clean context each iteration and keeping durable state on disk. The purest form is a shell one-liner that pipes a prompt file into a coding agent forever. The technique circulating under the name Ralph loop comes from Geoffrey Huntley, who was refreshingly honest about it: the loop is, in his words, deterministically bad in an undeterministic world, and building with it requires a belief in eventual consistency.

The foundation under the technique is solid and measured. Models degrade as input grows, and not only when the window fills. The canonical result is Lost in the Middle, which found a U-shaped accuracy curve as relevant information was slid through different positions in a long input: best at the very start or the very end, worst buried in the middle.² Chroma's Context Rot work tested 18 frontier models and found consistent degradation as input length grew, well before the window was full.³ An agent that searches, explores, and backtracks for an hour accumulates noise, and that noise makes every later decision worse.

So a clean context per turn is a real benefit. The cost is the part that gets left out of the reposts, and it is severe.

The cache is the whole economy

Prompt caching only works on an exact prefix match. Anthropic's documentation is explicit that cache hits require identical prompt segments up to the cached block, and advises putting stable content at the very beginning.⁴ OpenAI's caching behaves the same way, on an exact prefix match for prompts over 1,024 tokens.⁵ A cache read costs roughly a tenth of the normal input price.

Now look at what a naive loop does to that. If every iteration rebuilds the prompt from scratch, reads a different set of files, and reconstructs context in a different order, the prefix differs every time and the cache never hits. You pay full price on a large prompt, hundreds of times. This is not a rounding error. It is the difference between a loop that is merely wasteful and one that is ruinous, and OpenAI's own documentation warns about exactly this interaction with context-engineering techniques that rewrite the prompt.

The disciplined version fixes it by construction. The system prompt, the spec, and the standing instructions go at the front and stay byte-identical across iterations, so they cache. Only the volatile tail varies. That is a deliberate design decision, and it is the line between the two versions of this pattern.

Vendors converged on the shape of this independently, which is the strongest evidence the core idea is sound. Anthropic's writing on harnesses for long-running agents describes the problem precisely, as a project staffed by engineers working in shifts where each arrives with no memory of the previous shift, and their finding is blunt: even a frontier model looping across multiple context windows falls short of building a production-quality application when given only a high-level prompt.⁶ The naive loop does not work. Their fix is structural, an initializer agent that sets up the environment once and a coding agent that makes incremental progress while leaving clear artifacts behind.

In late July 2026 a short post about drawing agent workflows as graphs and handing the drawing to a coding agent went past 700,000 views in two days. The recipe was three lines long: draw a graph in any tool, even on paper, send it to the model, and ask it to write a script that implements the workflow. The most-liked reply was a single question, and nobody answered it cleanly.⁷

How is this not a loop?

It is a fair question, and the answer is precise. A loop is a graph. It is a graph with one node and an edge pointing back at itself, where the model decides at runtime whether to take that edge again. The difference between the two patterns is not topology. It is when control flow gets decided.

In a loop, the model chooses the next step during inference, every single iteration. You are paying tokens for routing decisions. In a graph, you decided the routing when you drew it, and the model only does the work inside each node. The edges are code. That single shift moves a recurring inference cost into a one-time authoring cost, and it is the entire economic argument for graphs.

Read that ladder as a trade, not a ranking. Moving down it, you buy determinism and give up adaptivity. A graph cannot discover a step you did not draw. If the shape of the work is genuinely unknown until the agent looks at the results, a graph will either fail or quietly route around the thing you needed it to notice. That is the failure mode people hit when they graph-max a problem that was never a graph.

There is a second, quieter benefit that the viral version undersells. A drawn graph is a review artifact. A colleague can look at it and say the retry edge is missing, or that two nodes should run in parallel, without reading a line of code or a single transcript. Loops do not produce anything a person can review at a glance. Their behaviour only exists in the run.

Every pattern above spends tokens on something other than the work itself. Naming those line items is what lets you predict the bill instead of discovering it.

Routing tokens. In a loop, the model re-reads enough state to decide what to do next, every iteration. That is real input cost spent on a decision, not on output. A graph pays this once, when you draw it. Over a few hundred iterations the difference stops being academic.

Re-grounding tokens. Each fresh context has to be told what it is doing. Spec, conventions, current state. In a disciplined loop this is exactly the part you keep byte-identical so it caches at roughly a tenth of the price. In a naive loop it is the part you pay full freight for, repeatedly, and it is usually the largest single block in the prompt.

Coordination overhead. Multi-agent structures multiply token consumption substantially compared with a single agent on the same task, because context gets duplicated across agents and results get passed between them. This is the cost that makes the reliability numbers in Part I bite: you are paying considerably more to run a structure whose end-to-end success rate is lower than its individual components.

Retry and repair. The one nobody budgets. When a handoff loses information, the next node produces something subtly wrong, and either a human catches it or a later node does. Both paths cost more than getting it right once. This is why the reasoning-on-disk discipline is an economic measure and not a tidiness one.

Premature completion

A later iteration looks around, sees that progress was made, and declares the job finished when it is not. Anthropic names this explicitly as one of two failure modes a naive loop hits.⁶ The tell is a run that terminates early and cleanly with an confident summary that does not match the artifacts. The fix is an explicit completion criterion checked against the spec, not against the agent's own judgement.

Half-built handoff

The other named failure. An iteration runs out of context mid-implementation and leaves the next one a partial feature with no explanation, and the next agent guesses at what happened. The tell is code that looks like two different authors disagreed. The fix is a checkpoint discipline where an iteration either completes a unit of work or explicitly records that it did not.

Silent re-litigation

The subtlest one, and the reason files-as-memory is not sufficient. A fresh instance reads the artifacts, does not see the deliberation that produced them, and undoes a deliberate decision because nothing recorded that it was deliberate. The tell is oscillation: the same file flipping between two approaches across iterations. The fix is writing decisions and their rationale as artifacts.

The undrawable edge

The graph-specific one. The workflow needed a step or a branch that was not in the drawing, so the run either fails at that point or routes around it and produces something plausible and wrong. The tell is a graph that succeeds on every test case you designed and fails on the first real input. The fix is not a bigger graph. It is recognising that this problem belonged in loop territory.