Inside my agentic workflow: spec, fan-out, verify

By Russ. Written 2026.07, from systems I run.

What this is

Not a tools list. This is the shape of how I get real work out of agents without shipping their mistakes. It is the same shape whether I am building a research engine, a resume system, or a document pipeline: write a contract, fan the work out, merge in plain code, and let an eval gate decide what ships.

Two concrete shapes recur. A sequential contract pipeline, where each pass hands a validated object to the next, and a parallel fan-out, where independent agents work at once and their outputs get reconciled at the end. I use both, often in the same project. The rest of this post is the parts that keep either one honest, drawn from three systems I have written up before: the fit judge, an LLM-as-judge with a regression suite; ODI Bot, a 19-pass multi-provider orchestration; and TrialEdge, a production ML pipeline that is still running.

The shape

Read it top to bottom. A spec becomes scoped units, the units run as agents, their outputs converge, a deterministic merge reconciles them, and an eval gate is the only thing that lets work ship. The dashed line is the part people skip: a failed gate loops back and regenerates, it does not wave the work through with a note to fix later.

Spec first, because agents drift

The single change that made agents reliable for me was writing the contract before the prompt. In ODI Bot every one of the 19 passes has an explicit input and output contract: this pass receives this shape, returns this shape, and nothing downstream runs until the shape validates. Generation is JSON-mode with a fallback parser and truncation detection, so a malformed response is caught at the boundary instead of poisoning the next pass. The prompt is the disposable part. The contract is the thing you version, test, and defend.

Spec-driven is not ceremony. It is what lets a pipeline be long without being fragile. A 19-step chain where any step can silently return prose instead of the object the next step expects is a coin flip you run 19 times. Contracts turn that into a chain that either produces a valid object or stops loudly at the exact pass that broke.

Fan-out: scoped, blind, parallel

When the work decomposes into independent pieces, I fan it out. Each agent gets a narrow scope and only the tools it needs, and critically, the agents are blind to each other. On a recent research job I ran ten agents at once, each scoped to a different slice of the source space, each required to verify its own findings against a primary source before returning anything. They did not see each other's work.

The blindness is deliberate. Agents that can see each other converge on the same answer, which means they also converge on the same blind spot. Independent agents fail independently, and independent failures are the ones a merge can catch. Diversity of failure is the entire point of running agents in parallel. Take it away and you have paid for redundancy without buying any coverage. Scope narrow, hand over only the necessary tools, and let them not talk.

The merge is where correctness is won or lost

Four agents return four sets of results and now something has to reconcile them: dedupe overlaps, resolve conflicts, drop the ones that failed their own verification. I do this in plain deterministic code, every time. Keying, set logic, explicit conflict rules. It is boring and it is where the correctness actually lives.

The tempting mistake is to hand the merge to another agent because it is fiddly. That reintroduces exactly the nondeterminism the fan-out was supposed to contain, at the one point where every result funnels through a single decision. Use agents for the judgment that needs judgment, and plain code for the reconciliation that needs to be right the same way every time. A merge you cannot reproduce is not a merge, it is a fifth opinion.

The gate: evals, not vibes

Nothing ships because it looked good in a demo. It ships because it passed a gate. The fit judge is the clearest version I have built: a rubric, a hand-labeled golden set, a regression suite that scores every case and fails the build on a miss. Building it taught me two things I now assume everywhere. The judge disagreed with my own labels twice and was right both times, so the golden set is not automatically ground truth. And a judge at temperature zero was not deterministic, so a single sample near a band boundary is a coin flip. The fix was self-consistency voting: sample an odd number of times, take the median per dimension, and the boundary cases stop flipping.

That gate is now a fixed cost I pay on every system worth trusting. An eval you trust blindly is worse than no eval, because it launders a guess into a number. The gate earns its keep on the cases your cheap checks get wrong, which is exactly the set you cannot see without it.

Reliability is a design feature, not a retry

Reliability gets engineered in at the boundaries, not bolted on as a try-again. ODI Bot regenerates automatically on malformed output, tracks cost per run so a pipeline cannot quietly get expensive, and routes across Claude and OpenAI per pass rather than betting the whole chain on one model. It enforces citations at the claim level, so an unsupported sentence is a caught error, not a thing a reader discovers later. TrialEdge is calibrated and evaluated before deploy and monitored after, which is why it is still running rather than still being debugged.

None of that is glamorous and all of it is the difference between a thing that demos and a thing that runs. The demo proves the happy path exists. The guardrails are what you build for every other path, which is most of them.

Context engineering: the unglamorous 80 percent

Most of the work that decides output quality is deciding what goes into the window. In ODI Bot the retrieval layer is Jina embeddings with reranking, so a pass sees the few passages that matter instead of everything that might. Tools are scoped so an agent is not handed capability it should not use. Context gets trimmed on the way in, not padded.

The model is rarely the bottleneck now. The bottleneck is whether the right facts, in the right shape, reached the agent at the moment it had to decide. Context engineering is unglamorous because it is plumbing, and it is the highest-leverage plumbing there is. I spend more time on what the agent sees than on how I ask it.

What I would tell someone building this way

Six things, blunter than the systems they came from. Write the contract before the prompt, and version the contract. Fan out blind, because independent failures are the catchable ones. Merge in plain code, because the reconciliation has to be right the same way every time. Gate on evals, and let the eval argue with your labels. Engineer reliability at the boundaries, not as a retry. And spend your time on the context window, because that is where the quality actually comes from.

The tools change every few months. This shape has not. It is how I ship work I am willing to put my name on, alone, without a team behind me to catch what the agents get wrong.