We Mined 99 of Our Own Sprint Runs. Here's Where Agents Actually Trip Up.

For the last two months, Sprint Engine v1 built Sprint Engine. Ninety-nine sprint runs, from single-feature slices to a 48-task platform build, all executed by agent teams against our own repo. Every run recorded its own telemetry: reviewers logged structured error counts with every verdict, agents filed friction reports when something slowed them down, and each task carried its evidence trail.

When v2 replaced the v1 run store, we archived all of it and did what we'd want any team to do with two months of agent history: mined it. This post is the result — the totals, the failure taxonomy, where agents actually trip up, and how the data changed what v2 became.

The dataset

The archive spans May 17 to July 6, 2026: 889 tasks (827 completed), 1,986 review-gate attempts by five reviewer roles, and 2,442 recorded assessments — 1,942 attached to reviewer verdicts and 499 agent self-reports. Along the way the agents ran 33,702 commands and touched 17,519 files.

What five review gates found

Every completed task in v1 passed through up to five reviewer roles — code, spec, structure, UI, and testing — and each verdict recorded what the reviewer found, by type:

Two things in this chart are worth sitting with. First, the top category is not a coding failure. Missed requirements — a tab that never got built, a state that never renders, an acceptance criterion quietly skipped — outnumber implementation mistakes. Agents are better at writing code than at noticing what they didn't write.

Second, hallucination is rare in this setting. Nineteen hallucinated claims in twenty thousand checked is 0.09%. When an agent works inside a repo, runs real commands, and knows a reviewer will re-run them, fabrication mostly disappears. The errors that survive are subtler: a claim that was true of the code before the last change, a test that passes for the wrong reason.

Three reviewers do almost all the rejecting

The tester's 5% is not leniency — it's a verification gap. Testers could rarely drive the built app (more on that below), so they fell back on evidence the implementing agent had already produced. A reviewer who can only re-read what the worker claimed will mostly agree with the worker. The reviewers with independent means of checking — re-running suites, re-reading diffs against the spec — are the ones who caught things.

What 537 rejection demands were about

When a reviewer rejected, they listed concrete required actions. Categorizing all 537 of them:

• Reviewers treated documentation drift as blocking, not cosmetic — changing a documented contract without updating the docs in the same publish drew 62 rejection demands.
• Tests built on fakes got rejected systematically. A suite that exercises a synthetic fixture instead of the production path proves the fixture works.
• A recurring pathology: commits that don't stand alone. An agent leaves a new file untracked, its published evidence shows green checks against the dirty worktree, and the committed code fails on a clean checkout. Reviewers caught this only by re-running from scratch.
• Security findings clustered on trust boundaries — path traversal via normalize-before-validate, fail-open defaults — rather than classic injection.

Where agents actually trip up

The richest corpus in the archive is 1,057 friction reports — agents describing, in their own words, what slowed or blocked them. Categorized:

The single most repeated blocker across the whole archive was the absence of a live app-automation harness. UI acceptance criteria that say "the panel renders and the button works" can't be proven from source inspection — so agents fell back on typecheck, focused tests, and reading the code, and logged the gap as residual risk. It became the most common approval caveat and the root of most test-gap findings.

The second-biggest category is the humbling one: the engine's own coordination. Repeated dispatch reminders, gates firing on tasks with no matching surface, lifecycle ambiguity at sign-off — 166 reports of the orchestration itself getting in the way. Add the 153 reports of truncated or stale review evidence and the 121 of sibling tasks' uncommitted files contaminating repo-wide checks, and a large share of all friction was overhead we built, not work the agents couldn't do.

The control group is instructive too: 29 reports say "no friction at all" — and nearly every one credits the same thing, a task card that named the exact files, the exact change, and the exact verification command. Precision in, autonomy out.

Approvals read like audit reports

We also classified the tone of all 1,610 approvals. The distribution says a lot about what a healthy review culture looks like:

What earns praise is consistent: reviewers who re-ran the tests themselves rather than trusting the summary, acceptance criteria walked line-by-line to the code, tight diffs that changed nothing they didn't have to. What gets waved through with caveats is equally consistent: "no live app run performed", pre-existing failures disclosed as out of scope, residual risk formally recorded and deferred. The caveats are almost never doubts about the work — they're disclosed limits of what could be verified.

Better rosters, fewer mistakes

The last 13 runs in the archive recorded which model ran each role — a curated roster with the most capable models on the architect and reviewer seats. Reviews of that work found errors at roughly a third of the earlier rate:

We won't over-claim this one: model attribution only exists for the July runs, so "better model" and "better process" arrive together in the data. But the direction matches everything else we've measured, and v2 records the model behind every role on every run — so this comparison gets clean from here on.

What this data did to Sprint Engine v2

Here's the finding that drove the redesign. 79% of review gates passed on the first attempt. Five reviewer roles examined nearly every task, and four times out of five, all that machinery confirmed the work was already good. The gates caught real bugs — 138 of them — but the pipeline paid for review on everything to catch problems in a fifth of it. Meanwhile the friction data shows the coordination itself — dispatch churn, gate context, evidence plumbing — was one of the biggest taxes on the agents.

So v2 is a much lighter engine. It keeps what the data says earns its cost and drops the ceremony that didn't. Working with it should feel like using your coding CLI directly — the same way you'd run Claude Code on a goal — except the sprint fans out across many agents and terminals at once, and every task has one accountable owner instead of a five-gate pipeline.

• One owner per task, not five gates per task. Review concentrates where the data says it changes outcomes — the plan and the risky changes — instead of re-confirming work that passes four times out of five.
• Far fewer tokens per task. The v1 pipeline re-read context across every reviewer seat; cutting the choreography cuts the spend, and sprints finish faster.
• Mixed models in one sprint, recorded on every run. The July pattern is the default posture: the most capable model spearheads the architecture and the review that remains; cheaper, faster models carry the implementation volume.
• Task cards do the work the friction data demanded: exact paths, exact acceptance criteria, exact verification commands. The 29 zero-friction reports all pointed here.
• Evidence without archaeology: task-scoped diffs attached to review context, so nobody re-derives what changed from a truncated dump.

The dataset that taught us this is exactly the kind of record we think agent teams should leave behind by default. Not a transcript — a ledger: what was claimed, what was checked, what was found, and what it cost. Two months of v1 leaving that ledger behind is why v2 exists.

The most expensive thing in the data wasn't agent mistakes — it was coordination that didn't change the outcome. v2 keeps the judgment and drops the ceremony.