When Agents Write the Code, Review Becomes Asking Which Check Catches Which Failure

Once agents start writing code at speed, the first thing to break is code review. Anthropic's monorepo merged around 2,500 changes a day in May 2026; by September, that number had passed 30,000. At that scale, "a human read every line" is no longer realistic. Clinging to it as the goal only burns reviewers out faster.

My view now is that we have to change the question code review asks. Stop asking "did a human read this?" Instead, ask: "How could this change fail, and which layer of checks—automated or human—is best positioned to catch it?" When working with an agent, trust also has to rest on the evidence it hands back, not on how many rounds you spent chatting with it.

Human Time Is a Two-Sided Cost

The Claude Code team treats human time as a two-sided cost: the time spent reviewing PRs, and the time spent fixing broken code. If you compress the first without bringing down the second, you haven't actually saved any human time—you've just moved it from review to firefighting.

That's why their goal is stated as two sentences tied together: across Claude Code repositories, most PRs land with at most one human in the loop (which can be the author), while revert rates and incidents do not go up. The second half is the guardrail. If the guardrail metrics degrade, human effort has simply been shifted somewhere else.

The result: between June and September, the number of PRs merged into Claude Code more than doubled, while the share of PRs requiring a revert dropped by over 40%. They didn't stop doing code review. Instead, they handed most of the "reading code" work over to automated pipelines and invested heavily in making rollbacks fast and painless.

Demand Proof Before Review Even Starts

The heavy lifting of review actually happens before a PR is even opened. Don't let an agent just say "done." Make it produce visible proof: run tests, execute builds, compare screenshots against design mocks. Then have a sub-agent with a clean context review the diff. Boris Cherny, the creator of Claude Code, estimates that giving Claude a verification loop improves output quality by roughly two to three times.

Thorsten Ball puts it even more bluntly in *How I work with Agents*: demand irrefutable proof. End-to-end demos, screen recordings, before-and-after benchmark numbers—these are things agents are great at producing. He also suggests explicitly spelling out "what does not count as proof" in AGENTS.md.

The companion rule is to stop playing trial-and-error ping-pong with the agent. Don't say "try this... no, try that." Have it generate 20 variants at once along with a small page where you can mix, match, and compare them. When debugging a performance regression, have it propose 4 hypotheses for the slowdown and verify all 4 one by one. Let the iteration happen inside experimental tooling the agent builds for itself, not across your chat turns.

There is another prerequisite here: whatever an agent cannot see, it will hallucinate. Tool use suppresses hallucinations precisely because it puts the agent in contact with the real environment. So anything a human can see—production logs, deployment status, release pipelines, feature flags, bug reports—should be made visible to the agent as well whenever possible.

A Single Change Passes Through Multiple Layers

On the Claude Code team, a change traveling from the author's dev environment to production goes through roughly six layers:

1. Pre-PR verification: running tests, builds, and screenshot comparisons, plus internal skills for bug hunting, shrinking diffs, and end-to-end verification.
2. AI review on every PR before a human looks at it. Multiple agents hunt for bugs in parallel and vote to filter out false positives, while pure style nits are stripped out completely.
3. Blocking security review—any open issue blocks the merge.
4. Blocking automated checks: linting, typechecking, cross-platform tests, and builds.
5. Risk-tiered human approval.
6. Merge queues, feature flags, and staged rollouts.

Beyond those six layers sits the final line of defense: fast detection and fast rollback.

Every layer misses some issues, but very few bugs can punch through all of them at once. The most valuable takeaway here is the principle behind layer five: review tiers are determined by the change itself, not by who wrote it. Updating docs and updating the auto-updater are nowhere near the same risk level. Billing logic, CI configurations, and paths marked as sensitive by code owners always require human approval. A bot's approval cannot substitute for a human's, and any new commit automatically invalidates prior approvals.

When the same class of issue keeps getting flagged repeatedly, fix it at the source: update a skill, add a lint rule, or update CLAUDE.md and the REVIEW.md file read by reviewers, rather than counting on a human to catch it by re-reading code every time.

Agent-on-Agent Review Must Converge

When an agent writes the code and an agent reviews the code, it is very easy to get stuck in an infinite loop. They ran into a CLI PR that ping-ponged through more than 50 rounds overnight before a human manually stepped in to stop it. Checking git blame revealed that roughly 59% of the issues repeatedly flagged by the review system had already been there in round one. It was barely finding new bugs; it was just unearthing a few more old ones each round.

It takes a human one second to dismiss a nit, but handing it to an agent turns it into a new commit and another full round of review. To fix this, they changed the rules: after three rounds, the author is asked to step back and re-evaluate the overall approach; after the first few rounds, only new diffs are reviewed; flagged issues must state the broader category of error so the author can fix the entire class of problems in one pass; and previously raised issues are never repeated.

What Humans Stay Around to Do

Humans no longer read trivial diffs line by line, but several responsibilities still belong strictly to humans:

• Deciding whether the thing should be built at all. Direction, taste, and product goals are set by people.
• Reviewing high-risk code: billing, auto-updaters, CI workflows.
• Reviewing architecture, because architectural choices become the conventions everyone else follows next.
• Checking the proof: test outputs, commands run, screenshots, review comments. If you can't articulate what those checks might miss, you shouldn't casually drop a 👍.
• Taking accountability. Every change that ships must have a specific human owner, regardless of who typed out the code.

The Claude Code team calls this "You land it, you own it": whoever merges the change is responsible for fixing it if it breaks. This works because individual changes are small, anything can be rolled back, code with a larger blast radius gets more eyes on it, and regressions get caught quickly through heavy internal dogfooding and telemetry.

Thorsten Ball's bar is more personal: he only reviews fewer than 5 core files plus database migrations. He no longer cares whether the code uses closures or classes, how pretty the types look, how variables are named, or even whether there's duplicated code. He just asks a few questions: Does it work? Can I understand it at a "boxes and arrows" architectural level? Did the agent run an end-to-end test? Will I still be able to change it a month from now?

Understanding the Codebase Without Reading Diffs

Code review used to serve a secondary purpose: reading teammates' diffs was the main way engineers stayed familiar with the codebase. Once machines read most of the code on our behalf, how are humans supposed to understand the system?

The Claude Code team's answer is on-demand explainers. Nobody holds a complete mental model of the Claude Code codebase in their head, and they don't think anyone needs to. Before touching an unfamiliar area, they ask Claude to generate a custom explainer for that part of the code—it takes a couple of minutes and is always up to date. You don't actually learn much from reading a one-line copy tweak line by line, whereas reading a structural overview of a module right before you modify it teaches you a lot.

How I Do It

I maintain perryOS along the same lines. I have a lead agent break down tasks, dispatch work, and verify results, while delegating actual execution to worker agents. During verification, the lead agent reads the diff; during integration, it runs the full test suite, linter, and typechecker, and spot-checks one or two key screenshots. Mechanical verification—running builds, taking screenshots, measuring layout dimensions—is handed off entirely to a testing agent, which is only required to return the final conclusion, key numbers, and screenshot paths.

In this setup, the single highest-leverage practice is requiring the worker agent to hand back concrete proof rather than a bare "fixed." The most common failure mode is writing task descriptions that are too vague: the worker ends up wandering blindly across the repo in serial. I once had a single-file change run for 23 minutes and fire off 94 model requests. So now, whenever I dispatch a task, I explicitly specify the target files, the exact outcome expected for each change, the scope boundaries, and the acceptance criteria. That is really the same idea as "specify up front which layer of checks catches which failure"—just pushed earlier into the task-dispatch step.

What I'm Still Unsure About

"Most PRs have at most one human in the loop" works for the Claude Code team inside their own repositories, but it rests on a heavy foundation: mature merge queues, feature flags, staged rollouts, one-click rollbacks, and a large base of internal employees dogfooding the product all day. For a small team or a solo project without that infrastructure, the time saved by cutting back on review can easily come back with interest as production incidents.

As for Thorsten Ball's advice to "stop worrying about slop," I only partly agree. I agree with not reviewing code style line by line. However, duplicated code and casually introduced abstraction layers can mislead the next agent that picks up the context. Part of the cost of maintainability doesn't vanish—it just shifts onto the agent's context window and token consumption.

The boundary for auto-merging low-risk changes is still moving forward. As for which exact changes can safely merge to main with zero human eyes on them, I haven't run enough of my own data to know yet.

Questions I Want to Figure Out Next

• For a solo-maintained project, what is the minimum set of check layers needed to safely read less code?
• Which specific rules should go under "what does not count as proof" in AGENTS.md to genuinely stop agents from phoning it in with a simple "tests pass"?
• How can I run controlled experiments to verify whether agents are actually following the rules written in AGENTS.md and skills?

References

• *How the Claude Code team does code review*: Published by Addy Osmani on X (October 26, 2026), compiling and verifying the Claude Code team's code review workflow. The framing of human time as a two-sided cost, the six layers of checks plus fast rollback, risk-tiered approvals, convergence rules for agent-on-agent review, the responsibilities retained by humans, using explainers to understand the codebase, and all metrics cited in this post come from this piece.
• *How I work with Agents*: Published by Thorsten Ball (Amp) on X (October 25, 2026). Demanding irrefutable proof, having agents build their own experimental tools instead of iterating turn by turn in chat, making the entire development workflow visible to agents, and only reviewing high-blast-radius code come from this post.