Most of the code my team ships is now written by AI agents. That is great for speed and bad for one old habit: a person reading every line before it merges.
Agents make big pull requests cheap. Past 1,000 lines a diff is hard to review well. Past 15,000, nobody can read it and say anything useful about it.
So a required approval isn't the answer. People would click approve, and the checkbox would record a review that never happened. We wanted a flow where a person still owns every change, but spends their attention where it matters.
Ask what could go wrong, not who read it
The idea came from Addy Osmani's write-up of how the Claude Code team reviews code. I found it through Matt Ström-Awn, whose writing on design and how teams build is worth following. Thanks to both of them, and to the Claude Code team for sharing their process so openly.
Two points from the article changed how I saw the problem.
"Did someone read the code?" is the wrong question. The useful one is what could go wrong with this change, and which check, automated or human, should catch it.
No single gate catches everything. You stack layers that each catch some bugs, and keep a fast way to roll back the few that slip past all of them.
What each layer is there to catch
- Intent
building the wrong thinga person, 2 min
- Size and risk
PRs too big to judgea script
- Claude review
bugs in the linesClaude, rolling out
- Approval
missing proofa person, by risk
- Release gate
a bad releasea person, release day
- Hotfix rule
the same miss twicea person, one line
1. Agree on the intent before any code
Before anyone writes code, the issue states its acceptance criteria and an owner says "build it, this way" or "don't". It takes about two minutes. It catches building the wrong thing, which no code reviewer can catch later.
2. Let the change set its size and risk
A check counts hand-written lines. Generated files, such as API clients, lockfiles and recorded test fixtures, don't count. Past 1,500 lines, the pull request has to be split into a chain: migration, then backend, then client, then UI.
The one exception is formatting, and it has to be proven. The check runs the formatter over the old version and confirms it reproduces the new one exactly. A reformat of tens of thousands of lines passes in seconds, with no person involved.
The same check sets a risk tier from the files touched, not from who wrote them. Migrations, auth, billing, permissions and CI are high. Docs, stories and tests are low. Everything else is medium.
3. Let Claude read the lines and write for a person
This layer is still rolling out, so I can't yet tell you how it holds up. Here is how it's set up.
Claude reviews every pull request marked ready against a REVIEW.md in the repo. That file lists the failure modes specific to our codebase. It says a finding only counts if it names the input that breaks the code, and it bans style nits. Claude posts its findings inline, plus one summary a person can decide from:
- what changed, in plain words
- the handful of risky lines, linked
- what it set aside, and why that's safe
- the proof: tests, screenshots, a preview link, or "missing"
- what the checks can't see
- how to revert
It stops after three rounds. A pull request that hasn't converged by then needs a person to rethink the approach, not another bot pass.
4. Scale human approval to the risk
Who signs off
- Low risk
the author and the checksdocs, stories, tests
- Medium risk
a second readereverything else
- High risk
a code owner, risky lines onlymigrations, auth, billing, CI
At medium risk, the second person reads Claude's summary and checks the proof. That takes about ten minutes, with no diff reading. At high risk, a code owner reads only the lines the summary points to.
An approval means "I checked the proof." If you can't say what the checks would miss, it isn't an approval.
5. Ship on release day, behind a gate
We ship on a set release day. Merges close the day before, end-to-end tests run that evening, and on release morning one person reads a single release summary and makes the go or no-go call. Anything unverified ships behind a feature flag, switched off.
6. Turn every hotfix into a rule
Each hotfix gets one line: which layer should have caught it? The answer becomes a new rule in REVIEW.md or a new test. This is the layer that makes the others better over time.
We swapped a testing tool for a Claude plan
We used to pay for an AI end-to-end testing tool. A $100 Claude plan costs less and now covers that work and more. It writes and maintains plain Playwright tests that live in our repo and run free in CI. It will write the review summary on every pull request. It runs the checks our agents do before a person ever looks.
An AI testing tool
- End-to-end tests
Claude plan, $100 a month
- Playwright tests in our repo
- A summary on every PR
- Checks before a person looks
We prefer plain tests we own and can read, next to a reviewer that reads every line. One caveat if you try this: check that a single plan's usage limits cover your volume of pull requests.
What people still own
Deciding whether to build something at all. Architecture. Anything touching auth, billing or releases. Checking the proof. And answering for every change that ships, whoever typed it.