Claude Code Team Publishes Its Verification Rules: 39 Checks, a Three-Round Limit and Observe Mode
A.I. / news
Claude Code Team Publishes Its Verification Rules: 39 Checks, a Three-Round Limit and Observe Mode
A Nov. 16 post says about 90% of the team's code is written and reviewed by Claude, and lays out the gates that keep it shippable.
The team behind Claude Code published its verification rules on Nov. 16, and the headline figure is its own: about 90% of the team's code is written and reviewed by Claude. The post, How the Claude Code Team Thinks About Verification, is bylined Addy Osmani and filed under Engineering.
Its argument is that verification loops decide how much work a person can hand to agents. Without them, the post says, a person can supervise one or two Claudes at a time. With them, dozens or hundreds in parallel.
The numbers in the post
| Rule | Figure |
|---|---|
| Items on the reliability checklist | 39 |
| Items Claude usually checks per diff | 3 to 10 |
| Failed rounds before a human is called | 3 |
| PRs a mined rule must appear in | 2 or more |
| Agents that try to disprove a rule | 3 |
| Commit gate modes | off, observe, enforce |
| Blocks before the next commit passes | 2 |
Each checklist item has a trigger that can be read from the diff. One example the post gives is "--resume loads old transcripts." If a check still fails after three rounds, Claude stops and brings in a human.
How rules are vetted and tested
A candidate rule mined from review comments or reverts must appear in at least two PRs. Three agents then try to disprove it before a human signs it off.
To validate a bug finder, the team replays three finders against a PR linked to a known incident. If at least two find the bug, it counts as detectable.
The commit gate and the self-edit rule
Commits are stamped with how they were verified, including "none." A gate that requires a code review or a verify step starts in observe mode, so false positives show up before anything is blocked. If a session has been blocked twice, the next commit goes through.
One rule is put bluntly: "If a change can modify the check that judges it, the check is broken." The post says Code Review reads REVIEW.md from the base branch, not from the PR, for that reason. It also says "A passing review on an out-of-date commit means nothing," so only current commits are reviewed.
Three kinds of test, and one stance on flaky ones
The post ranks end-to-end tests first because they describe real user flows and are hard to fool. It pairs them with property tests, which check invariants such as an account balance never going negative, using libraries such as fast-check for JavaScript and TypeScript, Hypothesis for Python and proptest for Rust. The third kind is differential testing, which runs old and new systems on the same inputs and treats the old one as the oracle.
On flaky tests the post is blunt. Claude may not skip them, add sleeps or retry until they pass. If the root cause cannot be found, the test is deleted and the reason documented.
It also says that "Adding a second, more expensive, reviewer on top of the primary one is often a worse approach than tuning the primary."
What the post does not show
The post says a separate grader checking work against a rubric improved task success by up to 10 points over a standard prompting loop in internal testing on structured file generation. It is the only measured result in the summary of the post that this report could extract, and it is the team's own. The post says "Claude Code isn't perfect," and gives no defect rate for the 90% of code Claude writes.
The same tooling questions show up elsewhere, from gating remote coding agents behind local approval to what Windows 11 reports about which agents you run.
The post's closing line sets the expectation for the next stage: "Your job shifts from reading every diff to defining what 'done' looks like and reviewing the evidence."
Sources
More in A.I.
- 01Runta's 27,280-Run Harness Eval Finds the Model Explains 92.8% of Agent CostFrontierHarness v1.1 ran 22 coding-agent harnesses against five open models, and Codex CLI went from first place to sixteenth when the model changed.
- 02Qwen3.8-27B Has 6.9 Million Downloads on Hugging Face. Artificial Analysis Rates It Slow and VerboseAlibaba's Apache 2.0 dense model beats its size class on an independent index of 20, and takes 38.8 minutes a task against a 12.6-minute median.
- 03Inkling 2 Small Weights Land on Hugging Face Under Apache 2.0, and the Card Asks for 500GB of VRAMThinking Machines opened the weights a week after launching on Tinker. The safety section still says release is pending.
- 04Fed Governor Barr Says AI Agents Could Accelerate Bank Runs That Unfold in MinutesIn a Nov. 16 speech in Milan, Barr also put AI firms' off-balance-sheet liabilities above $3 trillion and proposed no new rules.