Alibaba's OpenCodeReview Claims Claude Code Precision at One-Ninth the Tokens

Alibaba's OpenCodeReview, an Apache-2.0 command-line reviewer for Git diffs, says it beats Claude Code on precision and F1 while spending about one-ninth of the tokens. Those figures are the vendor's own, and the repository showed 51.2k stars on Nov. 14.

The claim comes from the project README, which describes a tool that "reads Git diffs, sends changed files to a configurable LLM via an agent with tool-use capabilities." It installs with npm install -g @alibaba-group/open-code-review, needs Git 2.41 or later, and exposes an ocr command. ocr scan reviews whole files.

What Alibaba measured

The README says OpenCodeReview reaches "significantly higher Precision and F1 with the same underlying model" than Claude Code. It publishes no precision or F1 values in the text fetched, and InfoQ's Renato Losio, writing on Sept. 20, wrote that the article gives no such values. The README also concedes that "its Recall is lower than general-purpose agents."

The benchmark is AACR-Bench, described in an arXiv paper: 200 pull requests and 1,505 review comments from 50 repositories in 10 languages. Five of its authors list Alibaba's TRE group as their affiliation, alongside researchers at Nanjing University and Southern Cross University. So the test is not independent of the company that scored best on it.

1. Generated by LLMs and experts1114 comments
2. Augmented from original PR reviews391 comments

Source: AACR-Bench paper, arXiv 2601.19494, accessed 2026-11-14

What the critics found

InfoQ quoted two practitioners. Tom Rochette, a senior developer at Shopify, said that "recall is deliberately lower than a general agent." Daniel Vaughan, head of forward deployed engineering at HCLTech, said the "deterministic dispatch that drives precision also limits discovery of cross-file and architectural issues," and concluded that the project's contribution "is not a better model but a better harness."

Vaughan reported that the best configuration reaches 20% recall on AACR-Bench. An independent run on 10 Martian-benchmark pull requests produced about 12% precision, according to InfoQ. The maintainer disputed that result as a tool-call anomaly and fixed it, and InfoQ said no independent check after the fix has been published.

Claim	Who made it	Status
Higher precision and F1 than Claude Code	Alibaba README	Vendor-supplied
About 1/9 of the tokens	Alibaba README	Vendor-supplied
Best recall about 20%	Vaughan, via InfoQ	Reported, not reproduced here
About 12% precision on 10 PRs	Independent run, via InfoQ	Disputed by maintainer

What sits underneath

The design splits the work: file selection, bundling, rule matching and checking comments against the diff are plain code, and the model handles analysis. The README says it "has served tens of thousands of developers and identified millions of code defects" inside Alibaba, a figure with no outside check. For another vendor-run comparison the site has covered, see Lightpanda's 9x benchmark, and for tooling that builds agent workflows in an IDE, JetBrains' ThinkRail.

The number to watch is recall. Until someone outside Alibaba publishes it on a benchmark Alibaba did not help write, the 1/9 token figure says how cheap the reviews are, and not how many bugs they miss.