Momentic's Mo Matches 269 of 1,396 Labelled Bugs on a Benchmark Momentic Built, at 3.3 Times Codex's Cost Per Bug

Momentic, a Y Combinator W24 company, says its new QA agent Mo matched 269 of 1,396 human-labelled bugs in a 60-minute run, against 86 for Codex driving Playwright MCP. The benchmark, QA Bench, was built and labelled by Momentic.

The figures come from Momentic's Launch HN post, which stood at 167 points on November 8, and from the QA Bench page it links. All of them are vendor-supplied. The announcement does not say any outside party has re-run them.

What Momentic says Mo found

Mo tests web, iOS and Android apps from plain-English instructions, and the launch post describes a graph of parallel sub-agents, each with its own browser or device. The headline comparison is against three other agents on the same time limit.

Agent	Bugs found	Recall	Precision	Cost per bug
Mo	269	19.3%	84%	$2.67
Codex (Playwright MCP)	86	6.2%	86%	$0.80
Claude Code (Playwright MCP)	59	4.2%	80%	$4.09
Devin	27	1.9%	84%	$3.89

Mo finds 3.1 times as many bugs as Codex, while Codex is two points more precise. Mo's 19.3% recall still means it missed four of every five labelled bugs. The post does not break the 269 down by severity.

1. Mo269 bugs
2. Codex (Playwright MCP)86 bugs
3. Claude Code (Playwright MCP)59 bugs
4. Devin27 bugs

Source: Momentic launch post on Hacker News and momentic.ai/qa-bench, accessed 2026-11-08 (vendor-supplied)

Cost cuts the other way. Multiplying the published cost per bug by the bug count implies roughly $718 of spend for Mo and $69 for Codex over the same hour. That is our arithmetic, not a figure either company published, and the page does not say what the cost includes.

What QA Bench does and does not disclose

QA Bench covers four open-source apps: Gitea with 476 labelled bugs, Metabase with 464, Twenty with 374 and Saleor with 82. Those sum to the 1,396 total. Each agent tests one product area of a pinned upstream commit at a time, in its own sandbox, with no access to the source code or the labels.

The page gives no per-app results for any agent, so a reader cannot tell whether Mo's lead holds on Saleor's 82 bugs as well as on Gitea's 476. It does not say how many runs each configuration received, who did the labelling, or what judge model scored the reports; the launch post says an LLM judge with human review did the scoring. It lists no limitations. The dataset is on Hugging Face and the code is on GitHub, which lets others re-run it.

Mo went from 177 bugs at 30 minutes to 269 at 60, a 52% gain, while Codex gained 16% over the same stretch, according to the post. A time-boxed benchmark rewards whichever agent keeps exploring longest.

Where the Hacker News thread pushed back

A commenter writing as akeel_mohamed argued that a benchmark built and labelled by the vendor will probably favour the vendor. Another, haukebri, said trust matters more than recall and asked for false alarms per run, a number the post does not give. Commenter Sky_Joy3 asked how a "bug" is defined and de-duplicated.

Replying in the thread, Henry, an engineer on Mo, said QA Bench has no planted bugs and that suspected bugs Mo cannot reproduce are discarded internally. Jeff, a co-founder, said bring-your-own-model support may come later after a commenter objected to token-based pricing. The post lists no prices beyond $250 of free Mo spend for each new signup.

The pattern is familiar: a compression proxy called its results third-party while hosting them in the author's own repo.

What Momentic had raised before the launch

Momentic announced a $15 million Series A on November 24, 2025, led by Standard Capital with Dropbox Ventures and existing investors including Y Combinator, according to TechCrunch. It followed a $3.7 million seed announced in March 2025. TechCrunch put the user base at about 2,600 and named Notion, Xero, Bilt, Webflow and Retool as customers. Valuation was not disclosed.

Momentic's own site counts 117,010 bugs caught but gives no funding or batch. Another YC company, Discovered Materials, also sells on a result it measured itself.

Henry said real-device support is coming "in the next few weeks". An independent QA Bench run with per-app numbers would confirm or cut the 19.3% recall figure.