Momentic's Mo Finds 19.3 Percent of Labelled Bugs in Its Own Benchmark

Momentic, a Y Combinator Winter 2024 company, posted a Launch HN on Nov. 2 for Mo, an agent that explores web, iOS and Android apps looking for bugs, and published a benchmark in which Mo found 269 of 1,396 labelled bugs.

The benchmark is Momentic's own. The company wrote it, ran it and says an LLM judged the results with a human reviewer, so every score below is vendor-supplied. The thread had 52 points and 34 comments when The Terminal read it.

What Mo does and what it costs

Two co-founders, Wei-Wei and Jeff, who posts as jeffan, wrote that hand-written Playwright and Selenium scripts break because they depend on selectors and implementation details. Mo instead builds a graph of agents as it explores, forks large product areas into parallel sub-agents, and gives each its own browser or device. A tunnel lets hosted agents reach private or local apps, and one run can use hundreds of concurrent browsers.

New signups get $250 of Mo spend, the post says. It gives no subscription price. Momentic's site lists a free tier and an enterprise tier marked "Contact sales", and says nothing about funding or founders.

The qa-bench numbers

The benchmark, called qa-bench, uses four open-source apps (Gitea, Metabase, Twenty and Saleor), 1,396 human-labelled bugs and 60-minute runs. Momentic says the harness is MIT-licensed.

Agent	Bugs found	Recall	Cost per bug
Mo	269	19.3%	$2.67
Codex (GPT-6.1 Sol with Playwright MCP)	86	6.2%	$0.80
Claude Code (Opus 5.5 with Playwright MCP)	59	4.2%	$4.09
Devin	27	1.9%	$3.89

Precision, the share of reported bugs that were real, was 84 percent for Mo, 86 percent for Codex, 80 percent for Claude Code and 84 percent for Devin.

1. Mo19.3 %
2. Codex6.2 %
3. Claude Code4.2 %
4. Devin1.9 %

Source: Launch HN: Momentic (YC W24), news.ycombinator.com, accessed 2026-11-02

Read the other way, the best score still leaves 80.7 percent of the labelled bugs unfound. Mo is also the second most expensive per bug. Multiplying the table, and assuming cost per bug is total spend divided by bugs found, Mo's run cost roughly $718 against roughly $69 for Codex. That is The Terminal's arithmetic, not a figure Momentic published.

The authors say Mo's count grows with run time, from 177 bugs at 30 minutes to 269 at 60, a rise of 52 percent, while Codex rose 16 percent. A time-boxed test favours the tool that keeps finding things.

What commenters pushed on

One commenter, aliasxneo, asked whether this was "yet another SaaS product that's going to attempt to charge me for token usage?" Another, adwn, asked "what will you sell when Opus 6.0 swallows Mo's edge in 5 months from now?"

Jeff replied that plugging vanilla GPT-6.1 Sol into Mo's harness "just doesn't yield a pareto optimal result", and that bring-your-own-key support may come as open-source models improve. A third commenter, kart23, wrote that "the harness is the lock-in with these subscriptions." Jeff said the regression product uses caching and a natural-language format that only Momentic's CLI runs.

What Momentic has raised

Momentic announced a $15 million Series A on Nov. 25, 2025, led by Standard Capital with Dropbox Ventures and existing backers including Y Combinator, according to Pulse 2. The company reported more than 390,000 bugs identified at that point. Its site now says 117,010 bugs caught, which does not match, and neither number is audited. The site does say its SOC 2 Type 2 audit was done independently.

Another Launch HN with disclosed funding is Simantic's firmware simulator, which raised $600,000. For a separate vendor-run score with caveats, see Lightpanda's web-test claim.

The next test is whether anyone outside Momentic runs qa-bench. Jeff invited contributions to add other agents to it.