Momentic's Launch HN Benchmark Has Its Agent Finding 269 Bugs at $2.67 Each, Against Codex's 86 at $0.80
Startups / news
Momentic's Launch HN Benchmark Has Its Agent Finding 269 Bugs at $2.67 Each, Against Codex's 86 at $0.80
The Y Combinator W24 company's own QA Bench, labelled by the company, shows more bugs found and a higher cost per bug. Commenters spent the thread on exactly that trade.
Momentic, a Y Combinator W24 company that sells AI agents for testing web and mobile apps, posted a Launch HN on November 2, 2026 reporting that its agent found 269 of 1,396 labelled bugs in a 60-minute run, at $2.67 a bug. OpenAI's Codex found 86 at $0.80 in the same test. The post reached 163 points and 127 comments, per the Hacker News thread.
The benchmark is vendor-supplied: Momentic built it, labelled the bugs and ran the comparison across four open-source apps. The Terminal has not reproduced it, and the thread's top objection is that a benchmark the vendor labels favours the vendor.
The numbers as posted
| Agent | Bugs found | Precision | Cost per bug |
|---|---|---|---|
| Mo (Momentic) | 269 | 84% | $2.67 |
| Codex, GPT-6.1 Sol with Playwright MCP | 86 | 86% | $0.80 |
| Claude Code, Opus 5.5 with Playwright MCP | 59 | 80% | $4.09 |
| Devin | 27 | 84% | $3.89 |
Recall, the share of the 1,396 labelled bugs found, works out to 19.3% for Mo and 6.2% for Codex. Multiplying bugs found by cost per bug gives roughly $718 for a Mo run and $69 for a Codex run, a calculation that assumes the cost is total run spend divided by bugs found, which the post does not spell out.
- Mo269 bugs
- Codex86 bugs
- Claude Code59 bugs
- Devin27 bugs
Source: Launch HN: Momentic (YC W24), Hacker News, accessed 2026-11-04. Vendor-supplied.
A commenter under the handle MGabrielDenton, replying in the thread, said Mo finds roughly five times as many bugs as Claude Code at about 35% less per bug. The table supports the second half exactly, $2.67 against $4.09, and the first half approximately, 269 against 59 is 4.6 times. Another account, momentichenry, said Mo is tuned for recall and that its higher cost against Codex is the price of that.
What the commenters pushed on
The objections cluster in three places. On method, akeel_mohamed argued that cost comparisons only hold under identical rules, and StilesCrisis said a large count of minor bugs is not useful when serious ones are what matter. On price, JulianPaul asked why anyone should not run Opus 5.5 with their own Playwright harness on an existing subscription, and aliasxneo objected to paying for proprietary tokens instead of bringing a key.
The founder account jeffan answered that the $2.67 is all-in, covering infrastructure and parallel agents, and that off-the-shelf models did not perform well enough inside Momentic's harness. He said bring-your-own-key support may come later.
On product, vfosnar reported that the product posted their email to Slack on login without clear notice, and jeffan apologised and said it should only affect opted-in corporate accounts. ice_cream_man_9 reported bug counts that confused them, every bug tagged medium and no export.
Who is behind it
Momentic's cofounders are Wei-Wei Wu and Jeff An, former developer-tooling engineers whose earlier employers include Qualtrics and WeWork, TechCrunch reported. The company raised a $15 million Series A on November 24, 2025, led by Standard Capital with Dropbox Ventures participating, after a $3.7 million seed announced in March 2025. Wu declined to share revenue then, and the figure still does not appear on the company's pages.
Momentic's site lists Notion, Webflow, Retool, Runway and Quora among named customers, offers a free start with no credit card, and publishes no prices. The Launch HN offers $250 of spend credit to new signups. The product runs tests written in plain-English YAML from the command line or CI.
The competing claim
The comparison most readers will make is against the cheapest option. The site's earlier coverage of Octomind's benchmark, where Codex came out cheapest, found the same agent winning on cost, and the pattern of a vendor benchmark with no published method also appears in Nvidia's auto-ontology claim.
Momentic has published the dataset and harness behind QA Bench, according to the post, which is what makes an independent rerun possible. That rerun, by anyone other than Momentic, is the number that would settle the dispute.
Sources
- 03Momentic
More in Startups
- 01PhysicsWallah Swings to ₹143.6 Crore Profit as Costs Fall 11%Other income supplies 40 percent of PhysicsWallah's pre-tax cushion against 18 percent at Lenskart, whose September-quarter results Inc42 dated Nov. 4.
- 02GIVA Raises ₹975 Crore From Multiples and Will Not Say What It Is Worth, Only That It Is Above ₹4,500 CroreThe jewellery brand's round mixes new money, secondary sales and an ESOP buyback. Revenue reached ₹865 crore in FY26 against ₹505 crore a year earlier, and profit after tax is promised for FY28.
- 03EloElo's $80 Million Series C Reportedly Prices It at $500 Million, About 68 Times Its Last Disclosed RevenueNBCUniversal and Hornbill Capital co-led the Bengaluru microdrama company's round on Nov. 4. The valuation is reported rather than announced, and the only revenue figure on file is for the year ended March 2025.
- 04Discovered Materials, a $9 Million-Seed YC Startup, Puts $1 Million on Five Problems Its Own AI Agents Could Not SolveThe San Francisco company says frontier models worked on each problem for days and made little progress. Investors in its August seed say the real bottleneck is making the materials, not finding them.