Octomind Solves 24 of 25 Tasks in Its Own Benchmark, and the Cheapest Run in the Table Is Codex at $14.86

Octomind, an Apache-2.0 coding-agent harness written in Rust, reports that it solved 24 of 25 real-pull-request tasks for $63.43 using the GLM-5.2 model, ahead of Claude Code at 23 and OpenCode at 19. The results sit in the project's README, and they were published by Octomind's own maker, Muvon, on its own benchmark.

The project reached Hacker News on Nov. 3 under the title "Octomind – open-source AI harness that saves your tokens, static rust." The README, at 368 stars and 55 forks when fetched, gives no token-savings percentage anywhere.

The table Octomind published

The benchmark, octobench, uses 25 tasks harvested from merged pull requests across Python, PHP, Rust, C++ and JavaScript. Each agent works in the pre-fix repository, and held-out tests from the real fix decide pass or fail. The README calls these the "published July 30, 2026 results."

Agent and model	Solved of 25	Cost	Wall time
Octomind, glm-5.2	24	$63.43	3.6 hours
Claude Code, claude-opus-5	23	$81.79	6.7 hours
Codex, gpt-5.6-sol	21	$14.86	1.0 hour
OpenCode, glm-5.2	19	$129.54	3.3 hours

1. Codex + gpt-5.6-sol14.86 USD
2. Octomind + glm-5.263.43 USD
3. Claude Code + claude-opus-581.79 USD
4. OpenCode + glm-5.2129.54 USD

Source: Octomind README and octobench BENCHMARK.md, accessed 2026-11-03

Octomind and OpenCode ran the same model, so the project argues the gap is the harness: 24 solved against 19 "at roughly half the cost." That comparison is the cleanest in the table. The headline comparison with Claude Code is not, because the models differ.

What the table leaves out

Codex's $14.86 is the cheapest result in the table and the page does not highlight it. By simple division, which is this report's arithmetic and not Octomind's, Codex cost about $0.71 per solved task against $2.64 for Octomind. It also finished in an hour. Octomind's page tells readers to "read the cost column, not the clock." Read that way, the column ranks Codex first.

The cost basis differs by row. The benchmark page says the GLM runs had no prompt caching, so input was billed at the $1.40 per million tokens list price, while Opus's context traffic was about 97 percent cache reads at $0.50 per million. Claude runs use the agent's own reported cost. Others are priced from token counts times rates in a config file. Muvon argues its GLM figure is therefore a worst case, though it does not quantify a cached figure.

Then there is the grader. The benchmark page says the judge is Octomind's own judge role, with its model set by an environment variable. The reproduction example names ollama:minimax-m3, and the page does not confirm that was the judge for the published numbers. A failed hidden test costs 25 points, and the judge score is out of 2,500. Some cases pass hidden tests with low judge scores: a spdlog case scored 55 for Opus and 60 for Octomind.

The page lists no versions for Claude Code, OpenCode or Octomind. It names only Codex 0.145.0 or later for gpt-5.6-sol.

How the token saving works, and the thread's verdict

The project's site says tools stay dormant until needed, so about six sit in context instead of roughly 80, and that compression is cache-aware. It offers spend caps per request and per session. Cloud use starts at $50 a month. Telemetry is on by default and reports commands, tools, models, timings, token counts and error kinds, but not code, prompts or file paths; DO_NOT_TRACK=1 turns it off.

The Hacker News thread had 32 comments. A commenter named donk8r defended the benchmark and said GLM-5.3 and GLM-5.2 performed "almost the same"; the README names 5.2. No commenter reproduced the results. The user unsloppable argued "Tokens are cheap. Quality is what matters," and the_duke called the README "AI slop."

The install route is a curl script among several options, and tamimio objected that such scripts "aren't secure." Homebrew, Scoop, cargo install octomind and Docker are also listed.

For other agent cost questions, see what AI agents cost across JavaScript frameworks; for agent tooling built on Git, see Worktrunk. Octobench's reproduction steps are public, so the first independent rerun of all 25 cases is the result to wait for.