OpenComputer's 3.3x Devin Claim Rests on One Task in Four Repos
Software / news
OpenComputer's 3.3x Devin Claim Rests on One Task in Four Repos
The open-source Slack coding agent's own benchmark lists completed runs of 5 of 6 for it and 4 of 9 for Replicas, against five rounds each.

OpenComputer, an Apache-2.0 coding agent that takes tasks from Slack, says in its README that it "opens PRs 3.3x faster than Devin." The figure comes from a benchmark its own team published on Oct. 30: one task, four repositories, five rounds per agent.
The README says the project is "open source from the compute layer up." The repository sits under the diggerhq GitHub organisation, was created Dec. 9, 2025, and had 761 stars and 82 forks on Nov. 21, according to GitHub's API. Stars measure interest, not adoption.
What the 3.3x compares
The benchmark post, by Igor Zalutski, Mohamed Habib and Utpal Nadiger, asked each agent over Slack to bump the zod validation library to its latest version in four repositories (sb-config-loader, sb-api-schema, sb-form-validator and sb-event-parser), run the checks and open a pull request in each. The repositories are public under diggerhq/speedbench.
Timing ran from the Slack message to the first push, or to all four pull requests open with CI green. The figures are medians of five rounds, and everything below is vendor-supplied.
- OpenComputer56 s
- Capy85 s
- Replicas87 s
- Hoplite95 s
- Devin185 s
Source: OpenComputer benchmark post, Oct. 30, 2026 (vendor-supplied), accessed 2026-11-21
The 3.3x is 185 seconds for Devin over 56 for OpenComputer. The README's second claim, 2.6x, compares OpenComputer's 24-second first push with a 62-second average for the other four agents. On first push alone, Devin's median is 72 seconds, so that gap is 3.0x.
Completion rates the headline skips
| Agent | First push (s) | All 4 PRs, CI green (s) | Runs completed |
|---|---|---|---|
| OpenComputer | 24 | 56 | 5 of 6 |
| Capy | 55 | 85 | 5 of 5 |
| Replicas | 53 | 87 | 4 of 9 |
| Hoplite | 69 | 95 | 4 of 6 |
| Devin | 72 | 185 | 5 of 5 |
The post says each provider ran five rounds, yet the completed-run denominators are 6, 5, 9, 6 and 5, and the post does not explain the mismatch. It also says some providers needed a nudge because they refused to rerun the task when a closed pull request already existed. Agent configurations and models are not described beyond the product names, and the post never defines "CI green".
The authors are the vendor and designed the test around their own product, which the post's own framing makes plain: "The bottleneck is speed, we realized." None of the pages read reports an independent run of the benchmark.
What runs where
According to the README, one VM runs a gateway called Calvin, which listens for Slack events and starts a session with Claude Code, Codex or OpenCode. "Each session is isolated with rootless Podman", a container tool that runs without root privileges. State goes to the VM's local disk first, then to a git-backed store.
The README recommends locking the VM down to outbound-only network access. The VM dials out to Cloudflare Workers and Durable Objects over a websocket, so nothing connects in.

"Support for Fly and other platforms is coming soon," the README says, so self-hosting today means a cloud VM, with AWS, GCP and Azure guides linked.
Agents that run commands need walls around them, and how thick those walls are varies by tool. Codex's sandbox lets commands read broadly by default, while OpenComputer puts each Slack thread's session in its own container. Another open-source tool aimed at developers is covered in the Onlook launch.
The hosted version carries a 3-day free trial with 600 work minutes. GitHub's API shows 94 open issues and pull requests combined, and the latest release tag is v0.6.0.50. The last push landed at 2:38 a.m. UTC on Nov. 21.
Sources
More in Software
- 01Brazil's Receita Federal Posts Tax Calculator Under MIT With 16 Open IssuesThe agency published three repositories on Nov. 16 for the new consumption taxes, and outside developers had filed bug reports against the first within days.
- 02OpenSRE Hits 13,884 Stars After Repositioning Toward CI/CD FailuresThe Apache-2.0 agent is labelled public alpha, ships a dated release almost every day, and sends analytics and crash reports unless users set an environment variable.
- 03Microsoft 365's 1,430-Minute Outage Traces to a July Windows ChangeMicrosoft's preliminary report blames a latent lock-contention defect, a Teams failover and 14 percent of gateway capacity already offline, and customers have until the end of December to claim service credits.
- 04Firecrawl's Alexandria Data Library Is Missing From Its 205,955-Star RepoThe AGPL-3.0 repository covers scraping and a self-hosted server, while the 550-provider data library and search run on Firecrawl Cloud.