Cua Ships CUA-S1-Forms With a Checkpoint It Can't Load

Cua, a San Francisco startup that builds infrastructure for AI agents that operate a mouse and keyboard, published a 706,048-parameter model called CUA-S1-Forms on Hugging Face on Sept. 18. The model scores which button or checkbox to click on a form rather than generating text, and Cua's founder said in a Hacker News post the same day that it beat a rival's hosted model on the company's own test.

Cua open-sourced the model, its training code and its 234,067-row synthetic dataset under the MIT license, according to the model card on Hugging Face. Founder and chief executive Francesco Bonacci, a former Microsoft engineer who wrote the pull request introducing CUA-S1 to the project's README on Sept. 19, described it on Hacker News as the first release in a family of small "System One" models built for narrow, repeatable interface tasks rather than general computer use.

What broke on the first try

A user filed issue #3977 against the trycua/cua repository on Sept. 19, a day after the model went up, reporting that the only checkpoint file distributed with the model was a legacy PyTorch pickle file, and that the library's own loader rejected it outright. The error message quoted in the issue reads: "legacy pickle-based checkpoints are not supported; convert the model to a safetensors file plus JSON configuration." The issue also flagged a dead link to a results document the model card pointed to.

According to Hugging Face's own commit log for the model repository, a Cua contributor pushed a safetensors checkpoint and matching JSON configuration file on Sept. 19 at 3:30 p.m. UTC, roughly 23 hours after the initial upload, with the commit message "so cua_s1's own loader can load it." The pickle file is still in the repository, now described in the documentation as a legacy format kept for compatibility rather than the default.

The benchmark, and who it's measured against

Cua's own comparison, posted to Hacker News and repeated on the model card, pits CUA-S1-Forms against a hosted model called Jev, made by a company called TypeSafe. On Cua's task, the smaller model scored 99.7 percent accuracy against 83.6 percent for the hosted alternative, while running the decision locally in 7 to 9 milliseconds against 260 to 280 milliseconds for a remote call to Jev. That comparison is Cua's own and was not run by an independent party; The Terminal could not verify it.

Metric	CUA-S1-Forms (local)	Jev (hosted)
Accuracy on Cua's task	99.7%	83.6%
Decision latency	7-9 ms	260-280 ms
Deployment	Runs on-device	Remote API call

The model card also reports 99.95 percent accuracy on a synthetic test set of about 15,000 decisions and a perfect score across 196 decisions drawn from three real forms, which is a small enough sample that Cua's own documentation warns the model is "not validated on arbitrary real-world forms beyond the demo set."

What the license actually covers

The repository's documentation draws a line Cua's own MIT badge does not make obvious: the MIT license applies to the CUA-S1 source code, but according to the project's own README, that license "does not apply to future official model weights, datasets, hosted services, or Cua trademarks." This first checkpoint is released under MIT regardless, but the carve-out means a later, better-trained version of the same model line is not guaranteed the same terms.

Cua joined Y Combinator's spring 2025 batch after Bonacci left Microsoft, where he had co-authored Windows Agent Arena, and the trycua/cua repository has drawn 24,399 GitHub stars as of Sept. 19, including 859 gained in the 24 hours the leads scrape covered. That growth follows a broader pattern since early 2026 of AI-agent tooling shipping fast and hitting reliability problems in the open, the same failure mode AWS ran into when its Lambda-hosted Pydantic AI agents kept re-billing customers for runs that had already failed. CUA-S1-Forms arrives days after Coder made its own self-hosted AI agents generally available, part of the same shift toward running agents on infrastructure a team controls instead of a vendor's cloud.

Cua has not published the results document the model card originally linked to, and has not said when the next model in the CUA-S1 family will ship or what task it will target.