Cognition's SWE-2 Undercuts Fable 5.1 on Price, Not Every Benchmark

Cognition released SWE-2 on Thursday, saying its newest coding model scores 50.0 percent on the FrontierCode 1.1 Main benchmark, within a point of Fable 5.1, at what the company's launch post says is 64 percent less cost to run.

SWE-2 is the fourth model in Cognition's SWE line and the first built on Moonshot AI's Kimi K3, a base model with 2.8 trillion parameters. Cognition said it post-trained Kimi K3 with reinforcement learning that produces three effort levels, medium, high and max, from a single training run, using a reward function the company's post describes as cost-penalized: it subtracts a multiple of inference cost from the score a fix earns.

The model ships today inside Devin Desktop and Devin CLI, Cognition's agentic coding products, and is rolling out to Devin Web and Fusion. The company has not published a standalone API, pricing for API access, or the model's weights, and did not say when, or whether, it will.

What the FrontierCode numbers show

On FrontierCode 1.1 Main, Cognition's own benchmark for whether an AI-written pull request would actually get merged, SWE-2 scored 50.0 percent against 50.9 percent for Fable 5.1 and 53.3 percent for GPT-6 Astra, according to Cognition. On a second benchmark, DeepSWE 1.1, SWE-2 reached 73.0 percent, ahead of Fable 5.1's 67.4 percent and just short of GPT-6 Astra's 74.1 percent.

Cognition also said SWE-2's medium effort setting needs a median of 18 steps to reach its first real edit to a file, down from 48 steps for SWE-1.7, the model SWE-2 replaces, and that it costs 81 percent less than SWE-1.7 to reach a fix on FrontierCode 1.1 Main.

These are vendor-supplied figures. Cognition ran the benchmarks and published the results itself; OfficeChai, which covered the release the same day, repeated the same numbers without independent testing of its own.

Where SWE-2 falls behind

Cognition's post includes a fourth benchmark, Terminal-Bench 4, that it does not lead with. There, SWE-2 scored 27.3 percent, against 55.8 percent for Fable 5.1 and 57.9 percent for GPT-6 Astra, a gap of roughly 30 points. The post does not explain what Terminal-Bench 4 tests differently from FrontierCode 1.1 Main, or why a model that nearly matches the frontier on one benchmark trails it by half on another.

The trust numbers Cognition included

The company also published what it called a trustworthiness evaluation: a 98.0 percent pass rate on a test for propaganda and censorship, and no measurable change in SWE-2's answers when the same question was framed five different ways across demographic groups. Cognition did not say who designed that test, whether anyone outside the company ran it, or how many questions the five-framing comparison covered.

The closest independent check available is OfficeChai's write-up, which restates Cognition's numbers rather than rerunning them; neither outlet's coverage includes a benchmark Cognition did not choose to publish. Cognition has not said when it will price API access to SWE-2, the step that would let developers outside Devin weigh its cost claim against Fable 5.1 and GPT-6 Astra on their own code rather than on Cognition's chosen benchmarks, or whether the 64 percent cost figure holds once real usage, rather than benchmark runs, sets the token counts.