PrismML's Ternary Bonsai 2 27B Hides a 75% Score

PrismML says its new Ternary Bonsai 2 27B model keeps 98.2 percent of full-precision performance while shrinking a 53.8-gigabyte model to 5.9 gigabytes, a compression ratio near 9-to-1. The number that matters is not that headline average; it is the 75 percent retention PrismML's own benchmark table shows on SWE-bench Verified and Terminal-Bench 2.1, the two evaluations that most resemble how an agent actually gets used.

PrismML released the model Sept. 17 under an Apache 2.0 license, built by compressing Alibaba's Qwen3.8-27B into ternary weights, each one set to −1, 0 or +1 with FP16 group-wise scaling, for an effective 1.76 bits per weight. The company's own writeup states plainly that "all results are PrismML's own and have not been independently reproduced," a caveat MarkTechPost repeated when it covered the release the next day.

What PrismML actually released Sept. 17

Ternary Bonsai 2 27B carries 27.36 billion total parameters, split into a 24.35-billion-parameter language backbone, a 2.54-billion-parameter embedding and output head, and a 0.46-billion-parameter vision tower, according to the model's Hugging Face card. It keeps Qwen3.8-27B's hybrid attention design, roughly 75 percent linear attention and 25 percent full attention, and the full 262,000-token context window. Running it requires PrismML's own fork of llama.cpp; the stock version of that software cannot load the ternary format.

The Hugging Face card lists two separate packings rather than one fixed file: PTQ1_0, at 1.75 bits per weight and 5.95 gigabytes, and PQ2_0, at 2.13 bits per weight and 7.21 gigabytes. PrismML's own headline figures, the 5.9-gigabyte size and the 1.76-bits-per-weight average, describe the smaller of the two, which means a developer who downloads the larger PQ2_0 packing for better accuracy is already working from a different, unpublicized set of retention numbers than the ones in PrismML's announcement.

The headline number, and the one that isn't

Across a 20-benchmark suite, PrismML reports an aggregate score of 83.9 against a full-precision baseline of 85.4, the 98.2 percent figure the company is promoting. Category by category, the compression is uneven: math retains 99.5 percent (96.57 versus 97.06) and coding retains 99.3 percent (81.58 versus 82.17), both close enough to the baseline to be within normal benchmark noise.

Benchmark	Qwen3.8-27B (FP16)	Ternary Bonsai 2 27B	Retention
Overall (20 benchmarks)	85.4	83.9	98.2%
Math	97.06	96.57	99.5%
Coding	82.17	81.58	99.3%
SWE-bench Verified	80.6	60.8	75%
Terminal-Bench 2.1	69.7	52.8	75%

SWE-bench Verified and Terminal-Bench 2.1 do not fit that pattern. PrismML's own table shows the ternary model scoring 60.8 against a baseline of 80.6 on SWE-bench Verified, and 52.8 against 69.7 on Terminal-Bench 2.1, both a 75 percent retention rate, well below the headline 98.2 percent and below every other category PrismML reports.

Why the gap concentrates in agentic tasks

SWE-bench Verified and Terminal-Bench 2.1 both score multi-step tool use inside a long-running session, the kind of task where a single wrong token early in a tool call can compound across dozens of subsequent steps. A static benchmark that scores one answer per question is far more forgiving of the small errors ternary quantization introduces than a benchmark that scores whether an agent's twentieth action still makes sense given its first.

PrismML's whitepaper does not explain the gap beyond noting it, and neither the company nor Hugging Face's model card names an independent party who has reproduced any of the 20 scores. Cactus Compute's Needle 3, a similarly compressed model already covered on this beat, made comparable claims about matching a larger model's tool-calling ability at a fraction of the size; the pattern of a strong aggregate score built on categories that do not include the hardest agentic tests is becoming familiar in on-device model releases generally, not unique to PrismML.

1. Nvidia RTX 5090142.5 tokens/sec
2. Nvidia RTX 409096.7 tokens/sec
3. Apple M5 Max46.8 tokens/sec

Source: MarkTechPost, accessed 2026-09-19

The throughput numbers are the part of the release with the clearest hardware story: PrismML reports 142.5 tokens per second on an Nvidia RTX 5090 and 46.8 tokens per second on an Apple M5 Max, both at batch size 1, figures that matter more to a developer deciding whether the model fits a laptop than the aggregate benchmark score does.

What would change this read

An independent evaluation of SWE-bench Verified or Terminal-Bench 2.1 against the released weights, run outside PrismML's own fork of llama.cpp, would settle whether the 75 percent retention is a property of the quantization or an artifact of PrismML's own harness. Until that exists, a developer choosing Ternary Bonsai 2 27B for an agentic coding task is relying on the one set of numbers in the release that the model's own maker did not lead with.

The denominator that matters for a buying decision is not the 20-benchmark average but whichever single benchmark looks most like the job the model is being bought to do. A chatbot deployment that never calls a tool is well served by a model retaining 96 to 99 percent of its baseline on math, coding and instruction following. An agent that edits files across a multi-step session is being asked to run on the 75 percent number instead, and PrismML's release puts that figure in a table three rows below the one it put in its own headline.