PrismML Shrinks a 27B Model to 5.9GB at 1.72 Bits

PrismML released Ternary Bonsai 2 27B on Wednesday, a version of Alibaba's Qwen3.8-27B rebuilt so nearly every weight takes one of three values, shrinking the model from roughly 54GB at FP16 to 5.95GB while the company says it keeps 98.2 percent of the original's benchmark score.

The model card describes a ternary format called g128: each weight is -1, 0 or 1, with one shared 16-bit scale factor for every group of 128 weights. PrismML counts the true cost at 1.72 bits per weight once that shared scale is amortized in, calling out that a widely used 2-bit build of the same base model actually runs 2.8 bits per weight once its own scaling overhead is counted. The company's own 14-benchmark suite puts the ternary version at an 84.78 average against 86.32 for the FP16 original, with math within half a point (96.57) and coding close behind (89.42) while vision showed the steepest drop.

An outside count lands lower

AlphaSignal, an independent AI newsletter, published its own read of the release the same day and reported an overall score of 83.9 against 85.4 for FP16, a 1.5-point gap, and put the effective density at 1.76 bits per weight rather than PrismML's 1.72. Neither figure is far from the other, and both describe the same direction, that Bonsai 2 narrows the retention gap from the roughly 95 percent PrismML's original Ternary Bonsai family claimed in April. But the difference in whose benchmark suite produced which number is exactly the kind of detail lost when only one side publishes.

The format needs its own tooling to run. PrismML's fork of llama.cpp carries custom kernels for the two packing formats it ships, called PTQ1_0 and PQ2_0; the model card warns that stock llama.cpp rejects both formats outright or, in the case of a related low-bit type it does recognize, loads the file and produces garbage without any warning, because it lacks the Hadamard-rotation step the format depends on.

What it takes to run

PrismML's own throughput table shows the packings trading places depending on the hardware:

Platform	PQ2_0 tok/s	PTQ1_0 tok/s
RTX 5090 (32GB)	129.9	120.5
H100 SXM (80GB)	113.9	86.9
RTX 4090 (24GB)	81.2	91.1
Apple M5 Pro (laptop)	28.1	not tested

On the Qwen3.8-27B base model, FP16 does not fit into a laptop's memory at all; PrismML's point is not a speedup ratio but that a 27-billion-parameter model now runs interactively on a machine that could not previously hold it. A separate PrismML build of the same weights for Apple's MLX runtime is available for native Apple Silicon inference, alongside the two llama.cpp packings.

PrismML, which grew out of Caltech research with backing from Khosla Ventures, Cerberus and Google, has not said whether the same ternary treatment is coming to larger models, the way Colibri's disk-streaming approach tackled a similar memory problem from a different angle, or whether the roughly 0.9-point gap between its own 84.78 average and AlphaSignal's 83.9 will narrow once more independent runs are published.