Colibri Runs GLM-5.2's 744 Billion Parameters on 16GB of RAM

Colibri, a free inference engine that streams a language model's unused parameters from a hard drive instead of loading them into memory, gained 2,035 GitHub stars Tuesday, pushing its total past 33,500 since Vincenzo Fornaro opened the repository July 1.

Fornaro built Colibri in C with no external dependencies, according to the project's GitHub repository, which he licenses under Apache 2.0. The engine's pitch is specific: run models with hundreds of billions of parameters, the kind that normally need a rack of data-center GPUs, on a single machine with a few hundred gigabytes of disk and as little as 16GB of RAM. The repository's most recent commit landed Tuesday, the same day its star count jumped.

How it fits a 744-billion-parameter model in 16GB

Colibri's own numbers, in its README, show how that works for GLM-5.2, a 744-billion-parameter mixture-of-experts model. The dense portion every request needs, including attention layers and embeddings, stays resident in RAM at 9.9GB when quantized to int4. The other 19,456 routed experts, roughly 370GB at int4, sit on disk and load only when a token actually routes to them.

Model family	Total parameters	Active per token
GLM-5.2 / 5.3	744B	40B
Kimi K3	2.8T	104B
DeepSeek V4.1 Flash	552B	16B
Qwen3.6	35B	3B

What that costs in speed

Colibri's own benchmark table is blunt about the tradeoff. Six RTX 5090 GPUs running the full model in memory manage 5.8 to 6.8 tokens a second. A 128GB CPU-only desktop drops to about 1.8 tokens a second. A single RTX 5070 Ti laptop configuration reaches 1.07. A 25GB "dev box," cold, falls to 0.05 to 0.1 tokens a second, closer to one word every 15 seconds than a conversation.

1. 6x RTX 5090, full residency6.8 tokens/sec
2. 128GB CPU-only desktop1.8 tokens/sec
3. Single RTX 5070 Ti1.07 tokens/sec
4. 25GB dev box, cold0.1 tokens/sec

Source: JustVugg/colibri README, accessed 2026-09-15

The README states its own limit plainly: "no SLA on speed," only "a hard guarantee on semantics," meaning Colibri promises the same output as running the model in full, not any particular pace of getting there. An independent test on an Apple M5 Max machine, published on DEV Community by a developer using the handle jamilxt, measured 1.06 to 1.83 tokens a second on the same GLM-5.2 model, in the same range as Colibri's own CPU figures. That test also logged a 30-to-60-second wait before the first token, and put the result at 10 to 100 times slower than cloud-hosted H100 inference.

Nine models, one open question

Colibri supports nine model families at launch, including DeepSeek's V4.1 Flash, Alibaba's Qwen3.6 and Moonshot's Kimi K3. The project's GLM-5.2 weights are released under MIT, separate from the engine's own Apache 2.0 license. Fornaro's repository does not say how many of Colibri's 3,523 forks or new stars come from people who have actually run a model end to end, as opposed to those who starred it after reading the pitch. GitHub trending pages, which is how Colibri crossed 33,000 stars Tuesday, count stars, not installs, the same distinction that applied when OpenClaw's own star count outran scrutiny of its code.