Colibrì Runs a 744B Model in 25GB of RAM

A single-developer project called Colibrì has gained 35,406 GitHub stars in the roughly ten weeks since its July 1 release, including 1,546 in the 24 hours before this piece was reported, according to the repository's own statistics. It does this by running a mixture-of-experts model, a design that routes each token to a small subset of the model's total parameters instead of all of them, with a total size of 744 billion parameters on a machine with as little as 25 gigabytes of free memory.

How it fits a 744-billion-parameter model into 25GB

Colibrì, written by developer Vincenzo Fornaro and released under the Apache 2.0 license, keeps a model's dense layers, roughly 17 billion parameters for GLM-5.2, resident in RAM as 4-bit integers, about 9.9 gigabytes. The other 370-odd billion parameters, the routed experts that MoE architectures only activate a few of per token, stay on disk and stream in on demand with an LRU cache, according to the project's README. The documented minimum is 16GB of RAM and about 380GB of free disk for the int4 model container; Colibrì's own quickstart guide recommends 24GB of RAM and states plainly that "the experts stream from storage, so a fast NVMe SSD is the single biggest factor in tokens/second." A GPU is optional; the engine runs CPU-only by default and is a single C file with no BLAS library and no CUDA dependency at runtime.

What speed actually looks like

The project documents four configurations, and the range between them is the entire story.

1. 25GB minimal setup0.1 tokens/sec
2. Single RTX 5070 Ti laptop1.07 tokens/sec
3. 128GB CPU desktop (warm)1.8 tokens/sec
4. 6x RTX 5090, full residency6.8 tokens/sec

Source: JustVugg/colibri README, accessed Sept. 17, 2026

At the low end, a 25GB setup runs at 0.05 to 0.1 tokens per second, meaning a single sentence of output can take several minutes. At the top end, six RTX 5090 GPUs holding the whole model in VRAM reach 5.8 to 6.8 tokens per second, which is still slow next to a hosted API but usable for interactive work. Colibrì also supports eight other model families at various sizes, including DeepSeek V4.1 Flash (552 billion total parameters, 16 billion active), Kimi K3 (2.8 trillion total, 104 billion active) and the smaller 7-billion-parameter OLMoE, so the RAM and disk figures above are specific to GLM-5.2 and shift with each model's own active-parameter count.

The two objections that showed up on Hacker News

The project's Show HN post reached 937 points and 240 comments, and the most substantive pushback fell into two categories. The first was simple usability: one commenter wrote that 0.05 to 0.1 tokens per second "isn't really usable for much," to which Fornaro responded that the intended use case is overnight batch jobs and asynchronous generation, not interactive chat. The second was a hardware-wear concern specific to the disk-streaming design: cold starts pull roughly 11 gigabytes of random reads per generated token, and commenters said that while reads themselves do not damage flash memory, the operating system's page-cache writes triggered by that traffic could accelerate wear on a laptop's soldered, non-replaceable SSD faster than ordinary use would, a real risk for anyone running the 25GB configuration on a machine they cannot easily repair.

A third comment thread questioned the premise more directly: llama.cpp, the widely used C++ inference engine, already supports memory-mapped model loading and quantization, so several commenters asked what Colibrì's disk-streaming approach adds that mmap-based swapping does not already do. Neither the HN thread nor Colibrì's own documentation, as read for this piece, settles that comparison with a benchmark against llama.cpp on the same hardware and the same model.

Colibrì is not the only project to jump from a single day's GitHub trending list to a five-figure star count; Alibaba's Open Code Review tool gained 3,215 stars in a day in September, and Cloudflare's security-audit skill picked up 1,249 in a day that same week. In all three cases, the star count measures how many people clicked a button, not how many kept the tool running a week later.

What the project does not promise

Colibrì's documentation is unusually direct about the tradeoff it is making. It states there is "no SLA on speed" but a "hard guarantee on semantics": running short on fast memory slows the engine down, but the README says it will never silently change a model's numerical precision or its expert-routing decisions to compensate. That is a narrower and more checkable claim than most local-inference tooling makes, and it is also one this piece has not independently verified against the underlying weights.

What would change the read on this

A companion project, Lumabri, extends the same engine to stream experts across a peer-to-peer swarm of machines rather than a single disk, which would change the economics again if it works at comparable reliability. For Colibrì itself, the open question is adoption rather than technique: the repository carries 135 open issues, and stars measure attention, not usage. What would change the read is whether the 0.05-to-0.1-token-per-second tier attracts real overnight workloads over the next few months, or whether, as several Hacker News commenters predicted, most users try it once on a 25GB machine and go back to a smaller, faster model.