Dettmers Says a 552-Billion-Parameter Model Fits on a MacBook

Tim Dettmers says his lab's framework runs a 552-billion-parameter model on a laptop. Nobody outside his lab has run the code, because it has not shipped.

The short version

Tim Dettmers, an assistant professor at Carnegie Mellon University and a research scientist at the Allen Institute for AI, wrote in a blog post published Sept. 21, 2026 that his lab's inference framework can run Alibaba's Qwen 3.8 Flash Next, a 125-billion-parameter model, on a single 24-gigabyte consumer GPU, and DeepSeek's V4.1 Flash, which he describes as a 552-billion-parameter model, on an AMD Strix machine, an Nvidia DGX Spark, or a MacBook with 128 gigabytes of memory. He also says the framework runs a smaller model, Qwen 3.6 35B-A3B, at 450 tokens per second while storing its weights at 1.5 bits each, which he says uses about a tenth the memory of ordinary 16-bit precision. None of this is available to try yet. Dettmers says the code ships as part of what he calls Open Source Week, delayed by a day from its original date and due out the day after his post.

The arithmetic that checks out

Take the one claim Dettmers actually shows enough to check. Sixteen-bit precision stores each of a model's weights in 2 bytes, so Qwen 3.6 35B-A3B's 35 billion parameters need about 70 gigabytes of memory at that precision. At 1.5 bits per weight, the same 35 billion parameters need about 6.6 gigabytes, a ratio of roughly 10.6 to 1. That lines up with the "about a tenth of the memory" figure Dettmers wrote, using math he did not spell out himself but which holds up once spelled out.

Model	Total parameters	Active per token	License
Qwen 3.8 Flash Next	125B (180B counting embedding and prediction modules)	6B	Qwen Community License 1.0
DeepSeek V4.1 Flash	552B	8B (prefill) or 16B (decode)	MIT

1. 16-bit precision70 GB
2. 1.5-bit precision6.6 GB

Source: Tim Dettmers, dlab Open Source Week blog post, accessed 2026-09-22

What is not released, and what is not said

The two bigger claims do not come with the same math. Dettmers does not say what technique lets a 125-billion-parameter model, itself a preview of the architecture behind Qwen 3.8's more conventional 27-billion-parameter release, or a 180-billion-parameter one once the embedding and prediction modules on Alibaba's own model card are counted, fit inside 24 gigabytes of GPU memory at any precision much above 1 bit per weight. A mixture-of-experts model like this one routes each token through only 6 billion of its parameters across 512 experts, which makes it plausible that a framework could keep the active experts on the GPU and the rest in slower system memory. Dettmers' post does not say that is what his framework does, and no code is public yet to check.

The 450-tokens-per-second figure, the 50 percent cost reduction Dettmers attributes to an auto-compaction technique he calls CliffCompaction, and the 45 percent reduction he says one unnamed partner measured after deploying it, are all self-reported in the same post, ahead of any release and without a benchmark harness, hardware configuration or run count attached. Dettmers writes that his lab's research system "beats deep research systems from frontier labs" and outperforms Sakana AI's and Google's ScientistOne on autonomous research tasks, again without a published score. Vendor-supplied figures without an independent run are exactly the kind of number Xiaomi published its own training cost for MiMo-V2.6-Pro alongside, and the caution is identical: a company or a lab describing its own unreleased work is not the same as an outside party measuring it.

The license nobody in the post mentions

Dettmers' post treats Qwen 3.8 Flash Next and DeepSeek V4.1 Flash as interchangeable examples of models that now fit on hardware people already own. They are not licensed the same way. DeepSeek ships V4.1 Flash under the MIT license, which places essentially no conditions on commercial use. Alibaba ships Qwen 3.8 Flash Next under its own Qwen Community License 1.0, which requires that "the licensee shall obtain a separate license from Qwen" before using the model for a model-as-a-service or AI work assistant business, according to the license file Alibaba posted alongside the model, and requires displaying the model's name on the interface of any product built on it once that product passes 100 million monthly users or $20 million in monthly revenue. A researcher running either model on a personal GPU will not notice the difference. A company shipping a product on top of one will.

What would change this read

The test is not whether the arithmetic on the small model holds up; it already does. It is whether Dettmers' framework, the tokens-per-second figure and the cost-reduction numbers survive contact with code anyone outside his lab can actually run. Open Source Week was due to begin the day after his post, releasing, in his words, two open-source projects and four papers. Until those repositories are public and someone other than Dettmers has run them on the hardware he names, the 552-billion-parameter-model-on-a-laptop claim is a professor's description of his own unreleased software, not a result.