FreeToken, With 16.2k Stars, Promises 290B Models on a Gaming PC and Publishes No Tokens-a-Second Figure

FreeToken, an Apache-2.0 serving engine for mixture-of-experts models, has 16,200 GitHub stars and a headline that reads "Run 290B+ frontier MoE models locally on your gaming PC at blistering interactive speeds." The repository page does not contain a single tokens-a-second number.

The project sits under the GitHub organisation FlashML-org, with a website at flashml.ai. The page names no company and no maintainers. It cites a paper, arXiv:2608.16157, "FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution", submitted on August 17, 2026 by eleven authors including Kurt Keutzer, Song Han, Matei Zaharia and Ion Stoica. The arXiv listing shows names only, with no affiliations.

What the repository says it does

The README lists bandwidth-adaptive CPU and GPU co-execution, LRU expert caching, a custom "FTW" weight format and semantic checkpoints for KV and recurrent caches. It also lists runtime VRAM reallocation between the expert cache and KV memory "without engine restarts or weight reloading." Named models are DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2, in MXFP4, NVFP4, FP8 and BF16.

The engine exposes Anthropic-compatible and OpenAI-compatible APIs, and the README lists Codex, Claude Code and OpenCode as clients. Hardware support is NVIDIA RTX 30, 40 and 50 series only. AMD and Apple silicon are not mentioned, although the headline says consumer hardware.

What is missing from the page

The README gives no throughput, no RAM or VRAM amounts, no GPU model with a memory size, and no comparison with llama.cpp, which appears only in the acknowledgments. The arXiv abstract adds model scale instead of speed: a 35B model on a laptop, a 284B model on a gaming desktop and the 753B GLM-5.2 on a single workstation GPU. It gives no speedup figure and no baseline. The 290B in the headline and the 284B in the abstract are different numbers, and the page does not reconcile them.

The contrast with a comparable project is stark. Colibri, another expert-streaming engine, publishes a table: Qwen3.6-35B-A3B at 104.8 tok/s on CUDA and 13.9 on CPU on an i5-13500 with 62 GB of RAM, plus GLM-5.2 at 1.83 tok/s on a 128 GB Ryzen AI Max+ 395. Whether FreeToken is faster is not answerable from its own page. The paper's full text may answer it; the abstract page did not.

Fact on the repository page	FreeToken	Colibri
Stars	16.2k	46.7k
Commits	198	5,150
Open issues	284	65
Licence	Apache-2.0	Apache 2.0
Tokens a second given	none	per model and machine

What the issue tracker adds

The issue list shows 12 pages. Issue #822, opened October 24, reports a performance regression from v0.1.3 to v0.2.0 on an RTX 4060 with 8 GB; it has 6 comments, the most of any recent non-roadmap issue. Issues #784, #782 and #847 report the desktop app failing to load weights, including Qwen3.6-35B-A3B in NVFP4 and Gemma-4-12b-it in NVFP4. Issue #826 reports an error installing an engine update, and #798 an engine crash with a high KV profile.

The desktop app is distributed from flashml.ai, not from GitHub Releases, which were empty on the captured page. The repository also holds an install.sh script, which this report did not read. The README's acknowledgments say code was reused from SGLang, vLLM, FlashInfer, LightLLM and llama.cpp.

A reader with a 24 GB card who wants numbers should wait for the paper's tables or run the benchmarks folder themselves, after reading what the desktop installer fetches. Issue #79, the 2026 roadmap, has 17 comments and is the place to watch for a baseline. The site has covered the same kind of gap before, in a terminal emulator's 870 fps claim with no hardware listed and an agent tool's macOS bug.