FreeToken, With 16.2k Stars, Promises 290B Models on a Gaming PC and Publishes No Tokens-a-Second Figure
Software / news
FreeToken, With 16.2k Stars, Promises 290B Models on a Gaming PC and Publishes No Tokens-a-Second Figure
The Apache-2.0 serving engine names ten authors and an arXiv paper. The repository page gives no throughput, no RAM size and no baseline, while its issue tracker carries a v0.2.0 slowdown report.

FreeToken, an Apache-2.0 serving engine for mixture-of-experts models, has 16,200 GitHub stars and a headline that reads "Run 290B+ frontier MoE models locally on your gaming PC at blistering interactive speeds." The repository page does not contain a single tokens-a-second number.
The project sits under the GitHub organisation FlashML-org, with a website at flashml.ai. The page names no company and no maintainers. It cites a paper, arXiv:2608.16157, "FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution", submitted on August 17, 2026 by eleven authors including Kurt Keutzer, Song Han, Matei Zaharia and Ion Stoica. The arXiv listing shows names only, with no affiliations.
What the repository says it does
The README lists bandwidth-adaptive CPU and GPU co-execution, LRU expert caching, a custom "FTW" weight format and semantic checkpoints for KV and recurrent caches. It also lists runtime VRAM reallocation between the expert cache and KV memory "without engine restarts or weight reloading." Named models are DeepSeek-V4-Flash, Qwen3.6-35B-A3B and GLM-5.2, in MXFP4, NVFP4, FP8 and BF16.
The engine exposes Anthropic-compatible and OpenAI-compatible APIs, and the README lists Codex, Claude Code and OpenCode as clients. Hardware support is NVIDIA RTX 30, 40 and 50 series only. AMD and Apple silicon are not mentioned, although the headline says consumer hardware.

What is missing from the page
The README gives no throughput, no RAM or VRAM amounts, no GPU model with a memory size, and no comparison with llama.cpp, which appears only in the acknowledgments. The arXiv abstract adds model scale instead of speed: a 35B model on a laptop, a 284B model on a gaming desktop and the 753B GLM-5.2 on a single workstation GPU. It gives no speedup figure and no baseline. The 290B in the headline and the 284B in the abstract are different numbers, and the page does not reconcile them.
The contrast with a comparable project is stark. Colibri, another expert-streaming engine, publishes a table: Qwen3.6-35B-A3B at 104.8 tok/s on CUDA and 13.9 on CPU on an i5-13500 with 62 GB of RAM, plus GLM-5.2 at 1.83 tok/s on a 128 GB Ryzen AI Max+ 395. Whether FreeToken is faster is not answerable from its own page. The paper's full text may answer it; the abstract page did not.
| Fact on the repository page | FreeToken | Colibri |
|---|---|---|
| Stars | 16.2k | 46.7k |
| Commits | 198 | 5,150 |
| Open issues | 284 | 65 |
| Licence | Apache-2.0 | Apache 2.0 |
| Tokens a second given | none | per model and machine |
What the issue tracker adds
The issue list shows 12 pages. Issue #822, opened October 24, reports a performance regression from v0.1.3 to v0.2.0 on an RTX 4060 with 8 GB; it has 6 comments, the most of any recent non-roadmap issue. Issues #784, #782 and #847 report the desktop app failing to load weights, including Qwen3.6-35B-A3B in NVFP4 and Gemma-4-12b-it in NVFP4. Issue #826 reports an error installing an engine update, and #798 an engine crash with a high KV profile.
The desktop app is distributed from flashml.ai, not from GitHub Releases, which were empty on the captured page. The repository also holds an install.sh script, which this report did not read. The README's acknowledgments say code was reused from SGLang, vLLM, FlashInfer, LightLLM and llama.cpp.
A reader with a 24 GB card who wants numbers should wait for the paper's tables or run the benchmarks folder themselves, after reading what the desktop installer fetches. Issue #79, the 2026 roadmap, has 17 comments and is the place to watch for a baseline. The site has covered the same kind of gap before, in a terminal emulator's 870 fps claim with no hardware listed and an agent tool's macOS bug.
Sources
More in Software
- 01Vino, the Rust DisplayLink Driver, Went Dark After Synaptics Cited Two UK PatentsThe kernel patch series never got a pixel onto a dock. A cease-and-desist letter and a reply on the Linux kernel mailing list now decide whether it gets further.
- 02Claude Sheets and Whiteboards Ship Off by Default for Enterprise; Claude Data Switches On Nov. 19Anthropic's help page renames Claude Dashboards to Claude Data and moves it to general availability, while two new beta tools wait for an admin to enable them.
- 03Chrome's Modern Web Guidance Gives Coding Agents 250 Web Features to Use Instead of LibrariesThe Apache-2.0 skill pack, stewarded by the Chrome, Edge, Firefox and MDN teams, installs into Claude Code and four other agents and targets Baseline browser support.
- 04Git Contributors Draft a Debian-Style Vote on Whether to Keep the Ban on AI-Looking PatchesThe project has not agreed who may vote, how ballots are counted or what the options are, and the policy under dispute rejects any patch that looks AI generated.