Llama-modchip Fits DeepSeek V4 Flash in 64GB RAM, but Its Baseline Is a 2-Bit Quant

A GitHub project called llama-modchip, owned by a user named synchopate, says it runs the 284B-parameter DeepSeek-V4-Flash on a CPU with 64 GB of RAM and no GPU. The repository had 283 stars, 20 forks and 43 commits on the main branch on November 8.

Every figure below comes from the project's own README, so all of it is developer-supplied. Only some of it has been reproduced by anyone else.

What the README claims

The README says its converted checkpoint is 56.56 GiB on disk, against 76.87 GiB for a standard IQ2_XXS file that it says runs out of memory on a 64 GB Linux machine. Peak resident memory at 256K context is listed as 57.61 GiB. DeepSeek's own model card gives the model as 284B total parameters with 13B active per token, a one-million-token context window and an MIT licence.

Measure (developer-supplied)	Figure
Checkpoint on disk	56.56 GiB
Peak resident memory, 256K context	57.61 GiB
Prompt prefill	44.36 tok/s
Decode, no draft model	9.06-10.74 tok/s
Decode with speculative drafter	11.5-14.7 tok/s

The project credits a custom 3.75-bit-per-weight format and a speculative-decoding drafter for the gains. The repository's own licence badge says Apache 2.0.

The baseline is already a 2-bit file

The README's quality table has a row labelled "Standard IQ2_XXS (76.87 GiB, Unpatched)" with a cosine similarity of 1.00000 and a KL divergence of 0.0000. The project's default configuration is then also listed at 1.00000 and described as "Bit-Exact".

Read the table carefully: the reference is the 2-bit quantisation, not DeepSeek's released FP4 and FP8 weights. A match against that row says the project's kernels reproduce the IQ2_XXS output. It says nothing about how far either file sits from the original model. The faster settings, which prune experts from six per token to five or four, are listed at cosine similarities between 0.99126 and 0.99714 against the same reference.

What you cannot do with it

The converted tensors use a custom type, GGML_TYPE_Q2_0, ID 40. The README warns that unpatched llama.cpp, Ollama and LM Studio will fail with "unknown tensor type 40". Users have to run the project's own ./modchip wrapper or its prebuilt Linux x86_64 binary, which means trusting a single-maintainer binary.

The maintainer's headline speeds also come from a Ryzen 9 9955HX. Community benchmark issues opened on November 8 are lower: 20.78 tok/s on a Ryzen 9 9950X with 64 GB of DDR5-6800, and 31.22 tok/s on an EPYC 7402P. The issue titles do not say which model each used, so they cannot be set beside the 284B figures.

The only open issue that is not a benchmark submission reports that the web UI always shows the address 127.0.0.1.

For other open-weight releases trending this week, see our look at NVIDIA's Nemotron transcription model. The project also advertises a mode that points Claude Code at the local model; for how agent harnesses handle context, see our report on Anthropic's agent memory study.

What to watch next is the leaderboard issue template. Until someone posts quality scores for the 56.56 GiB file against DeepSeek's original release, "fits in 64 GB" is the only claim that has been checked by a measurement a reader can see.