DeepSeek's V4.1 Flash Tops Its Own Charts, Fails MindStudio's Tests
A.I. / news
DeepSeek's V4.1 Flash Tops Its Own Charts, Fails MindStudio's Tests
The 552-billion-parameter model beat DeepSeek's flagship on the company's benchmark suite, but an independent tester watched it botch a Rubik's Cube simulation two days after launch.
DeepSeek released DeepSeek-V4.1-Flash on Sept. 10, a 552-billion-parameter model the company said beats its own flagship, V4-Pro, on cost and speed. Two days later, an independent tester watched the model fail a Rubik's Cube simulation and a Microsoft Paint-style portrait task that its benchmark scores gave no hint of.
The Hangzhou company announced the release on its own site and detailed pricing in its API changelog. V4.1 Flash uses what DeepSeek calls an asymmetric causal encoder-decoder architecture, activating 8 billion parameters to read a prompt and 16 billion to generate a response, out of 552 billion total. The model adds native visual understanding, reads a context window of 1 million tokens, and is licensed under MIT with weights posted on Hugging Face, where it has drawn 482,270 downloads in the past month, joining Alibaba's Qwen3.8-27B among the open-weight releases also drawing heavy download counts this month.
What a million tokens costs, on- and off-peak
DeepSeek priced the new model below its predecessor. Cache-hit input tokens run $0.003 per million off-peak and $0.006 at peak; cache misses run $0.15 and $0.3. Output tokens cost $0.6 per million off-peak and $1.2 at peak. Peak hours are set as 01:00 to 04:00 and 06:00 to 10:00 UTC on weekdays, excluding Chinese public holidays.
| Token type | Off-peak (per 1M) | Peak (per 1M) |
|---|---|---|
| Input, cache hit | $0.003 | $0.006 |
| Input, cache miss | $0.15 | $0.3 |
| Output | $0.6 | $1.2 |
DeepSeek said the model was trained on a 45-trillion-token multimodal corpus, with context extension to 1 million tokens carried out over an additional 34 trillion tokens. A new key-value caching scheme, which the company calls CSA2 paired with FP4 storage, cuts memory per token to roughly 890 bytes, about a quarter of what V4-Flash needed.
Why V4-Pro is being switched off
V4-Flash and V4-Flash-Vision-Exp are retired outright; API calls using their old names now route silently to V4.1-Flash. V4-Pro gets a longer runway: DeepSeek began routing all V4-Pro requests to V4.1-Flash, billed at V4.1-Flash's lower rate, starting at 04:00 UTC on Sept. 14, and said it will keep doing so until a V4.1-Pro model ships. The company has not given a date for that release.
The scores are DeepSeek's own
DeepSeek's benchmark suite credits V4.1 Flash with 90.9 on GPQA Diamond, a 3,471 Codeforces rating, 90.6 on Terminal-Bench 2.1, 74.2 percent on DeepSWE v1.1, 88.1 on CyberGym and 65.4 on NL2Repo-Bench, all ahead of V4-Pro on the same tests. Every one of those figures is vendor-supplied; DeepSeek's release notes do not name an outside lab that reproduced them, and the company has not published the raw transcripts a third party would need to check the runs. It would not be the first open-weight release this month to hide a shortfall behind a strong score: PrismML's Ternary Bonsai 2 had the same problem.
What broke when MindStudio tried it
MindStudio ran four hands-on tests on Sept. 12, two days after launch, edited by Luis Chavez-Mattos, the site's director of product. An SVG-drawing task finished in 59 seconds for under two cents but looked visibly worse than the benchmark scores implied. A Rubik's Cube simulation drew a geometrically correct cube, then broke: colors changed incorrectly during scrambling, and the solve function simply replayed the scramble moves in reverse instead of computing a solution. A request to recreate a portrait in the style of Microsoft Paint produced a flat image lacking the layered brushwork stronger models managed. A ray-traced water-droplet simulation ran but looked unconvincing.
MindStudio measured throughput at 200 tokens per second, fast enough to write a 1,000-word essay in about six seconds. Its writeup put the gap plainly: the model "builds plausible-looking scaffolding quickly, but the underlying logic frequently breaks under scrutiny." DeepSeek's own announcement does not mention any of the four tasks MindStudio ran, and nothing in the release addresses the gap between fixed-rubric benchmark scores and open-ended tasks that require the model to reason consistently from step to step.
DeepSeek has not said when V4.1-Pro will replace the model V4-Pro users are being routed away from, which is the number to watch next: it decides how long V4.1-Flash has to carry both tiers of traffic on its own.
Sources
More in A.I.
- 01TypeSafe's $40M Model Launch Ran Into a Researcher's Year-Old PaperNandakishor Mukkunnoth says his March 2025 paper described the same non-autoregressive decision architecture TypeSafe is now calling a breakthrough, and he answered by open-sourcing a faster version.
- 02NASA and IBM Open-Source a Moon Model That Warns Against Trusting ItThe model cuts ice-detection error by 22 percent against a standard baseline, but its own documentation says its ice maps are not measured ice and it is not fit for landing-site decisions.
- 03Alibaba's Qwen3.8-27B Fits a Frontier Model on One GPUThe 27.8 billion-parameter dense model, released Aug. 14, ties DeepSeek V4 Flash on Artificial Analysis's Intelligence Index while needing one GPU instead of two, according to Yotta Labs.
- 04OpenAI Publishes Six Misalignment Reports, No Deadlines AttachedThe framework sorts incidents into three review tracks, but it sets no publication deadline and names no outside body that can check what OpenAI chooses not to disclose.