DeepSeek's V4.1 Flash Tops Its Own Charts, Fails MindStudio's Tests
A.I. / news
DeepSeek's V4.1 Flash Tops Its Own Charts, Fails MindStudio's Tests
The 552-billion-parameter model beat DeepSeek's flagship on the company's benchmark suite, but an independent tester watched it botch a Rubik's Cube simulation two days after launch.
DeepSeek released DeepSeek-V4.1-Flash on Sept. 10, a 552-billion-parameter model the company said beats its own flagship, V4-Pro, on cost and speed. Two days later, an independent tester watched the model fail a Rubik's Cube simulation and a Microsoft Paint-style portrait task that its benchmark scores gave no hint of.
The Hangzhou company announced the release on its own site and detailed pricing in its API changelog. V4.1 Flash uses what DeepSeek calls an asymmetric causal encoder-decoder architecture, activating 8 billion parameters to read a prompt and 16 billion to generate a response, out of 552 billion total. The model adds native visual understanding, reads a context window of 1 million tokens, and is licensed under MIT with weights posted on Hugging Face, where it has drawn 482,270 downloads in the past month, joining Alibaba's Qwen3.8-27B among the open-weight releases also drawing heavy download counts this month.
What a million tokens costs, on- and off-peak
DeepSeek priced the new model below its predecessor. Cache-hit input tokens run $0.003 per million off-peak and $0.006 at peak; cache misses run $0.15 and $0.3. Output tokens cost $0.6 per million off-peak and $1.2 at peak. Peak hours are set as 01:00 to 04:00 and 06:00 to 10:00 UTC on weekdays, excluding Chinese public holidays.
| Token type | Off-peak (per 1M) | Peak (per 1M) |
|---|---|---|
| Input, cache hit | $0.003 | $0.006 |
| Input, cache miss | $0.15 | $0.3 |
| Output | $0.6 | $1.2 |
DeepSeek said the model was trained on a 45-trillion-token multimodal corpus, with context extension to 1 million tokens carried out over an additional 34 trillion tokens. A new key-value caching scheme, which the company calls CSA2 paired with FP4 storage, cuts memory per token to roughly 890 bytes, about a quarter of what V4-Flash needed.
Why V4-Pro is being switched off
V4-Flash and V4-Flash-Vision-Exp are retired outright; API calls using their old names now route silently to V4.1-Flash. V4-Pro gets a longer runway: DeepSeek began routing all V4-Pro requests to V4.1-Flash, billed at V4.1-Flash's lower rate, starting at 04:00 UTC on Sept. 14, and said it will keep doing so until a V4.1-Pro model ships. The company has not given a date for that release.
The scores are DeepSeek's own
DeepSeek's benchmark suite credits V4.1 Flash with 90.9 on GPQA Diamond, a 3,471 Codeforces rating, 90.6 on Terminal-Bench 2.1, 74.2 percent on DeepSWE v1.1, 88.1 on CyberGym and 65.4 on NL2Repo-Bench, all ahead of V4-Pro on the same tests. Every one of those figures is vendor-supplied; DeepSeek's release notes do not name an outside lab that reproduced them, and the company has not published the raw transcripts a third party would need to check the runs. It would not be the first open-weight release this month to hide a shortfall behind a strong score: PrismML's Ternary Bonsai 2 had the same problem.
What broke when MindStudio tried it
MindStudio ran four hands-on tests on Sept. 12, two days after launch, edited by Luis Chavez-Mattos, the site's director of product. An SVG-drawing task finished in 59 seconds for under two cents but looked visibly worse than the benchmark scores implied. A Rubik's Cube simulation drew a geometrically correct cube, then broke: colors changed incorrectly during scrambling, and the solve function simply replayed the scramble moves in reverse instead of computing a solution. A request to recreate a portrait in the style of Microsoft Paint produced a flat image lacking the layered brushwork stronger models managed. A ray-traced water-droplet simulation ran but looked unconvincing.
MindStudio measured throughput at 200 tokens per second, fast enough to write a 1,000-word essay in about six seconds. Its writeup put the gap plainly: the model "builds plausible-looking scaffolding quickly, but the underlying logic frequently breaks under scrutiny." DeepSeek's own announcement does not mention any of the four tasks MindStudio ran, and nothing in the release addresses the gap between fixed-rubric benchmark scores and open-ended tasks that require the model to reason consistently from step to step.
DeepSeek has not said when V4.1-Pro will replace the model V4-Pro users are being routed away from, which is the number to watch next: it decides how long V4.1-Flash has to carry both tiers of traffic on its own.
Sources
More in A.I.
- 01Anthropic's Levent Alpöge Claims a Claude-Assisted Riemann Proof, and Lean Checks Are InImperial's Kevin Buzzard and MIT's Andrew Sutherland have examined the formalization, according to Scientific American. No journal review has been reported.
- 02Thinking Machines Ships Inkling 2 Mini, a 26.4B Apache 2.0 Model Sized for One DGX SparkThe model card lists 3.4B active parameters, a 1M-token window and a 20GB checkpoint, but does not say who ran most of its benchmark rows.
- 03Anthropic Closed Its Debut at $97.17, 17% Over the $83 IPO Price, as Nasdaq Fell 2%The $81.34 billion offering opened 34% higher, then gave back most of the gain while Nvidia, AMD and Micron dropped.
- 04Google's Gemini Tool Retrieval Preview Adds 26 Tokens Per Idle Tool and Publishes No Savings FigureAnthropic and OpenAI each reported token reductions for the same defer_loading flag. Google's November 19 post reports none.