Qwen3.8 Answers Shift 18 Points Toward GPT-5.5 Pro in New Test

Prefilling Alibaba's Qwen3.8-2.4T-A95B with a fragment of OpenAI's GPT-5.5 Pro reasoning shifted its answers 18.18 percentage points closer to GPT-5.5 Pro's own, the largest movement among four open models tested, according to an analysis posted Sept. 9.

The analysis, published on GitHub by a developer using the handle wsxiaoys, applies a technique from "Stealing Reasoning Traces from Proprietary LLM APIs," an Aug. 10 paper by eight researchers including Jonas Geiping, a research group leader at the ELLIS Institute Tübingen. The paper showed that Anthropic, OpenAI and Google each return encrypted chain-of-thought blocks, the hidden reasoning steps a model produces before answering, that can be replayed into a weaker model to force them into plaintext. The paper's authors said they recovered 367 pieces of personal information and 182 credentials from 315,320 reasoning blocks scraped from public repositories.

What the test measured

For 45 problems, split evenly across STEM, non-STEM and synthetic puzzles, the analysis generated one ordinary response from each model and one response prefilled with the first 1 percent of GPT-5.5 Pro's decoded reasoning. It then scored how much of GPT-5.5 Pro's visible answer showed up in the first 100 tokens of each model's reply, averaging unigram, bigram and trigram overlap.

Model	Unprefilled	With GPT-5.5 Pro prefill	Change
Qwen3.8 A95B	16.79%	34.97%	+18.18 pp
Kimi K3	31.11%	35.65%	+4.54 pp
Inkling	19.99%	20.45%	+0.46 pp
DeepSeek V4 Flash	27.30%	26.13%	-1.17 pp

Qwen's shift was largest on the 15 STEM problems, up 26.99 percentage points, and smallest on the 15 non-STEM problems, up 12.80 points. The 15 synthetic puzzle questions, generated privately rather than pulled from public benchmark sets, moved 14.75 points.

Why the puzzle result matters more than the STEM one

The author wrote that Qwen barely moved toward Anthropic's Opus 4.8 in an earlier version of the same test posted before Sept. 9, but moved 18.18 points toward GPT-5.5 Pro here, and concluded that "Qwen may have learned from GPT-5.5 Pro, or from a closely related GPT model, rather than from Opus." The synthetic puzzles carry more weight in that argument than the STEM results, since STEM problems are more likely to already appear, worked out in similar ways, across public training data that both labs could have used independently.

Commenters on the Hacker News discussion of the analysis raised that same confound: the only GPT-5.5 Pro reasoning traces publicly available were the ones decoded in the Aug. 10 paper itself, so Qwen's training set may already have absorbed those exact examples rather than a broader pattern of GPT outputs. The analysis does not rule that out, and neither Alibaba nor OpenAI has published a response to it.

Alibaba released Qwen3.8-2.4T-A95B's weights on Hugging Face Aug. 12 under a custom license, describing it as a 2.4-trillion-parameter mixture-of-experts model with about 95 billion parameters active per token. The company's release materials do not describe what reasoning data, if any, the model's post-training used. That gap in disclosure is not new to this release: this outlet found OpenAI's own GPT-6 Astra shipped safety benchmark scores that were entirely vendor-supplied, and Anthropic's Claude Fable 5.1 update added a data-retention change it did not flag as a breaking one. The next test that would firm up the distillation claim is a prefill run using GPT-5.5 Pro reasoning traces the Aug. 10 paper's authors have not yet published.