Researcher Says Claude Fable 5.2 Learned Protein Sequences; Test Shows a 45-Point Gap
A.I. / news
Researcher Says Claude Fable 5.2 Learned Protein Sequences; Test Shows a 45-Point Gap
Xinming Tu's probe of 668 obscure bacterial proteins finds the newest Claude models predict real sequences far better than shuffled ones. GPT, Gemini and open-weight models do not.

An independent researcher says the latest Claude models were trained on protein sequences at scale, and that GPT, Gemini and seven open-weight models were not. The claim, published Nov. 18 by Xinming Tu on a personal blog, rests on a test that asks models to continue the sequences of little-studied bacterial proteins.
The post cites no statement from Anthropic. The result is a single-author blog post and has not been peer reviewed, the author says.
How the test works
Tu built a set of 668 bacterial and archaeal proteins from 167 Pfam families of unknown function. No family has a solved structure or more than three papers, and no protein has a relative at 90 percent identity or more in UniProt. The idea is that models cannot have learned these from papers about them.
Each protein is cut at five random points, giving 3,340 cuts. The model gets the start and must predict the next residue. As a control, the same protein is shuffled, which keeps its amino acid composition and destroys order. The score is accuracy on real sequences minus accuracy on shuffled ones, in percentage points.

The numbers
The repository, XinmingTu/LLM-bioseq-probe on GitHub under an MIT licence, lists the results with 95 percent confidence intervals.
| Model | Real vs. shuffled | Gap, points |
|---|---|---|
| Claude Fable 5.2 | 54.1% vs. 9.0% | +45.1 |
| Claude Opus 5.5 | 34.9% vs. 7.5% | +27.4 |
| Claude Fable 5.1 | 21.4% vs. 6.6% | +14.9 |
| Claude Opus 4.7 | 16.1% vs. 7.2% | +8.9 |
GPT-5.5, GPT-6 Astra, GPT-6.1 Sol, Gemini 4 Carbon, Opus 4.5 and 4.6 all fall between +0.6 and +2.9. Gemini 2.5 Pro and seven open-weight models fall between 0.0 and +1.9, the README says.
- Claude Fable 5.245.1 points
- Claude Opus 5.527.4 points
- Claude Fable 5.114.9 points
- Claude Opus 4.78.9 points
Source: XinmingTu/LLM-bioseq-probe README, accessed 2026-11-18
Tu compares the Claude results with protein language models. ProGen3-3B scores +32.5 and ProtGPT3-10B scores +27.7, so Fable 5.2 beats both and Opus 5.5 is roughly level with ProtGPT3-10B. On 56 proteins first published in August 2026, Fable 5.2 still shows +22.1 and Opus 5.5 about +13, while GPT-6.2 Sol, the model in this pricing report, sits near zero.
The gap is positive in 98 percent of families for Fable 5.2 and 94 percent for Opus 5.5. It rises with family size, as it does for protein language models.
What the post does not show
Tu writes that the results do not show how the sequences entered training. Opus 5.5 predicts human and mouse coding DNA but not non-coding regions or bacterial genes, so Tu concludes it "learned proteins, not genomes."
The Fable models were run through OpenRouter, and Anthropic's filter blocked 44 percent of Fable 5's questions, so Fable 5 is left out of the main figures. The post-cutoff set is small, and gaps under about 14 points are not reliably detected on it. The July 2026 training cutoff is an assumption. The full test cost about $730 in API fees.
Objections on Hacker News
The post reached 158 points on Hacker News on Nov. 18. A commenter, pcrh, wrote: "Testing the model against shuffled sequences is also kinda cheating, as these are no longer random pool of amino acids, but one whose source already exists in the databases." Another, octoberfranklin, asked how there can be a correct next residue for a shuffled protein that does not occur in nature.
The shuffled control answers a narrower question than those comments raise. It measures whether order carries information the model has seen, and the author says nothing about whether the model could fold a protein or rank variants. Tu lists both as untested.
The code, test sets and outputs are public in the repository, so an outside group can rerun the probe. The sources reviewed contain no description from Anthropic of its protein training data.
On the product side, the site's report on a 39-item verification checklist for Claude Code teams covers how one team checks Claude output.
Sources
More in A.I.
- 01Gemini 4 Carbon Takes 36 Minutes a Task; Kernel Counts 7x the Steps of AstraArtificial Analysis puts Carbon at 36.0 minutes per Intelligence Index task against a 12.4-minute median, and a Kernel study traces most of the gap to extra steps.
- 02Tencent's Hy4-flash Weights Carry a License That Requires a Separate Deal Above $50 Million in RevenueThe 12B-active model is ungated on Hugging Face, but it ships under the Tencent Hy License, not Apache 2.0.
- 03OpenAI's GPT-6.2 Astra Lists at $8 per Million Input Tokens, 20% Under Its Predecessor, While Coding Scores Stay FlatArtificial Analysis ranks Astra second at 62 behind Claude Fable 5.2, and Sol's coding-agent score fell 3 points.
- 04Google Put Gemini 4 Hydrogen Behind Search for Everyone on Nov. 12, and Published No Click FiguresThe one randomised study of AI Overviews found a 39.8% drop in outbound clicks on early-2026 Search.