Claude's Watermark Quality Claim Rests on Single Responses, Blogger Says

Anthropic says watermarking "does not impact the quality of Claude's output." A blog post dated August 24 argues that the evidence behind that sentence covers single responses only.

The post, by a writer who publishes as Keyvan, resurfaced on Hacker News on Saturday. It does not show that Claude's output got worse. It points to a gap between what Anthropic and Google DeepMind tested and what people do with chatbots, which is ask again.

What Anthropic announced on August 14

Anthropic's announcement is dated August 14, 2026 and was updated on September 1 with details of a detection API. It says future Claude models will generate text that carries a watermark. The company ties the change to the EU Code of Practice on Transparency of AI-Generated Content, which it says it signed in July 2026 along with about 190 other signatories.

The method is a version of SynthID-Text, which Google DeepMind published in Nature in 2024. Anthropic says the approach descends from a 2022 proposal by Scott Aaronson. Claude still picks words at random when several are equally good. The watermark changes where that randomness comes from: a secret key and the preceding words decide the choice instead of a standard random number generator.

Anthropic says nothing is added to the text, no hidden characters and no extra tokens. It says internal testing found no effect on content, creativity or readability. It applies the watermark worldwide at launch because it has no durable way to limit it by region.

The SynthID-Text paper and the diversity trade-off

Keyvan's argument starts from the Nature paper itself. The main text says the method preserves response quality. The 56-page supplement, Keyvan wrote, describes a "diversity/detectability trade-off" and says watermarked responses to one prompt are more similar to each other than unwatermarked ones.

The post quotes the supplement saying this "could lower the overall quality of a system which generates many responses then selects the best one." It also quotes the supplement saying the method "does not necessarily preserve diversity across multiple responses." The terminal did not retrieve the supplement, so those lines are reported as quoted by the post.

Anthropic's own page describes how Google tested for quality on its Gemini models. Google DeepMind served watermarked Gemini responses to part of its traffic and found no statistical difference in thumbs-up and thumbs-down ratings, according to Anthropic. A controlled study found human raters could not tell watermarked answers from unwatermarked ones.

Both tests rate one answer at a time. Neither, as Anthropic describes them, compares the spread across many answers to the same prompt.

Turrado's five-prompt test

Keyvan's only new evidence is a reproduction by Natzir Turrado, who ran SynthID-Text on an open model with the paper's settings. He asked for five descriptions of a memorable coastal trip.

In the watermarked run, 4 of 5 responses opened in a near-identical way. In the unwatermarked run, 3 of 5 opened in different ways, per the post. That is five prompts on one model, and it is the only quantitative result in the piece.

Claim	Who made it	Evidence offered
Watermarking does not affect quality	Anthropic, August 14	Internal testing, Google's Gemini traffic test, a human-rater study
Quality holds per response, diversity may not	SynthID-Text supplement, as quoted by Keyvan	Paper's own trade-off section
Watermarked samples open more alike	Natzir Turrado	One open model, five prompts

Keyvan also cites a reply from Sumanth Dathathri on X. The post quotes him saying "if you tried the same prompt with Gemini/Claude many times, you'd get very diverse responses." Keyvan wrote that the "easy fix" Dathathri mentioned was never specified and that Turrado's follow-up had no reply when the post went up.

What other researchers have said about the trade-off

The post cites a February 2026 paper, A Unified Framework for LLM Watermarks, by Thibaud Gloaguen, Robin Staab, Nikola Jovanović and Martin Vechev. The abstract says the framework "highlights an understudied quality-diversity-power trade-off." That language describes a field-wide tension, not a measurement of Claude.

Google's own blog post says SynthID "embeds an imperceptible watermark without impacting the quality, accuracy, creativity or speed" of generation. The same page says the method is "less effective on responses to factual prompts" and that confidence scores "can be greatly reduced" after heavy rewriting or translation.

Anthropic lists its own limits. Detection is unreliable on short samples, watermarks are sparse where only one answer is correct, and a full rewrite will remove the mark. It also says detection cannot separate text Claude wrote from text Claude heavily edited. The detection API is in private preview for regulators, law enforcement, media, fact-checkers, researchers, educational organisations and EU civil society groups.

What the evidence does not show

None of the sources listed below measured diversity in Claude's outputs. Turrado's test used an open model, not Claude. The company's quality statement, as published, does not say whether diversity across repeated samples was tested.

Keyvan's mitigation is rotating the watermark key per request. The post says that dilutes the effect but weakens detection on short texts and raises detection cost. It did not test that either.

Commentator John Gruber has argued that giving up "an iota of clarity, coherence, meaning, quality" for provenance is unacceptable, according to Keyvan's summary. Daniel Jalkut has argued watermarking can preserve randomness. Neither supplied a measurement.

The Claude Fable 5.1 breaking changes the site covered earlier are separate from this. Anthropic's page says older models will gain the watermark "over the coming months" because the EU transition period covers models launched before August 2. Anthropic has not said whether it will publish a diversity measurement, and the detection API stays in private preview.