Design Creativity Bench Scores GPT-6 Astra 65.5 Against Humans' 92.3, and the Paper Never Names Its Judge

A new benchmark puts the best model it tested, GPT-6 Astra, at 65.5 on UI-design creativity against a human reference of 92.3, and the arXiv paper behind it does not say who or what did the scoring. The numbers surfaced on Hacker News on Oct. 14 through a post by Aman Rusia titled "Why do LLM pelicans look like this?".

The post argues that language models converge on nearly the same picture when asked for an SVG of a pelican riding a bicycle. Rusia calls it "the LLM diversity collapse problem no one is talking about."

Where the pelicans come from

The pelican image is thirteen SVGs from recent models, sourced to runs by Simon Willison, each at the model's highest reasoning setting that returned an SVG. Model names appear only inside the image, not in the post's text.

Willison's tag page calls the test "deeply unscientific." Its latest entries include Gemini 4 Argon on Oct. 13 and Muse Wonder on Oct. 12, and one Claude Sonnet 5.5 entry dated Sept. 28 where the max thinking setting ran out of tokens before producing an SVG.

Willison also links a study by Dylan Castillo that found no evidence labs train for the pelican, which he calls "pelicanmaxxing." That matters here: convergence on one image can come from training data and shared habits as well as from lack of imagination.

What the paper reports

The arXiv paper, "Design Creativity Bench: Measuring creativity in LLM-Generated UI," was submitted on Oct. 8. Its authors are Aman Rusia, Abhijit Bhole, Prashank Gupta and Dipanjan Dey. The arXiv page lists no affiliations, and the post's benchmark links point to a site called Kombai.

The abstract measures originality, creative range and appropriateness. Originality is 0.592 for model-to-model pairs against 0.764 for human-to-model pairs. Creative range is 0.581 across models against 0.902 for human designs.

Measure	Models	Human designs
Originality (pairs)	0.592	0.764 (human-model)
Creative range	0.581	0.902
Appropriateness, best	99.2%	98.0%

The third row cuts against the post's framing. Appropriateness, the share of a brief's acceptance criteria a design meets, exceeds 90 percent for every model, and the best beats the humans.

The leaderboard and what is missing

The post's leaderboard shows GPT-6 Astra at 65.5, GPT-6.1 Sol at 63.7, Grok 4.7 at 62.2 and Gemini 3.8 Flash at 60.6. The post says 16 frontier models were tested on 80 open-ended UI tasks, each adapted to two product domains. The arXiv page gives no model count.

1. Human reference92.3 points
2. GPT-6 Astra65.5 points
3. GPT-6.1 Sol63.7 points
4. Grok 4.762.2 points
5. Gemini 3.8 Flash60.6 points

Source: arcfu.com post by Aman Rusia, Oct. 14, 2026; benchmark authors' figures, not independently run

The arXiv page does not say how the human designs were collected, by whom or how many. It does not say what scored the designs. For a creativity benchmark, the judge decides the result, and a figure like 92.3 cannot be read until the authors say. These are the authors' own figures, and no independent party has reproduced them.

The post shows only four rows. Models the site has reported on this week, such as Meta's Muse Wonder and Gemini 4 Argon, do not appear in them. Willison found Argon produced excellent pelicans on Oct. 13. The dataset is on Hugging Face under the name Kombai/design-creativity-bench, and the authors' leaderboard page is the place to look for the full 16 rows.