Trakkr Traced 1,285 Statistics From Five AI Assistants, and 40% Reached a Study
A.I. / news
Trakkr Traced 1,285 Statistics From Five AI Assistants, and 40% Reached a Study
A Trakkr Research test of ChatGPT, Perplexity, Gemini, Claude and AI Overviews found most cited numbers end at pages that cannot be checked.
Trakkr Research traced 1,285 statistics quoted by five AI assistants back to their origins and found that 40 percent reached a study saying the same thing, according to a study published Oct. 28 by Mack Grenfell. Another 40 percent stopped at a source that could not be checked, and 11 percent led nowhere. The page does not give Grenfell's job title.
Trakkr published the study on its own site, and the numbers below are Trakkr's, from its report, not from an independent audit. The Terminal has not rerun the test.
How the test was run
Trakkr asked ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews 200 questions that call for a number. It used each assistant's API with web search switched on and a US location, on Oct. 25. That produced 1,000 answers and 3,167 statistics, 2,419 of them with citations. Trakkr then followed a random sample of 1,285 cited statistics up to five pages deep. An AI judge scored the trails, and Trakkr checked 150 by hand.
| Outcome | Share of traced statistics |
|---|---|
| Reached a study saying the same thing | 40% |
| Reached a study, but number or context changed | 8% |
| Stopped at an uncheckable source | 40% |
| Led nowhere | 11% |
The shares sum to 99 percent in Trakkr's report, a rounding gap the page does not explain. Trakkr gave a 95 percent confidence range of 34 percent to 46 percent for the first figure.
Which assistants did best
ChatGPT had the highest sourced rate at 57 percent, and the other four ranged from 38 percent to 46 percent, Trakkr said. The cited page contained the figure 82 percent of the time, but only 21 percent of cited pages were the original research. Company blogs made up 54 percent of first citations, and 37 percent of those trails were sourced.
Of the statistics that did reach a source, 17 percent had changed on the way, most often in context: what was measured, the year, or who was asked. On 28 of 71 questions where all five assistants gave a percentage, they gave four or five different figures.
- ChatGPT (sourced rate)57 %
- All assistants combined40 %
Source: Trakkr Research, Study 013, published 2026-10-28
The examples Trakkr names
Trakkr's report names "44% of salespeople give up after one follow-up", which all five assistants quoted. Every trail ended at a page that cites no research, including a freelance writer's sample article. It also names "94% of first impressions are design-related", which began as a 2004 study of 15 women's comments about health websites.
The report's most prominent AI example is the MIT NANDA figure that 95 percent of organisations saw no return from generative AI. Trakkr says the study reports no return for 95 percent of organisations, not that 95 percent of pilots fail. The Register's August 2025 summary of the same report gives the sample as 52 interviews, a survey of 153 business professionals and more than 300 public initiatives, and does not define what counts as zero return.
The Terminal has linked claims like this before. A colonoscopy study stretched over SRE work and a vendor-run benchmark are the same failure at different scales.
Trakkr said "a citation tells you where the AI found a number, not where the number came from". The study covers one day of queries in one country, and Trakkr has not said whether it will rerun it.
Sources
More in A.I.
- 01GPT-Rosalind-6 Reuses GPT-6 Astra's Weights, Cuts Biology Refusals to 9.4% and Stays Behind a GateOpenAI's Oct. 30 system card addendum says no training was added, so the difference from Astra is deployment controls and who is allowed in.
- 02Architect Says Claude Opus 5.5 Drew a House Set Good Enough for a Permit After Two Redline RoundsStephen Gallagher's informal test of 13 models gave Opus 5.5 the most human feedback, and the drawings still missed a required second exit.
- 0350 GPT Agents in Minecraft Built a Ration Ledger, and 34 Died Doing ItY Combinator's Labs group and Olam Labs ran GPT-6.1 Sol and Claude Opus 5.5 swarms through Minecraft and Mindustry. GPT won both head-to-head games, but each result comes from one harness and one setup.
- 04China's Announcement No. 48 Puts 23 AI Technologies on a Draft Export ListExporters of listed technology must already register with MOFCOM. Comments on the draft catalogue close on November 15, and the sources disagree on whether the thresholds are published.