Stanford's 'Intelligence per Watt' Metric Says Local AI Handles 88.7% of Queries

Twenty models, eight accelerators, one million queries

A team led by Jon Saad-Falcon, working with collaborators including Christopher Ré, tested more than 20 local language models against eight hardware accelerators, running 1 million real single-turn chat and reasoning queries drawn from public traffic. The paper, "Intelligence per Watt: Measuring Intelligence Efficiency of Local AI", posted to arXiv on November 11, 2025, and last revised September 6, 2026, proposes a single number for judging whether a model running on a laptop is good enough to skip the cloud: task accuracy divided by watts consumed.

The metric, intelligence per watt (IPW), measures accuracy per unit of power rather than raw throughput. A chip that answers fast but wrong, or fast only at full power draw, tells a buyer nothing about whether it is worth deploying. Emergent Mind, an independent research-tracking site that reviewed the same paper, summarized one implication: strategic routing between local and cloud systems could yield 64% to 80% energy savings, depending on how accurately a deployment decides which queries to keep local.

The headline number is 88.7%, not 100%

Local models answered 88.7% of the 1 million queries accurately enough to match a frontier cloud model's response, the paper reports, with accuracy varying by domain. That figure assumes routing each query to whichever local model performs best for it, not to one fixed model. By 2025, the strongest single performer in the study was GPT-OSS-120B, running on an Apple M4 Max inside a Mac Studio. Intel took its own swing at the tokens-per-watt question this month with its Crescent Island accelerator, though Intel never published a comparable number of its own, leaving this paper as the more complete public accounting so far. A smaller on-device model, OpenBMB's 2.5-billion-parameter MiniCPM5, reflects the same push toward keeping inference off the cloud entirely.

Efficiency rose 5.3 times since 2023, split between software and silicon

Table 2 in the paper tracks three representative pairings. In 2023, Mixtral-8x7B running on a Nvidia Quadro RTX 6000 scored 7.92×10⁻⁴ in accuracy per watt. By 2024, Llama-3.1-8B on an RTX 6000 Ada reached 1.80×10⁻³. By 2025, GPT-OSS-120B on the Apple M4 Max hit 4.18×10⁻³, a 5.3-times improvement over two years. Emergent Mind's summary attributes 3.1 times of that gain to model improvements and 1.7 times to accelerator hardware, meaning software carried most of the load.

Query coverage grew alongside it. The share of real traffic a local model could answer without falling back to the cloud rose from 23.2% in 2023 to 71.3% in 2025, according to the paper's longitudinal analysis.

Year	Model	Accelerator	Accuracy per watt
2023	Mixtral-8x7B	Nvidia Quadro RTX 6000	7.92×10⁻⁴
2024	Llama-3.1-8B	Nvidia RTX 6000 Ada	1.80×10⁻³
2025	GPT-OSS-120B	Apple M4 Max	4.18×10⁻³

1. 202323.2 %
2. 202571.3 %

Source: Saad-Falcon et al., "Intelligence per Watt: Measuring Intelligence Efficiency of Local AI," arXiv:2511.07885, accessed 2026-09-14

Cloud silicon still wins on raw efficiency

The paper's least comfortable finding for anyone pitching local inference as a wholesale cloud replacement: local accelerators return at least 1.4 times lower IPW than cloud accelerators running the identical model. Nvidia's B200 posted 1.40 times higher IPW than the Apple M4 Max on matched workloads, and SambaNova's SN40L reached 1.78 times higher. The gap traces to purpose-built server components and unified-memory tradeoffs the paper says local hardware has not caught up to.

That is the number that matters more than 88.7%. Local inference is viable for most queries not because it beats the cloud on efficiency, but because it is efficient enough while carrying no marginal cost and no network round trip.

The 1 million queries the paper draws on come from WildChat, a public corpus of real chatbot conversations, plus a set of reasoning benchmarks, according to Emergent Mind's summary of the methodology. That matters for how far the 88.7% figure travels: a corpus of everyday chat and homework-style reasoning questions is not the same traffic mix as a coding agent chained through a dozen tool calls, or a customer-support system pulling from a private knowledge base. The paper's authors do not claim otherwise; the 88.7% figure describes the traffic they measured, not every workload a company might consider moving off the cloud.

What would change the read

The authors measured power with vendor-specific tools: NVML for Nvidia parts, ROCm SMI for the AMD Instinct MI300X, and Apple's powermetrics for the M4 Max, sampled every 50 milliseconds. They flag their own limitation directly: software-based power readings can carry a 10% to 15% error against hardware measurement, wide enough to move some of the closer accelerator comparisons. The study also covers single-turn interactions only, with multi-turn workloads relegated to an appendix rather than the headline results, and reasoning accuracy slowed on the hardest problems even as chat performance held steady across difficulty levels. If a follow-up study finds that reasoning gap widening rather than closing, the case for routing most queries to local hardware gets weaker, not stronger.