Lawson's 12-Framework Test Puts the AI Cost of One Pokémon App Between $2.67 and $4.97
Software / analysis
Lawson's 12-Framework Test Puts the AI Cost of One Pokémon App Between $2.67 and $4.97
Nolan Lawson had Claude Opus 5.5 build the same app in each stack. Angular cost 1.86 times as much as Hono+htmx, and plain Vite beat eight frameworks on price.
Nolan Lawson asked a coding agent to build the same Pokémon app in 12 JavaScript stacks and priced each run. The cheapest median was $2.67 for Hono+htmx; the dearest was $4.97 for Angular, a 1.86x spread on a task with identical requirements.
Lawson published the results on Oct. 28, 2026 in a post on his blog. The costs are API-equivalent prices reported by Claude Code, he wrote, not money actually spent.
What Lawson measured across 12 stacks
Each agent ran Claude Opus 5.5 at high effort and built an app against PokéAPI. Output was graded with an accessibility-tree conformance suite. Lawson banned Tailwind and component libraries, and gave agents each framework's official MCP server or skill where one existed.
Every framework got the median of 5 runs, except Angular, which got 3 because 2 runs produced apps that did not work. Tests ran in Chromium on a Framework 13 laptop under Ubuntu 24.04, with 4x CPU throttling and Slow 4G.
- Hono+htmx2.67 USD
- Astro2.76 USD
- Nuxt2.82 USD
- Vanilla (Vite)2.98 USD
- Next.js3.14 USD
- Qwik3.27 USD
- TanStack Start3.54 USD
- SolidStart3.79 USD
- React Router3.81 USD
- SvelteKit4.28 USD
- Gracile+Lit4.65 USD
- Angular4.97 USD
Source: Nolan Lawson, The AI cost of JavaScript frameworks, 2026-10-28
Where the dollars went
Time and tokens track cost closely. Angular took 17.2 minutes and 9.76 million tokens against 8.6 minutes and 4.26 million for Hono+htmx, and its 2,009 lines of code were the most of any stack. Lawson says Angular's cost followed its code size.
SvelteKit is the odder case. It cost $4.28, and Lawson says its agents read the framework's v3 migration guide on every run. That guide is the sticking point: a document written for humans upgrading from v2 was pulled into an agent's context 5 times out of 5.
| Framework | Minutes | Tokens (M) | JS shipped (KiB) |
|---|---|---|---|
| Astro | 8.2 | 4.13 | 0 |
| Next.js | 10.9 | 4.99 | 142 |
| SvelteKit | 10.8 | 8.53 | 36 |
| Angular | 17.2 | 9.76 | 93 |
Vanilla Vite lost to three frameworks
Plain Vite with no framework cost $2.98 and 5.00 million tokens. Three stacks were cheaper (Hono+htmx, Astro and Nuxt) and eight were dearer. Lawson reads that as evidence that some framework beats none, since the agent without one spent more than agents using Astro or Nuxt.
The multi-page stacks led on page weight. Astro shipped 0 KiB of JavaScript and a 810 ms largest contentful paint; Next.js shipped 142 KiB and 2,090 ms. Lawson says Lighthouse and Core Web Vitals favour multi-page apps and that he measured only the home page.
The agents mostly ignored the official tooling
Lawson found agents used framework tooling inconsistently. Nuxt's MCP server was never used. Next.js agents read its skill file in 1 of 5 runs. He writes that he suspects current MCPs and skills may do more harm than good, a suspicion the benchmark was not designed to test.
Agent spend is a live question elsewhere: Cisco's unlimited-token deal and an SRE deskilling study are both about what coding agents cost and change.
Quality barely moved with cost
Lighthouse did not separate the stacks the way the bill did. Performance and Accessibility scores were close to saturated, with most frameworks at 100 out of 100. Angular scored 98 on Performance and Next.js 99, while Vanilla scored 97 and was the only stack with a non-zero layout shift, 0.111.
Cost and page speed also came apart. Gracile+Lit was the second most expensive build at $4.65 yet shipped 8 KiB of JavaScript and a 956 ms largest contentful paint. Qwik, an SPA with client-side routing, posted 979 ms, close to the multi-page stacks, at $3.27.
Angular's problems were of another kind. In 2 runs its agents built server-rendered apps that rejected requests from anything but localhost, which broke the spec.
One more benchmark, with a conflict of interest
Encore published a backend benchmark on May 26, 2026, written by Ivan Cernja, using claude-sonnet-4-6 on Encore, Express, Fastify, Hono and NestJS. Across three runs it put Encore at $6.29 and NestJS at $12.69.
It is not comparable with Lawson's: different model, different task, different price basis. And Encore is one of the five frameworks it ranks. It shows only that a second setup also found framework choice moving agent spend, and the party that ran it has a product in the table.
What the numbers cannot say
Lawson lists his own limits: results are stochastic, the app is one small content-heavy page set, agents were not told to optimise performance, and pre-generated static HTML was prohibited. A 3 to 5 run median on one app cannot rank frameworks for a complex product.
The spread is about $2.30 per build between top and bottom. Lawson pins versions in an addendum, including Next.js 16.4.3, SvelteKit 3.1.0 and Vite 8.3.5, and invites others to fork the benchmark on GitHub and rerun it.
Sources
More in Software
- 01OmniDiff, a Rust Diff Tool, Reports 60 Percent Exact Matches Against a 90 Percent GoalThe README publishes its own scorecard on 1,419 hand-labelled fixtures. It also says VS Code cannot yet use it as the built-in diff view.
- 02Enlightenment 0.28.0 Adds NetworkManager and PipeWire Support 658 Days After 0.27The window manager's first release since January 2025 needs EFL 1.29.0 and adds NetBSD support, a clipboard history and battery charge limits.
- 03Python 3.15 Ships a Built-In sentinel After Five Years of PEP 661The new type replaces the object() idiom with a one-line constructor that survives copying, pickling and type checking, but a Python 3.15 floor is the price of using it.
- 04Lightpanda 1.0 Passes 80% of Chrome's Web Tests, by Its Own CountThe Zig-built headless browser left beta on Oct. 2, and its own benchmarks put it ahead of two Rust rivals on a 940-page crawl, with no hardware listed.