Agent-Built FPGA Runs a Ternary LLM at 213 Tokens a Second, Beating Rented CPUs per Dollar but Not a T4
Hardware / news
Agent-Built FPGA Runs a Ternary LLM at 213 Tokens a Second, Beating Rented CPUs per Dollar but Not a T4
Ankit Gupta spent about $10,000 in model tokens on an AWS F2 design; his own numbers fall to 76 tokens a second at 8,000-token context.
A developer used coding agents to build an LLM inference engine on an AWS FPGA that reaches 213.13 tokens a second on a ternary-weight model, up from 6.43 tokens a second in its first bit-exact version. The author, Ankit Gupta, spent about $10,000 in Claude and OpenAI tokens to get there and says he does not recommend the setup for production.
Gupta posted the account on Nov. 20 as "Vibecoding an inference chip". It is a single-author write-up with no linked repository, so every figure below is his own.
What he built, and on what
The target is an AWS F2 instance, specifically the f2.6xlarge at $1.98 an hour on demand, according to Gupta. AWS's F2 page lists the part as an AMD Virtex UltraScale+ HBM VU47P with 16 GiB of high-bandwidth memory, and the f2.6xlarge as one FPGA with 24 vCPUs and 256 GiB of host memory. Gupta's design uses 8 compute cores in two clusters across three dies and 32 HBM channels.
The model is a ternary-weight model from the DeepGrove team with an 8,192-token context window. The post gives no parameter count. The code is mostly C++ with some Verilog, written by agents over several days, with hundreds of experiments, about 3 million lines of code written and 30 million lines of output parsed.
The final build runs at a 225 MHz kernel clock and 425 MHz memory clock. A soak test found random faults at 231.25 MHz, so he stepped down. Two hangs showed up only on hardware, and a larger integration lost about 38 hours to them.
Throughput against rented alternatives
Gupta compared the FPGA with three other setups on the same 8-chat, 3-turn benchmark. The post does not give a batch size beyond that. The last column is The Terminal's arithmetic: tokens per second times 3,600, divided by the hourly instance price.
| Setup | $/hour | Tokens/s | Tokens per dollar |
|---|---|---|---|
| FPGA (f2.6xlarge) | 1.98 | 213.13 | about 387,500 |
| CPU, 32 threads (m8a.8xlarge) | 1.95 | 129.5 | about 239,100 |
| CPU, 24 threads (c7i.12xlarge) | 2.14 | 99.6 | about 167,600 |
| Nvidia T4 community CUDA port (g4dn.xlarge) | 0.53 | 166.3 | about 1,129,600 |
- FPGA, f2.6xlarge213.13 tokens/s
- T4 CUDA port, g4dn.xlarge166.3 tokens/s
- CPU, 32 threads129.5 tokens/s
- CPU, 24 threads99.6 tokens/s
Source: Ankit Gupta, 'Vibecoding an inference chip', Nov. 20, 2026 (author's own measurements)
The FPGA wins on speed and beats both CPU runs per dollar. The T4 comes out about 2.9 times cheaper per token than the FPGA, but Gupta reports 0 of 24 turns bit-exact on it, while the CPU twin and the FPGA match a pinned reference exactly. His exactness rule also ruled out numerics shortcuts that other designs use, so the comparison is deliberately strict.
Where the number falls apart
The headline throughput is for short prompts. Gupta reports about 233 tokens a second at 16 to 64 tokens and about 76 tokens a second at 6,000 to 8,000 tokens of context. At the context length a document workload sends, the FPGA delivers roughly a third of its headline figure. The post does not say which context length the 213.13 benchmark used, so the CPU and T4 rows may not hold at 8,000 tokens either.
Power, die area and cost per token on the FPGA itself are not reported, and the post is explicit that it is not a production recommendation. It also shows what agent-written hardware code costs: $10,000 in model spend for a design that a $0.53-an-hour T4 beats on price.
Chip design built by agents is a live question, as Frostbite, an AI-designed CPU, showed with its unnamed baseline, and small-footprint models are the other half of the equation, as in Nemotron 3.5 Bolt's 4 GB name and 6.18 GB file.
What would change this read is a published power figure: at the F2's hourly price the dollars are known, but whether a 225 MHz FPGA design wins on watts per token is the number Gupta did not measure.
Sources
More in Hardware
- 01Medtronic's Hugo Wins FDA Clearance for General Surgery, With No Install Count or Trial FiguresThe Nov. 17 clearance covers about 3.5 million U.S. operations a year, but Medtronic's release names no price, no systems installed and no study results.
- 02Agility's Digit 5 Weighs Over 120 kg and Runs 90 Minutes a Charge, but No Source Gives a Digit CountThe first public pure-play U.S. humanoid maker reports $300 million in preorders, while its co-founder's own specs imply about 18 hours of motion in a 20-hour shift.
- 03Rockwell Buys AGILOX, Which Has 2,500 Robots in the Field, at an Undisclosed PriceRockwell says the deal more than doubles its AMR business and closes in fiscal Q2 2027; the last comparable purchase, Clearpath, was valued at $609 million.
- 04California Bars Waymo From Freeways After Three Cars Entered a US-101 Closure, Months After a 3,900-Car RecallThe DMV order took effect November 11. Neither report says whether the cars on 101 had the software Waymo recalled in June.