Amazon's TorchNeuron Is Beta Code, and Its Deepgram Result Spans 11% to 70%
Hardware / news
Amazon's TorchNeuron Is Beta Code, and Its Deepgram Result Spans 11% to 70%
AWS says a one-line device change moves PyTorch code onto Trainium2 and Trainium3, but TechCrunch and AWS's own case study give different ranges for the headline customer gain.
Amazon Web Services released TorchNeuron on Monday, a PyTorch backend that it says runs existing PyTorch code on its Trainium2 and Trainium3 chips after a single configuration change. The code is Apache-2.0 licensed and sits in a public repository marked beta.
The announcement is aimed at CUDA, Nvidia's programming layer, which is the reason most PyTorch code is written for Nvidia GPUs first. Money is still flowing to challengers: Tenstorrent raised $1.5 billion at a $6.2 billion valuation. TechCrunch's report by Julie Bort quotes Colin Brace, vice president of technology software at AWS's Annapurna Labs chip unit, on the goal of making PyTorch more hardware-agnostic for everyone, not only Trainium users.
What the repository says it is
The aws-neuron/torch-neuronx repository describes a native backend that adds a neuron device through PyTorch's PrivateUse1 mechanism, the hook PyTorch provides for third-party hardware. It lists support for eager execution, torch.compile, distributed training through DTensor and FSDP, tensor parallelism and custom NKI kernels, Amazon's own kernel language.
The current release is version 2.14.3.0.x, targeting PyTorch 2.14 and Neuron SDK 2.33, and it needs Python 3.12, 3.13 or 3.14 on Trn2 or Trn3 instances. The page labels it beta. It shows 52 stars, no forks and a single commit in its history, so the "hundreds of thousands of AI models" that Amazon's announcement says it tests on every build are not visible in the public history.
Earlier torch-neuronx releases are a different product. The repository calls them the separate XLA-based product, so version numbers alone do not tell a reader which one is installed.
Deepgram's figure depends on which page you read
The customer number is where the sources part. TechCrunch reports that Deepgram, a speech-to-text company, processes 70% more audio per dollar on Trainium, attributing the figure to Brace.
The AWS case study on Deepgram, published the same day, gives a range: 11 to 70 percent more audio per dollar than the compute Deepgram ran before. It does not name the GPU or instance used as the baseline, and it prices both sides at instance rates on the day of the evaluation.
| Source | Deepgram claim | Baseline named |
|---|---|---|
| TechCrunch, Oct. 19 | 70% more audio per dollar | No |
| AWS ML blog, Oct. 19 | 11 to 70 percent more audio per dollar | No |
- Low end of range11 %
- High end of range70 %
Source: AWS Machine Learning Blog, Oct. 19, 2026, accessed 2026-10-19
The workload matters as much as the range. The post says the model was Nova-3 speech-to-text on offline batch jobs such as contact-center analytics and medical transcription, run as eight replicas on one Trainium2 device in a single trn2.3xlarge instance. Batch work tolerates latency that live voice agents do not, so the figure says little about real-time serving.
The post gives no word error rate. It says only that accuracy had to stay in line with Deepgram's reference values, and that a Rust runtime reproduced the reference token sequences exactly at every batch size tested. Both numbers are vendor-supplied.
What the demo and the adoption claims leave out
Amazon says its engineers ran the same training script on a conventional chip and on Trainium, changing only the device setting, and that the model learned the same things at the same rate. The post gives that as a sentence. It carries no loss curve, no step time and no cost per step.
Amazon names Cartesia and the Lieber Institute for Brain Development as early users, and says Hugging Face has added native Trainium support. Vendor tables reward a close read of their own columns, as KiviDB's benchmark did when its own table put a rival ahead. TechCrunch reports 85% as the share of AI apps written in PyTorch; Amazon's own page words it as 85% of AI researchers and developers. Those are different denominators.
Neither piece gives pricing or regional availability for Trainium3 instances. The open-source release itself is scheduled for the PyTorch Conference North America on Oct. 20, although the repository is already public and marked beta; that is the date to watch for the first independent step-time numbers.
What would change this read is a third party posting tokens per second per dollar for the same model on Trainium3 and an Nvidia part at one batch size and context length. Until then, the only cost comparison is the one Amazon chose.
Sources
More in Hardware
- 01Ukraine's Drone Evacuation Flew 62 km in 74 Minutes; the Hexacopter Is Rated for 25 kmA modified Ayudag-6 carried a wounded 30th Brigade soldier out of a mined position, and nothing published explains the gap between flight distance and rated range.
- 02Linux Sees 116 GiB of CUDA Memory on an RTX Spark Laptop, 6.3 GiB More Than WindowsA single-author port of Arch Linux ARM to the ASUS ProArt P16 runs a local model at 30.9 tokens a second, on a machine ASUS sells with Windows only.
- 03Mind Robotics Hires Doug Field After $1.015 Billion in Three Rounds and No Public RobotRivian's factory-robot spinoff has named the former Ford EV chief as president while disclosing no unit count, headcount, valuation or hardware design.
- 04Robots Have 4,000 Public Hours of Training Data and Need an Estimated 1 Billion, at $10 to $100 EachConstruction Physics sizes the robot data gap against text. The estimate rests on collection prices that range from $10 to $150 an hour inside the same company.