Tech
Amazon opens TorchNeuron, a native PyTorch backend for Trainium, ahead of a Tuesday Apache 2.0 launch at PyTorch Conference.
PyTorch Blog · 2026-10-19 · Source ↗
Amazon opens TorchNeuron, a native PyTorch backend for Trainium, ahead of a Tuesday Apache 2.0 launch at PyTorch Conference.
PyTorch Blog · 2026-10-19 · Source ↗
Amazon's AWS Neuron team and Meta's PyTorch group published the TorchNeuron introduction on Monday, a day before the Tuesday keynote at PyTorch Conference North America in San Francisco. TorchNeuron registers Trainium through PrivateUse1, the official PyTorch extension point for custom hardware, so standard APIs (torch.nn, torch.tensor, torch.autocast, torch.distributed, torch.profiler) all run on a device named "neuron" without framework changes. The library will be released as open source under Apache 2.0 at github.com/aws-neuron/torch-neuronx and go live as a public beta on October 20. Trainium2 and Trainium3 are supported from day one.
How it works
The backend supports eager execution, torch.compile for production, and the Neuron Kernel Interface (NKI) for custom kernels. An adaptive eager mode (experimental) narrows the gap between eager and compiled runs by fusing operations without a code change. TorchDynamo captures the model into an FX graph, AOTAutograd traces the backward graph ahead of time, the Neuron backend lowers through Torch-MLIR into StableHLO, and the Neuron compiler produces a NEFF file that executes on hardware. FlexAttention is supported natively. The two inference stacks that ship on top are vLLM Neuron and vLLM Omni Neuron.
"PyTorch's vision is straightforward: developers should have a consistent experience across hardware platforms. Native support for AWS Trainium expands that choice, enabling researchers and engineers to experiment and iterate using the eager execution and distributed APIs they already know." — Jana van Greunen, Director of PyTorch Engineering at Meta
Partner numbers
Deepgram's CTO Adam Sypniewski reported that after a multi-month engagement with the AWS Generative AI Innovation Center to port Deepgram's model to PyTorch native on Trainium, "Trainium processes about 60% more input audio seconds per dollar than our existing GPU baselines." Cartesia co-founder Arjun Desai said the stack let the Sonic text-to-speech and Ink speech-to-text models run with minimal PyTorch-code changes, which freed the team to hand-write high-performance NKI kernels only for the layers that drive end-to-end latency. Hugging Face's VP Product Jeff Boudier framed TorchNeuron as making Trainium a native backend for Transformers and Diffusers so the familiar library stack runs with no device-specific adaptation; TRL enables fine-tuning and reinforcement learning, and Neuron kernels deliver optimization automatically.
Distributed and testing
DDP, FSDP2, SimpleFSDP, tensor parallelism, pipeline parallelism, and compositions like FSDP2 plus tensor parallelism and ZeRO all run natively. Collectives execute over Trainium's dedicated collective cores intra-node and over the Elastic Fabric Adapter inter-node. The team ran a blind sweep of 100,000 Hugging Face models covering 99.9 percent of real-world usage by download count on both eager and torch.compile. The August 2026 upstream test suite run produced about 154,000 outcomes, of which roughly 69,000 were skipped with a valid reason; the remaining 39,000 Neuron-specific skips break into 22,000 operators without a native Neuron lowering yet, a block of CUDA-specific tests that upstream eligibility checks did not flag, and features TorchNeuron chose not to support in the first release.
"Voice is becoming a first-class modality for AI, and Deepgram processes quadrillions of audio tokens a year for more than 200,000 developers and 1,400 organizations." — Adam Sypniewski, CTO, Deepgram
Ecosystem partners
Design partners named at launch include Cartesia, Hugging Face, Anthropic, Databricks, Rivian, Salesforce, and poolside. Ecosystem partners building with TorchNeuron include Anyscale, Decart, Kernelize, Loka, Luminal, Makora, Modular, Nota AI, Radix, Wafer, and Yasp. Trainium-tuned recipes live in a torchtitan-neuron fork that upstreams into mainline TorchTitan as changes mature. The pip install is torch-neuronx from the AWS Neuron pip repository.
The byline lists contributors Apoorv Gupta, Arash Jamalian, Rui Silva, Peter Sam Raj, Kunind Sahu, Kamran Khan, Shreya Srivastava, Charan Shettyhalli Guruswamy, Alex Hoffman, Abigail Fernandes, and Sam Jin. Public resources go live October 20 at github.com/aws-neuron/torch-neuronx.