Amazon is taking its biggest shot at Nvidia yet with new TorchNeuron dev tool | TechCrunch
On Monday, Amazon Web Services (AWS) unveiled a new development tool named TorchNeuron, designed to simplify the process for artificial intelligence (AI) startups looking to transition their AI applications to Amazon's AI chip, Trainium. This move represents Amazon's most significant challenge yet to Nvidia, a company that has leveraged its software advantages to become the leading hardware provider for AI applications.
TorchNeuron is compatible with PyTorch, an open-source framework widely used by developers to create, train, and deploy AI models. According to Amazon, the tool enables AI applications built with PyTorch to operate on Trainium with minimal effort, requiring only a single line of configuration change rather than a complete code rewrite.
Around 85% of AI applications are developed using PyTorch, which was initially created by Facebook and has been part of the Linux Foundation since 2022. Over the years, Nvidia has invested heavily in ensuring that its CUDA software—an essential infrastructure for running applications on Nvidia GPUs—supports PyTorch seamlessly. This has resulted in Nvidia's hardware becoming effectively synonymous with the PyTorch toolkit.
Although Amazon is not the first to create a tool that facilitates the transition of applications to its chips, TorchNeuron is tailored for developers already familiar with PyTorch. Other companies, such as AMD and Apple, also allow PyTorch users to migrate their applications to their respective hardware with similar configuration adjustments.
However, Amazon contends that moving to Trainium not only simplifies the process but also provides startups with access to a comprehensive cloud environment. AWS argues that its platform is superior to some smaller cloud providers that are closely linked to Nvidia's hardware, such as CoreWeave, Nebius, and Firmus. Colin Brace, Vice President of Technology Software at AWS Annapurna Labs, stated in an email to TechCrunch, "You’re building on AWS. You’re not betting on a single-purpose compute provider and scrambling to build everything else around it." He added that AWS offers a robust infrastructure that enables startups to scale efficiently from prototype to production.
Amazon claims that the dual functionality of Trainium—capable of handling both training and inference tasks—combined with its tight integration with AWS's cloud services, leads to cost savings for startups, even if AWS's pricing per token isn't the lowest available. Brace noted, "Trainium’s value is price-performance plus the fact that it’s one platform for both training and inference, which improves the economics across a full workload." He cited the example of Deepgram, an AI voice application provider, which reported a 70% increase in audio processing efficiency per dollar after switching to Trainium using native PyTorch support.
Beyond merely facilitating the migration of PyTorch applications to its cloud, Amazon is also working to reduce PyTorch's inherent reliance on Nvidia. The company plans to contribute code changes to the PyTorch project, collaborating with the community to eliminate dependencies that tie the framework to specific hardware. In a blog post announcing TorchNeuron, Amazon emphasized its commitment to making PyTorch more hardware-agnostic for all developers, not just those using Trainium.
While this strategy may benefit companies competing with Nvidia in the hardware and cloud markets, it raises questions about whether it will genuinely offer developers greater flexibility or merely introduce new configuration complexities. The outcome remains to be seen, but if Amazon is successful in reducing PyTorch's association with Nvidia, the implications for the GPU leader could be significant. As Brace remarked, "The real story is removing the lock-in that kept developers from having a real choice at all."