Google Open-Sources TorchTPU to Challenge Nvidia's AI Dominance
Google has open-sourced TorchTPU, a native PyTorch framework enabling seamless migration of AI applications to TPUs without code rewrites. By partnering with ecosystem leaders like Meta and Hugging Face, Google aims to challenge Nvidia's dominance in the AI hardware and software stack.
Full Report
On October 21, Google announced on Tuesday that TorchTPU, a native PyTorch framework for Google TPUs, is now open-sourced on GitHub. With this move, Google joins Amazon in an attempt to break Nvidia's monopoly in the AI industry. Currently, the vast majority of AI applications are developed and run by default on Nvidia chips, specifically Graphics Processing Units (GPUs).
PyTorch is a popular open-source library relied upon by AI application developers and startups to build, train, and deploy AI models. Statistics suggest that 85% of AI applications use PyTorch. Originally developed by Meta in 2016—a time when few realized that Large Language Models (LLMs) would change the world—PyTorch was built with many features specifically tailored to adapt to Nvidia GPUs, which were already the standard hardware for AI research at the time. In 2022, Meta transferred PyTorch to the Linux Foundation, a non-profit organization responsible for managing many critical open-source projects.
Google's TorchTPU enables AI applications to run on Google's proprietary Tensor Processing Units (TPUs). It also facilitates the easy migration of existing PyTorch applications to TPUs and subsequently to Google Cloud. Developers can complete the migration simply by modifying configurations, without needing to rewrite application code.
Amazon released similar news on Monday, launching its own version of PyTorch called TorchNeuron. TorchNeuron allows AI applications to be ported to Amazon's proprietary Trainium chips and Amazon Web Services (AWS) without requiring code rewrites. Amazon further stated that it aims to reduce Nvidia's control over PyTorch. The company is submitting code to the main PyTorch project to decouple various features from Nvidia chips, enabling PyTorch to natively support diverse hardware.
However, IT Home noted that Google's efforts may go further than Amazon's in certain aspects. Google stated that it has directly collaborated with Meta, Hugging Face, Inferact (the developer of vLLM), RadixArk (the developer of SGLang), and other parties to ensure that TorchTPU's functionality and codebase maintain compatibility with these mainstream AI development tools, just like the original PyTorch.
In addition, the search giant launched a dedicated tool suite, the TPU Agent Suite, to help developers migrate applications to Google's chips and cloud platform, as well as troubleshoot and fix errors during the migration process. Google also upgraded LiteRT, a widely used tool for running AI models on smartphones, laptops, and embedded devices. The upgrade provides better support for AI applications developed using PyTorch.
These developments do not imply that AI application developers will collectively abandon Nvidia hardware simply because migrating to other chips and cloud platforms has become convenient. Nevertheless, both Amazon and Google are clearly striving to end Nvidia's dominant position.