Google and Amazon make it easier to move AI apps off Nvidia chips

Google and Amazon have each released a free tool that lets developers move AI apps off Nvidia chips without rewriting them. Google's tool, TorchTPU, arrived Tuesday. Amazon's, TorchNeuron, came a day earlier. Both go after Nvidia's long hold on how AI software is built.

What changed

PyTorch is a free, open source toolkit that developers use to build, train and run AI models. By some accounts, about 85% of AI apps use it. Meta started it in 2016, when Nvidia's chips, called GPUs, were already the standard for AI research. Many PyTorch features were built to work with them. In 2022, Meta gave PyTorch to the Linux Foundation, a nonprofit.

Google's TorchTPU runs AI apps on Google's own AI chips, called Tensor Processing Units, or TPUs. Developers can move an existing app with a simple configuration change and no rewriting. That also moves the app to Google Cloud.

Amazon's TorchNeuron does the same for its Trainium chip and AWS, again with no rewriting.

What each company added

• Amazon is submitting code to the main PyTorch project so more features work with any hardware by default.
• Google says it worked with Meta, Hugging Face, Inferact and RadixArk so popular AI tools work with TorchTPU.
• Google also launched a TPU agent suite to help developers move apps and fix errors along the way.
• Google updated LiteRT, which runs AI on phones, laptops and embedded devices, to work better with PyTorch apps.

Why it matters

Most AI apps are written by default to run on Nvidia chips. That makes it hard and costly to switch. If moving gets easier, developers gain more choice. Over time, that could mean lower cloud costs and less lock-in, which is when a company is stuck with one supplier. Businesses that rely on AI products could feel the effect indirectly.

What is not known yet

Nobody is expected to leave Nvidia overnight. Being able to move easily does not mean developers will. The material does not say how the new tools compare with Nvidia on speed or price, or how many developers will try them. Amazon and Google are clearly working to weaken Nvidia's advantage, but the results are still to come.