Google Cloud now offers encrypted AI processing on NVIDIA's newest chips

Google Cloud has made a privacy feature generally available for AI work on its G4 machines, which use NVIDIA's RTX PRO 6000 Blackwell GPUs. The feature keeps data and AI models encrypted while they are being used, not just while stored or sent. Google says it is live in 36 zones, so it is not a test.

What changed

Confidential Computing on G4 was in preview earlier this year. It is now open to all customers. It works on Compute Engine virtual machines and on GKE nodes, which are the machines behind Google's Kubernetes container service.

Google says the hardware encrypts memory and keeps the encryption keys in a form that cannot be exported. It also supports remote attestation. That lets a customer check the code and setup of a machine before sending it sensitive data.

What it protects

Google says the setup shields prompts, outputs and proprietary model files from several parties:

• Host system administrators
• The software layer that runs virtual machines, called a hypervisor
• The cloud provider itself

Google also says no code changes are needed. Customers can use standard Ubuntu images, NVIDIA drivers and NVIDIA CUDA-X libraries. Polaris Cloud, a customer quoted by Google, said it needed one flag at launch and little else.

What it costs in speed

Google tested the g4-standard-48 machine type against the same machine without the protection. It used Llama 3, Llama 3.3, Resnet-50 and SDXL. Throughput dropped by less than 3.5%, measured in tokens or images per second. Google notes that results vary by setup.

One customer quoted by Google measured a 5.2% rise in end-to-end latency, meaning the total wait for a response. That customer said it would make that trade every time. The material does not give pricing, so cost is still an open question.

What happens next

Support for Confidential Space is due to reach general availability in the coming weeks. It lets several companies combine sensitive data for analytics or model training without exposing the underlying data to each other.

What to do

Businesses that feed client, patient or financial data into cloud AI tools can ask their AI vendors whether they use hardware-protected processing like this. The protection only helps if the vendor turns it on, and Google's own numbers come from its own tests.