Google's TPU v5: the custom chips powering Gemini

2025

Google has been designing its own AI accelerator chips, Tensor Processing Units (TPUs), since 2016. By TPU v5, these custom chips gave Google a significant advantage in training and serving Gemini.

TPUs are specifically designed for the matrix multiplication operations that dominate AI workloads. They sacrifice general-purpose computing flexibility for raw throughput on exactly the operations AI needs.

The advantage is twofold: cost and access. While competitors scramble to buy NVIDIA H100 GPUs (expensive and supply-constrained), Google manufactures its own chips. They're not subject to supply chain bottlenecks or NVIDIA's pricing.

TPU v5 pods (clusters of thousands of chips) can train models in weeks that would take months on equivalent GPU clusters. This speed advantage means Google can iterate faster: try more architectures, run more experiments, and get to better models sooner.

The chips also optimize inference: serving the trained model to users. Lower serving cost per query means Google can offer more generous free tiers, deploy to more products, and experiment with applications where margins would be too thin with rented GPU compute.

Jeff Dean, Google's chief scientist, called it "the single biggest strategic advantage in AI infrastructure."