Google showed systems with its own TPUs on which Gemini and Apple Intelligence AI models are trained

A large lab at Google’s headquarters in Mountain View, California, houses hundreds of racks of servers, running tasks other than Google Cloud’s search engine or workloads. Tensor Processing Units (TPUs) developed by Google itself are tested here. CNBC journalists managed to look inside this laboratory.

Image source: alban/unsplash.com

Google’s TPUs appeared in 2015 and became available to cloud clients in 2018 – they are used, in particular, to train Apple Intelligence and Google Gemini models. Google was the first cloud provider to build its own AI chips—Amazon didn’t announce its Inferentia until three years later, and Microsoft didn’t unveil Maia until late 2023. But this primacy did not help Google become a leader in the generative AI race: Gemini came out more than a year after OpenAI ChatGPT. At the same time, offerings in the field of AI helped Google Cloud gain momentum: this segment of the company showed growth of 29%, and quarterly revenue exceeded $10 billion for the first time, according to Alphabet’s latest quarterly report.

Google came up with the idea of ​​​​creating its own chip when in 2014 they thought about what resources they should have so that all users of the company’s services could use the voice interface for at least 30 seconds a day. As it turned out, the number of computers in data centers needed to be doubled. Google TPU has helped improve efficiency in some tasks by up to 100 times. The company still uses both traditional CPUs and Nvidia GPUs. But Google TPU is a special purpose integrated circuit (ASIC) designed only for a specific type of task. The company has another such chip at its disposal – Video Coding Unit, and it is used for video processing.

Google, following the example of Apple, began to use its own chips in devices of its own design: in the Pixel 9 smartphones it is the fourth generation Tensor G4 processor, and in the Pixel Buds Pro 2 headphones it is the Tensor A1. But Google’s big differentiator is its server TPU, which now has a 58% market share among its own cloud AI accelerators. Nvidia’s GPUs are more flexible, but also more expensive and in short supply in today’s environment, when the AI ​​boom has sent the company’s shares soaring that it now competes with Apple and Microsoft to be the world’s most valuable public company. The real test of Google’s TPU comes when the Apple Intelligence platform goes live on iPhone and Mac next year.

Developing an alternative to Nvidia accelerators is almost a feat. The process is so complicated and expensive that even Google can’t do it alone. Since the introduction of the first TPU, the company has enlisted the support of chipmaker Broadcom, which helps Meta✴ in solving the same problem. The fully developed chip design is sent to a semiconductor contractor – TSMC, which produces 92% of the world’s advanced semiconductor products.

Trillium. Image Source: CNBC/Marc Ganley

This year Google will release the sixth generation of Trillium TPU; In addition, last April the company announced Axion, its first central processor, which will appear at the end of the year. And Google is not the first here: Amazon released its Graviton in 2018, China’s Alibaba followed suit in 2021, and Microsoft introduced the Cobalt 100 chip last November. All are based on the Arm architecture, which is more flexible and power-efficient than the x86 that Intel and AMD are committed to.

Axion CPU. Image Source: CNBC/Marc Ganley

Efficiency is key: by 2027, AI servers will consume as much energy per year as Argentina, according to projections. Google’s latest environmental report showed that the company’s emissions from 2019 to 2023. grew by 50%, partly due to the growth in the number of AI data centers. Cooling servers for training and running AI requires huge volumes of water, so from the third generation of Google TPU cooling is done directly on the chip – Nvidia has resorted to the same scheme with the latest Blackwell. And despite the attendant difficulties, Google’s AI equipment continues to be in high demand, and the company has not yet noticed its weakening.