Google services such as Google Search (RankBrain), Street View, Google Photos, and Google Translate have one thing in common: they all use Google’s Tensor Processing Unit, or TPU, for their computations. 

You might be thinking what is a TPU and what is so great about these services? All these services use state-of-the-art machine learning algorithms in the background, and these algorithms involve large computations. TPUs help to accelerate the neural network computations involved. Even AlphaGo, the deep learning program that defeated Lee Sedol in the game of Go, was powered by TPUs. So let us see what exactly a TPU is.

A TPU is a custom application-specific integrated circuit (ASIC) built by Google specifically for machine learning and is tailored for Tensorflow. It is built on a 28-nm process, it runs at 700 MHz, and consumes 40 W of energy when running. It is packaged as an external accelerator card that can fit into the existing SATA hard disk slots. A TPU is connected to the host CPUs via a PCIe Gen 3×16 bus, which provides an effective bandwidth of 12.5 GB/s.

The first generation of TPUs is, as of now, targeting inference, that is, use of an already trained model. The training of DNNs, which usually takes more time, was still done on CPUs and GPUs. The second-generation TPUs announced in a May 2017 blog post(https://www.blog.google/topics/google-cloud/google-cloud-offer-tpus-machine-learning/) can both train and infer machine learning models.