Tensor Processing Unit
Subject: Computer and Technology noun /ˈtɛn.sər ˈprɑː.sɛs.ɪŋ ˈjuː.nɪt/ 1 definition
- artificial-intelligence
- machine-learning
- computer-architecture
- computer-hardware
-
Definition (noun)
Entry #218 · UpdatedA Tensor Processing Unit (TPU) is an application-specific integrated circuit developed by Google to accelerate neural network and machine learning workloads. Built around large matrix multiply units—using systolic array architectures that pass data directly between neighboring compute elements—alongside high-bandwidth memory hierarchies, a TPU executes dense, reduced-precision tensor operations such as matrix multiplications and convolutions with minimal instruction-control overhead. Unlike CPUs optimized for general-purpose control flow or GPUs built for broader parallel rendering and compute, a TPU trades architectural flexibility for higher throughput and energy efficiency during large-scale model training and inference.
Common Forms
Single-Host Cloud TPUs: Virtualized server instances attached directly to a pool of one to eight TPU chips, typically used for model prototyping, fine-tuning, and low-latency inference serving.
Cloud TPU Pods: Supercomputer-scale clusters uniting hundreds to thousands of TPU chips over dedicated 2D or 3D torus inter-chip interconnects to handle distributed training of massive models.
Edge TPUs: Compact, low-power coprocessors (such as Google Coral modules) engineered to run quantized 8-bit integer inference locally on embedded hardware, industrial gateways, and IoT devices.
Mobile SoC TPUs: Integrated tensor-acceleration blocks built directly into smartphone systems-on-chip (such as Google Tensor processors) to execute on-device vision, speech, and language tasks in real time.
Example
An AI research team trains a 70-billion-parameter language model on a 256-chip Cloud TPU v5p pod slice using JAX, sharding model weights across the cluster in bfloat16 precision. Because the chips exchange gradients directly over high-speed 3D torus links and stream matrix tiles through systolic arrays, the cluster maintains high compute utilization without bottlenecking on host CPU coordination.