Google TPU 8tx and 8i Max: Confidential inference, large-context processing

1. Two additions bring the lineup to four systems

Google added two systems in its November 9 announcement. They follow TPU 8t for training and TPU 8i for real-time inference, unveiled in April. The company said this was its first time introducing four TPU systems in one year. Rather than designing separate new chips, it broadened their uses with different security, memory, and networking configurations around existing chips.

Some eighth-generation performance comparisons appear in Google's specification sheet. The figures below compare the 2025 Ironwood generation with the 2026 TPU 8t generation at the pod level, a system grouping multiple chips. Compute figures use FP4 (4-bit floating-point operations).

Table: TPU 8t and Ironwood specifications

Conditions: Google's comparison of training system specifications. Figures refer to pod configurations for Ironwood (2025) and TPU 8t (2026). Compute uses FP4, and bandwidth figures are per chip.

Calculated from the table, chips per pod rise by about 4.2%, while FP4 compute increases to about 2.85x. Bidirectional scale-up bandwidth per chip is 2x, and scale-out network bandwidth is 4x. The configuration emphasizes compute and communication bandwidth more than chip count. According to the company, 8t offers nearly 3x the compute performance per pod of the previous generation.

Chart: TPU 8t specification multiples relative to Ironwood. Conditions: Each TPU 8t figure in the table above is divided by the corresponding Ironwood figure. Decimal multiples are rounded approximations, and bar lengths are proportional to the multiples. Unit: Each Ironwood metric = 1

Ironwood baseline for each metric          ████             1
TPU 8t chips per pod                      ████▏            about 1.04
TPU 8t FP4 compute per pod                ███████████▍     about 2.85
TPU 8t bidirectional scale-up BW per chip  ████████         2
TPU 8t scale-out network BW per chip      ████████████████ 4

Google also gave performance comparisons in its November announcement. It said 8t delivers up to 2.7x the training performance of the previous generation, while 8i improves inference price-performance by 80%. These improvements target training throughput and inference cost efficiency, respectively.

2. 8tx uses hardware to protect data in use

Unlike data at rest or in transit, data in use must be decrypted in memory. 8tx is designed to protect and isolate sensitive data and model weights at the hardware layer during processing. Google targets businesses in regulated industries, governments, and users running proprietary models or autonomous agents with sensitive data.

It uses the same silicon as the training system, with fifth-generation AMD EPYC hosts. It pairs this with the low-latency Boardfly network developed for real-time inference.

According to the company, 8tx offers up to 2x lower latency than 8t, with almost no performance penalty from security. Google says combining training-class compute with a fast inference network also makes it suitable for small-scale training.

Google says users can fine-tune models, distill them, and perform reinforcement learning without moving internal data to another cluster. Fine-tuning adds training for specific tasks, distillation transfers knowledge from a larger model to a smaller one, and reinforcement learning uses rewards based on actions' outcomes. This keeps model operation and customized training on one system while reducing sensitive data movement.

3. 8i Max addresses memory bottlenecks and rack density

As multiple agents handle long inputs such as entire codebases or lengthy videos, the speed of feeding data to compute units can become a bottleneck. 8i Max combines two low-latency 8i chips in one package, increasing both compute and memory.

Table: TPU 8i Max configuration expansion over standard TPU 8i

Conditions: Based on Google's announcement. This compares the specifications of TPU 8i Max, which combines two TPU 8i chips in one package, with standard TPU 8i.

Unit: Multiple relative to standard TPU 8i

According to the company, all three metrics double together. Rather than increasing compute alone, the design keeps required data nearby and supplies it quickly to reduce waiting when processing long contexts.

The standard 8i has 288GB of high-bandwidth memory and 384MB of SRAM, fast on-chip working memory. Google says its SRAM capacity is 3x that of the previous generation.

Google says Boardfly reduces the maximum number of hops in chip-to-chip communication by more than 50%. An on-chip engine accelerates collective operations, which gather or distribute data across chips, reducing on-chip latency by up to 5x. The focus is on cutting the time chips spend waiting while exchanging data.

4. Software lowers adoption barriers and idle costs

Google outlined its software support in an announcement. It said it open-sourced TorchTPU so PyTorch code can run on TPUs without being rewritten. This lowers adoption barriers by letting developers switch accelerators while keeping existing workflows.

Agent Compute Engine pauses sessions and moves working memory off-chip while agents wait for external tools or human responses. This reduces the cost of tying up resources when no computation is taking place.

TPU Sync treats chip and host memory, shared storage, and networking as one memory hierarchy. Google says it supports moving large key-value caches, which store computation results from previously processed context, and synchronizing reinforcement learning weights. The company reported reductions of more than 90% in cache transfer latency and time to first token.

Dynamic slicing in GKE, Google Cloud's managed Kubernetes service, divides resource groups to match workload size. It combines TPU groups into larger units or splits them into smaller ones as needed. According to the company, job startup is up to 5x faster and failure recovery up to 4.5x faster.

The system configuration carries both benefits and trade-offs. 8i Max provides more compute in the same rack footprint, but achieves that density by deploying twice the silicon. The design's central point is that memory bandwidth and data movement management must also scale to reduce the time compute resources spend waiting.

The eighth-generation TPU expansion goes beyond chip performance competition toward choosing data center systems to match security needs and context scale.