Fermi Numbers for AI Datacenters

for back-of-the-envelope datacenter stat estimates

I noticed that I’d hear news headlines like “OpenAI completes $X billion datacenter” or “Anthropic completes X megawatt datacenter” and not really have a felt sense of what that actually means.

Here’s a collection of numbers that’ll get you a lot closer to doing back-of-the-envelope Fermi estimates when you hear a datacenter stat.

Compute

It can be handy to measure compute in H100 equivalents. Here’s the GPU lineup:

A100 (~0.3x) · H20 (~0.3x) · H100 (1x) · B30A (~2x) · B300 (~4x)

10k H100 equivalents = GPT-4. GPT-4 was trained for 3 months. You can either 2x the GPUs or 2x the time, so 10k H100s for 3 months = 5k H100s for 6 months.

The largest current AI datacenter is around 275k H100 equivalents (xAI Colossus Memphis Phase 3) — about 30x what GPT-4 was trained on.

Energy

Reference points for power:

1 H100 in a datacenter ≈ 1,400W (including cooling/networking overhead) ≈ 1 American household
City of Seattle ≈ 1 GW
Washington state ≈ 10 GW
Total US electricity ≈ 485 GW

Datacenter power consumption:

GPT-4 datacenter: ~0.03 GW (28.9 MW)
Largest current AI datacenter (xAI Memphis Phase 3): ~0.35 GW
A 1M H100 datacenter would be ~1.5 GW (1.5x Seattle)
A 10 GW datacenter would be ~30x the current largest, similar to all of Washington state

The 1,400W per H100 rule holds up well. xAI Memphis Phase 3 has 275,796 H100e — this predicts 386 MW, actual power is 352 MW.

FLOPs

~10^24 flops = GPT-3.5
~10^25 flops = GPT-4
~10^26 flops = Grok-4

Anchoring on GPT-4 is nice. If you hear a model was trained with 10^27 flops, just subtract 27-25 = 2. That’s 100x GPT-4.

Datacenter Spending

Ballpark: ~$40k per H100 equivalent (whole datacenter, not just chips).

Datacenter	Cost	H100e	$/H100e
Anthropic-Amazon New Carlisle	$15B	300k	$50,000
xAI Colossus 2	$9B	280k	$32,000
OpenAI Stargate Abilene	$8B	250k	$32,000

So if a datacenter costs $10B, it probably has around 250k H100e.

Cost breakdown (excluding labor): Chips ~52% · Server components ~24% · Networking ~18% · Energy ~6%

Geopolitics

The US currently has ~6x as many H100 equivalents as China.

US: 1.395 million H100e
China: 231,651 H100e
Tennessee alone (583k H100e) has 2.5x the compute of China

Global distribution: US ~75% · China ~15% · EU ~5% · Rest of world ~5%

The most advanced chip legally exported to China is the H20 (~0.3x H100).

Most of this is a hodgepodge of stats I stole from Epoch, Semianalysis, AI Futures, and Situational Awareness. Thanks to Adam Khoja and Arunim Agarwal for discussions, and Konstantin Pilz for answering a clarifying question.

PS: If someone turns these into a nice Anki deck (with pretty good prompts), I’ll Zelle you $10.