Tokens per joule: the metric for the AI era

Speed built the models. Energy will decide who gets to run them.

For a decade, AI progress was measured in FLOPS and wall-clock time: how fast can you train a bigger model. That race produced the models — and a habit. We learned to judge every chip by raw speed. The habit no longer fits the workload that now dominates AI compute.

Inference, not training, is where the energy goes. For every hour spent training a model, thousands are spent running it. And running it is mostly not computing. On a modern GPU, roughly 90% of the energy spent on an inference is spent moving data — shuttling weights between memory and the compute units — not on the arithmetic that constitutes thinking.

A better unit

So the number that actually governs the economics and the sustainability of AI at scale isn't operations per second. It's tokens per joule: how much useful intelligence you get for each unit of energy spent.

Measured this way, the hardware question inverts. The winning chip isn't the one that computes fastest; it's the one that has to move the least to compute at all. Energy spent on data movement is pure overhead — it produces no tokens. Drive it toward zero and tokens per joule rises with it.

Why it matters now

• Cost. Inference is the line item that scales with usage. Tokens per joule is unit economics.
• Sustainability. Inference is on track to consume a growing share of global energy; efficiency is the only path that scales.
• Capability. Lower energy per token unlocks workloads impractical today — always-on, on-device, agentic.

A flagship datacentre GPU today delivers on the order of half a token per joule. Phynomy's architecture is designed for tens of times more — figures that remain indicative, drawn from early simulation and independent research, not a shipped product. But the direction is the point. The next decade of AI will be won on intelligence per joule, and the chip that wins it will be built around that metric from the first principle.