AI Tokens vs GPU Hours: Same Chip, Different Bills

EXPLAINED · Versus · Last reviewed: 22 Oct 2026 · Next review: after Alphabet and Microsoft results (28 Oct 2026, US time)

AI tokens and GPU hours are two meters on the same chip. Tokens are how a cloud bills the model — by the million chunks of text, image or sound it reads and writes. GPU hours are how a cloud, or a neocloud, bills the accelerator — by time on a live card. The money only matches if the card turns enough billed tokens in the hour to cover the hour.

AI tokens and GPU as a service are the Mechanism pages. This is not training vs inference or capex vs opex.

What is the difference between AI tokens and GPU hours?

The unit on the invoice. A token is a chunk the model reads or writes; clouds bill most model APIs by the million, with output priced above input. A GPU hour is time on an accelerator in a powered hall, billed as an instance hour, an accelerator-hour, or a dated capacity block.

The same NVIDIA H100 can sit under both meters. A lab that hires the card pays GPU hours. An app that calls Gemini or GPT pays tokens. A hyperscaler that owns the card and sells the API collects tokens and carries the hours.

List prices are not realised prices. Sources dated under Sources.

What is a GPU hour?

Time on an accelerator that is installed, powered and offered. GPU as a service sells that hour.

On-demand is start-and-stop. AWS's on-demand price file, published 15 Oct 2026, lists Linux p5.48xlarge in US East (N. Virginia) at $55.04 an hour for eight H100s — $6.88 per GPU-hour. The one-GPU p5.4xlarge is the same $6.88. Azure's retail API lists Linux Standard_ND96isr_H100_v5 in East US at $98.32 an hour for eight H100s, $12.29 per GPU-hour (our arithmetic).

Capacity blocks are a dated window paid up front. Amazon EC2 Capacity Blocks for ML, checked 22 Oct 2026, list p5.48xlarge in US East (N. Virginia) at $47.757 per instance-hour, or $5.970 per H100-hour. AWS says that table updates next in January 2027. On 24 Sep 2026 the same cell printed $5.191.

Nebius's public on-demand H100 is $4.50 per GPU-hour. CoreWeave lists an eight-GPU HGX H100 node at $49.24 an hour, $6.16 per GPU (our arithmetic, $49.24 ÷ 8).

What is an AI token on the bill?

The chunk the model meters. AI tokens, explained is the definition; this Versus needs the invoice.

Google bills Gemini 3.8 Flash at $0.75 per million input tokens and $3.75 per million output tokens through 31 December 2026, then $1.50 and $7.50 from 1 January 2027. Output costs more because the model writes one token at a time and holds the chip. Thinking tokens are billed as output.

OpenAI's platform list, checked 22 Oct 2026, spans the same unit: gpt-6-luna at $0.10 / $0.50 per million input / output, gpt-6-sol at $2.00 / $10.00, gpt-6-astra at $10.00 / $50.00.

Google prints a token clock on the call. On 22 July 2026 Sundar Pichai said model APIs were processing about 22 billion tokens a minute, up from 16 billion a quarter earlier, and that Google continued to be "supply constrained." Nearly 500 Cloud customers each processed over 1 trillion tokens in the past year. 22 billion a minute is about 950 trillion a month: 22 billion × 60 × 24 × 30 (our arithmetic). Google does not say how many are paid.

How many tokens does a GPU hour need to cover itself?

Enough billed tokens, at the token list, to equal the hour list. That is a conversion ratio, not a claim that Gemini runs on an AWS H100.

Our arithmetic, 22 Oct 2026 lists: tokens needed = hour price ÷ $3.75 × 1,000,000; per second = that figure ÷ 3,600. Mix, discounts, batch and thinking tokens move the real ratio. This is list-on-list, not anyone's margin.

Against OpenAI's list the ratio jumps. At gpt-6-luna output of $0.50 per million, $5.970 is 11.94 million output tokens — about 3,320 a second. At gpt-6-astra output of $50 per million, the same hour needs 119,400 output tokens — about 33 a second (our arithmetic). Expensive tokens cover the machine with less throughput. Cheap tokens need the card packed.

Those speeds are billed output tokens per second that would match the two lists, not a benchmark. Real serving mixes input, output, cache and idle. GPU utilisation is why an idle second still costs the hour.

Who gets paid — and where is the stuck step?

Cash sticks at the API when tokens bill faster than the hour costs. It sticks at the operator when the fleet is full — Oracle's ~20% renewal premium on older GPUs is one dated print. It sticks upstream when CoWoS or HBM hold the next tray, or when a hall waits on watts. A useful test, not a ticker call: if token growth holds while GPU-hour list rises, conversion is still scarce. If utilisation falls while token prices are cut, the machine meter has slack.

Google's TPU is one way to change the conversion without buying a GPU hour from anyone else. Azure and AWS still sell both meters on NVIDIA silicon. Who the hyperscalers are is the buyer map.

What to check next

Alphabet and Microsoft both report on 28 October 2026, US time. Check Google's tokens per minute against 22 billion, Google Cloud's margin against about 35%, and whether Microsoft still says demand exceeds capacity. Then two dated price clocks: AWS Capacity Blocks update next in January 2027, and Gemini 3.8 Flash doubles on 1 January 2027.

Common misconceptions

• People assume tokens and GPU hours are two names for the same bill. Actually one meters the model, the other meters the card. A hyperscaler that owns the card and sells the API collects tokens and carries hours.
• People assume a cheaper token means a cheaper GPU hour. Actually the hour list is set by supply of live cards. AWS's reserved H100 hour moved from $5.191 on 24 Sep 2026 to $5.970 on 22 Oct 2026 while Gemini 3.8 Flash is still on a launch price until year-end.
• People assume list GPU-hours are what the operator earns. Actually realised price after discounts, reserved mix and utilisation decide the earn. Nebius still offers commitment discounts off $4.50.
• People assume 22 billion tokens a minute is 22 billion dollars. Actually Google does not report revenue per token, and the count includes unpaid and batch traffic. Treat it as a usage clock, not an invoice.

Not investment advice. Do your own research.

FAQ

Related terms

• AI tokens — the customer meter, by the million.
• GPU as a service — the machine meter, by the hour.
• GPU utilisation — why idle hours earn nothing.
• AI training vs inference — the two workloads that share both meters.
• TPU vs GPU — Google's chip versus Nvidia's, under the same token bill.

Markets and stack doors

Layer: Hyperscalers. Company model: What does Google do?. Sibling layers: neoclouds (who bill the hour) · semiconductors (who build the card). Markets: US — Virginia's 3,000 MW campus waiting on first power · China. Wire: Nebius GPU-hour prices.

Get the Wire — short daily notes when token counts, GPU-hour lists or capacity clocks move.

Sources