What Is GPU Utilization? Why Idle GPU Hours Earn Nothing
GPU utilization is the share of time a GPU does useful work. A GPU earns only when it is busy, so idle hours are the stuck step. The contract decides who pays for them.
EXPLAINED · Mechanism · Last reviewed: 08 Oct 2026 · Next review: after Nebius and CoreWeave Q3 2026 results (November 2026)
GPU utilization is the share of time, or of capacity, that a GPU spends doing useful work.
In short. GPU utilization is the share of time a GPU does useful work. A rented GPU earns only in the hours a customer pays for. The idle hours still carry power, space and wear. At Nebius’s list price of $9.50 an hour, one B300 earns about $83,220 a year if every hour is billed, and about $8,322 less for every 10 points of idle time. Idle hours are the stuck step. The contract decides who pays for them: the customer under take-or-pay, the operator under on-demand.
Why an investor cares: a GPU is paid for once, but a rented GPU earns only by the hour. Every idle hour is cost with no bill against it, so utilization decides whether a GPU fleet pays back. For how those hours are sold, see GPU as a service.
What is GPU utilization?
It is a share of time or of capacity. The catch is that people use the same words for different numbers.
An engineer means how often the chip is running a task. An AI lab means how much of the chip’s maths a model really uses. A cloud operator means how much of its fleet customers are using or paying for. All three are called utilization.
| Measure | What it counts | Dated example |
|---|---|---|
| Busy time (nvidia-smi) | Percent of a short sample when at least one task ran on the GPU | Sample period of 1 second to 1/6 of a second (NVIDIA docs) |
| MFU (model FLOPs utilization) | Share of the chip’s peak maths a model actually uses | 38–43% for Llama 3 405B on up to 16,384 H100s (Meta, 2024) |
| Effective training time | Time spent on useful training over time elapsed | Above 90% for Llama 3, with 466 interruptions in 54 days (Meta, 2024) |
| Fleet utilization | Share of installed GPUs that customers are using or paying for | 97.9% at Oracle in its quarter to August 2026 (Oracle call, 10 Sep 2026) |
Sources: NVIDIA nvidia-smi documentation (checked 8 Oct 2026); Meta, The Llama 3 Herd of Models (2024); Oracle Q1 FY2027 call (10 Sep 2026). Oracle did not say how it measures its figure.
So a GPU can read 100% busy and still use well under half its maths. The figure an investor needs is the last one: the share of hours that turn into a bill.
How is GPU utilization measured?
Most people see it in nvidia-smi, NVIDIA’s command-line tool. NVIDIA defines GPU utilization there as the percent of time, over a sample period, when one or more kernels was running on the GPU. The sample is between 1 second and 1/6 of a second. A kernel is a task the GPU runs. One small task counts as busy.
AI labs use model FLOPs utilization, or MFU. It compares the maths a model actually does with the chip’s peak. Meta trained Llama 3 405B on up to 16,384 Nvidia H100 GPUs and reported 38% to 43% MFU. More GPUs meant slightly lower MFU: 43% on 8,192 GPUs, 41% on 16,384.
Labs also track effective training time: the share of time spent on useful training. Meta reported above 90%. In one 54-day stretch it had 466 job interruptions, 419 of them unplanned. About 78% of those were confirmed or suspected hardware problems, and GPU faults alone were 58.7%. That is about eight unplanned stops a day (419 ÷ 54, our arithmetic). Meta notes that one GPU failure may force a restart of the whole job.
Why does GPU utilization matter to investors?
Because a rented GPU earns by the hour. Power, space, staff and wear run whether or not a customer is using it.
Take one Nvidia B300 at Nebius. From 1 October 2026 its on-demand list price is $9.50 per GPU-hour, up from $7.85. A year has 8,760 hours. Billed every hour, one B300 brings in about $83,220 a year at list. Every 10 points of idle time takes away about $8,322 (our arithmetic).
| Share of hours billed | Revenue per B300 per year | Lost against 100% |
|---|---|---|
| 100% | $83,220 | $0 |
| 90% | $74,898 | $8,322 |
| 70% | $58,254 | $24,966 |
| 50% | $41,610 | $41,610 |
Source: Nebius AI cloud prices, HGX B300 on-demand $9.50 per GPU-hour effective 1 Oct 2026 (checked 8 Oct 2026). Revenue = $9.50 × 8,760 hours × share billed, our arithmetic. This is list price. Commitments are up to 35% cheaper, so realised revenue per hour is lower.
Utilization also changes the accounting. Nebius now spreads the cost of its servers over five years, up from four before 2026. It said the change is “based on usage patterns and current utilization commitments.” A longer life lowers the yearly cost on paper. It only holds if the GPUs keep earning. See GPU depreciation.
Who pays for idle GPU hours?
The contract decides. The same idle hour can land on the customer or on the operator.
| Contract | Who pays for an idle hour | Dated print |
|---|---|---|
| Take-or-pay | The customer. It pays for reserved hours, used or not | 98% of CoreWeave’s revenue in Q2 2026 (10-Q, 11 Aug 2026) |
| On-demand | The operator. No hour run, no bill | Nebius B300 $9.50 per GPU-hour from 1 Oct 2026, up from $7.85 |
| Spot | The operator, partly. It sells spare hours cheap and can take them back | Nebius B300 spot from $0.99 per GPU-hour, about a tenth of on-demand (our arithmetic) |
| Short term (3–6 months) | The customer, for the term, at a premium | Nebius: “priced at a significant premium” (Q2 2026 letter) |
Sources: CoreWeave Form 10-Q (11 Aug 2026); Nebius AI cloud prices (checked 8 Oct 2026); Nebius Q2 2026 letter to shareholders (12 Aug 2026). $0.99 ÷ $9.50 = 10.4%, our arithmetic.
CoreWeave sells mostly take-or-pay. Its 10-Q says it sells access “either through committed contracts, which are take-or-pay, or on-demand, which are pay-as-you-go.” Committed contracts were 98% of its revenue in the quarter to 30 June 2026. So most idle hours on its fleet are the customer’s cost, not CoreWeave’s. See take-or-pay GPU contracts.
Nebius keeps more hours open on purpose. In its Q2 2026 letter it said: “We could sell our entire 2027 capacity on these terms today. We are deliberately not doing so.” It keeps some capacity for immediate customer needs, sold on-demand or on short contracts of three to six months at “a significant premium.” That earns more per hour, but only while those hours fill. An unsold hour can go to spot, where a B300 starts at $0.99, about a tenth of on-demand.
Where is the stuck step in GPU utilization?
Idle hours. The chip and memory makers were paid when the GPU shipped. The operator is paid only when an hour is billed. Everything between is the stuck step.
Idle hours come from a few places. Gaps between contracts. Failures and restarts, as Meta’s numbers show. And in AI inference, spare capacity held for speed. Nebius said on 1 October 2026 that cold starts, the time a model needs to load, leave assigned GPUs idle at launch, during demand spikes and when weights update mid-run. That, it said, “forces platforms to hold spare capacity just to hit their service-level targets.”
Nebius bought Inferize that day to cut what it called this “idle GPU tax.” It did not disclose the price or say how much idle time it expects to remove. For what that bet implies for the company, see our Nebius idle-GPU analysis and the Wire on the Inferize deal.
A full fleet looks different. Oracle said its GPU utilization was 97.9% in its quarter to August 2026, after adding 850 MW and more than 300,000 GPUs. GPUs that came up for renewal were renewed or resold at a 20% premium, and most were four years old or more. Busy older GPUs still earned more. See the Oracle Wire.
What to check next
Two dated checks, both expected in November 2026. Neither company had set a date as of 8 October. Nebius’s Q3 2026 results: does it report any gain from Inferize, and does it keep holding capacity back for short, premium deals? CoreWeave’s Q3 2026 results: does the committed share stay near 98%? Then watch Oracle’s next utilization figure against 97.9%.
Common misconceptions
- People assume 100% in nvidia-smi means the GPU is fully used. Actually it means a task ran at some point in each short sample. Meta’s Llama 3 used 38–43% of peak maths.
- People assume all utilization figures are the same. Actually busy time, MFU and fleet utilization measure different things. Oracle’s 97.9% does not say which.
- People assume idle GPUs cost nothing. Actually power, space and wear keep running. A spare B300 at Nebius forgoes $9.50 of list price every hour.
- People assume the operator always eats idle time. Actually under take-or-pay the customer pays for reserved hours, used or not.
FAQ
What is GPU utilization?
GPU utilization is the share of time, or of capacity, that a GPU spends doing useful work. Engineers, AI labs and cloud operators each measure it differently, so always ask which number you are looking at.
How is GPU utilization measured?
The common tool, nvidia-smi, reports the percent of time in a short sample when at least one task (a kernel) was running on the GPU. It does not tell you how much of the chip’s maths was in use. AI labs measure that with model FLOPs utilization (MFU).
What is a good GPU utilization?
It depends on the measure. Meta reported 38% to 43% MFU while training Llama 3 405B, which was a strong result at that scale. Oracle reported 97.9% utilization for its rented GPU fleet in its quarter to August 2026, on its own measure.
Why does GPU utilization matter to investors?
Because a rented GPU earns only in the hours a customer pays for. At Nebius’s 1 October 2026 list price of $9.50 an hour, one B300 earns about $83,220 a year if every hour is billed, and about $8,322 less for every 10 points of idle time.
Who pays for idle GPU hours?
The contract decides. Under a take-or-pay contract the customer pays for reserved hours even if they sit idle. Under on-demand pricing the cloud operator carries the idle hours itself.
Is this a list of stocks to buy or sell?
No. Second Order Explained pages explain who does what and how the money moves. They do not give buy or sell calls.
Related terms
- GPU as a service — how GPU hours are sold.
- Take-or-pay GPU contract — when the customer pays for idle hours.
- GPU depreciation — how the cost of a GPU is spread over its life.
- Neocloud — the specialist GPU clouds that sell these hours.
Stack doors
Layer: Neoclouds. Markets: how CoreWeave’s contracts become cash. Analysis: if take-or-pay bills idle hours, where does cash stick?.
Get the Wire — short daily notes when GPU prices, contracts or utilization figures move.
How we check this
Figures come from company filings, results calls, published price lists, NVIDIA’s own documentation and Meta’s published research, each dated in the list below. Where a company reports a utilization figure without saying how it measures it, we say so. Where we work something out ourselves, like revenue per GPU at a given utilization, the arithmetic is shown in full and uses list prices, not realised prices. This page explains how the money moves. It does not give buy or sell calls.
Last reviewed: 08 Oct 2026 · Next review: after Nebius and CoreWeave Q3 2026 results (November 2026)
Spot an error? Tell us and we will correct it and note the change here.
Sources
- NVIDIA — nvidia-smi documentation, definition of GPU utilization (v610, checked 8 Oct 2026)
- Meta — The Llama 3 Herd of Models, arXiv 2407.21783 (31 Jul 2024, revised 23 Nov 2024)
- Nebius — Nebius acquires Inferize, EQS corporate announcement (1 Oct 2026)
- Nebius — AI cloud prices, on-demand and spot GPU-hour (effective 1 Oct 2026; checked 8 Oct 2026)
- Nebius Group — Q2 2026 letter to shareholders (12 Aug 2026)
- CoreWeave — Form 10-Q for the quarter to 30 Jun 2026, SEC filing (11 Aug 2026)
- Oracle Q1 fiscal 2027 earnings call transcript (10 Sep 2026)
