AI Training vs Inference: Same GPUs, Different Cheques

EXPLAINED · Versus · Last reviewed: 12 Oct 2026 · Next review: after CoreWeave’s and Nebius’s Q3 2026 results (expected Nov 2026)

AI training builds a model by feeding it data until it learns. AI inference runs the finished model to answer each new request, such as a chat reply or a line of code.

Why an investor cares: the two are paid for in different ways. Training fills a neocloud’s backlog with long contracts that make cash predictable. Inference is billed as it is used, often at higher prices, and is now the bigger share of AI compute. For the inference side on its own, see what AI inference is.

What is the difference between AI training and inference?

Training is the learning step. A model lab runs huge amounts of text, code or images through thousands of GPUs until the model gets good at predicting the next word or pixel. It is done in big planned runs before a model launches, then again for each new version.

Inference is the using step. Every chat reply, code suggestion or search summary is the finished model running once more. It never stops while people use the product.

From Deloitte’s TMT Predictions 2026, CoreWeave’s and Nebius’s Q2 2026 reports and NVIDIA’s Q2 fiscal 2027 call. “Our read” marks our own judgment.

The line is blurring. Nebius told shareholders in its Q2 2026 letter that a growing share of inference demand is generated by training itself: reinforcement learning, evaluations and synthetic data all need the model to answer questions while it is still being trained.

Which uses more compute, training or inference?

Inference, and its share is growing. Deloitte estimates inference was about a third of all AI compute in 2023 and half in 2025, and expects about two-thirds in 2026. It also expects chips built for inference to be a market of more than $50 billion in 2026.

That does not mean training is shrinking. Deloitte says growth in compute for training new models has slowed but is still growing, while newer methods such as post-training and long reasoning at answer time add compute on top. More answers, and longer answers, both land on the inference side.

How is AI training paid for compared with inference?

Training is usually bought as a reserved block. A model lab needs thousands of GPUs at once, in one place, for a planned run, so it signs for capacity ahead of time. CoreWeave told investors on 11 August 2026 that a typical contract runs five years, with the capex spent up front and funded by debt, customer prepayments and other capital. How that commitment works: take-or-pay GPU contracts.

Inference is billed as it is used. A customer can rent the GPU by the hour and run the model itself, or pay the cloud per token and let it run the model. AI tokens explains the second meter.

CoreWeave Q2 2026 call (11 Aug 2026), Nebius Q2 2026 letter, Nebius price pages read 10 Oct 2026. Which rows are mostly training and which mostly inference is our read; the companies do not split it that way.

Shorter is pricier. CoreWeave said shorter contracts typically come at a higher price and margin, and that it raised prices by about 25% across its GPU types, partly because customers were earning more from its platform as they shift to inference. Nebius prices its three-to-six-month deals at a significant premium.

Who gets paid: training vs inference

The money moves in the same order for both. What changes is how the neocloud’s cheque arrives.

• 1. The chip seller is paid first. GPUs are bought before any training run or answer. NVIDIA told investors on 26 August 2026 that its platform is “great at training, great at inference”, so the same purchase serves both. See TPU vs GPU for the custom-chip alternative.
• 2. The hall and the power. Both need a powered building. Training wants a big block on one campus; inference can use smaller halls in more places (our read). More in what an AI data center is.
• 3. The neocloud, two ways. Training pays a fixed amount over years, which lenders like. Inference pays per hour or token, at higher prices, but only while used. CoreWeave’s managed inference business went from $1 million to more than $100 million of booked annual run rate in a few months, and it expects at least $250 million by the end of 2026. That is still small next to $2.6 billion of revenue in one quarter. How neoclouds work: neocloud explained.
• 4. The model maker or app. Training is a cost the lab pays up front. Inference is where it earns, every time a paying user asks. NVIDIA said on its August call that frontier labs are growing faster than their balance sheets can support, which is why who finances GPU clusters matters.

Where is the stuck step?

GPUs that are installed, powered and switched on. Training and inference compete for the same racks, so whichever side is growing, it waits for the same thing. CoreWeave’s chief executive said on the August call that you don’t build infrastructure for training or for inference, you build AI infrastructure.

CoreWeave had 1.5 gigawatts of active power at its Q2 results against 4.2 gigawatts contracted, so most of what it has signed is not yet live (our arithmetic: about 36%). It said its managed inference growth was constrained only by near-term capacity. Our read on what that gap means: CoreWeave’s power-on stuck step.

Inference adds a second, smaller bind: spare GPUs. Requests come in bursts, so a provider has to hold capacity ready, and an idle GPU earns nothing. Nebius now sells spare H100 hours from $0.79, about 18% of its $4.50 on-demand rate (our arithmetic). See Nebius’s move to spot prices and GPU utilization explained.

What would prove this wrong?

• Inference moves to cheaper chips. If inference-only chips take most new inference work, it stops competing with training for the same GPUs, and the two get separate stuck steps.
• Short-contract prices fall. If on-demand and short-contract prices drop below long-contract rates, there is spare capacity, and live GPUs are no longer the bind.
• Training demand fades. If labs stop signing big reserved blocks, neocloud backlogs shrink and inference has to carry the build-out on its own.

What to check next

CoreWeave’s and Nebius’s third-quarter results, expected in November 2026. Look for managed inference run rate against CoreWeave’s $250 million target, active against contracted power, and whether short-contract and on-demand prices hold. Our US note on CoreWeave and Vera Rubin tracks the same gap.

Common misconceptions

• People assume inference is cheap, so it needs few GPUs. Actually each answer is cheap, but Deloitte expects inference to be about two-thirds of all AI compute in 2026.
• People assume training and inference run on different hardware. Actually most of both runs on the same GPUs, and neoclouds sell the same racks for either.
• People assume training is over once a model launches. Actually each new version is trained again, and post-training keeps going; Nebius says a growing share of inference demand comes from training itself.

FAQ

Related terms: AI inference; AI tokens; capex vs opex; GPU cloud providers.

Stack doors

Layer: Neoclouds. Also: Hyperscalers, Semiconductors, AI Data Centers. Markets: US.

Get the Wire — short daily notes when GPU contracts, prices and capacity move.

References