Discussions of AI’s electricity demand often treat “AI load” as a single thing. It is not. The two phases of an AI system’s life — training, when the model learns, and inference, when the model answers questions and generates output — impose fundamentally different demands on the grid. The distinction matters for forecasting, for interconnection, and for understanding where the growth is actually headed.
What training looks like
Training a large AI model means running enormous numbers of calculations across thousands or tens of thousands of specialized processors, working in tight coordination for weeks or months at a time. The processors — graphics processing units (GPUs) and similar accelerators — are the most power-hungry computing hardware ever deployed at scale. A single AI accelerator chip can draw 700 watts to more than 1,000 watts, several times what a traditional server processor draws. Packed densely into racks, these chips create power densities that would have been unthinkable in a data center a decade ago.
A modern AI training cluster — the industry term for the coordinated group of servers that trains one model — can draw tens to hundreds of megawatts. The largest training facilities in operation or under construction are sized in the hundreds of megawatts, with some planned campuses targeting a gigawatt or more. To put that in perspective: a single AI training campus can consume as much power as a medium-size American city, drawn continuously, 24 hours a day, for the duration of the training run.
The load profile of training is distinctive. Utilization runs extremely high — often above 90 percent of the cluster’s capacity — because the hardware is expensive and the operator wants maximum throughput. But the draw is not perfectly flat. Training involves synchronized computation: thousands of processors compute, then pause briefly to exchange results, then compute again. These synchronization cycles create rapid, repeated fluctuations in power draw — ramps of tens of megawatts in seconds — that propagate up through the facility’s power systems and, potentially, to the grid.
This matters for power quality. The grid is designed to handle gradual changes in demand over minutes and hours. Extremely fast, repeated swings can stress voltage regulation equipment and, in the worst case, interact with the stability of the local transmission system. Grid operators and utilities are now studying these dynamics and, in some cases, requiring large AI facilities to install equipment that smooths their draw before it reaches the grid.
Another distinctive feature of training is its lumpiness in time. A training run lasts weeks to months, then ends. The cluster may sit partially idle, be reconfigured, or move to a new workload. From the grid’s perspective, this means a facility whose maximum draw is enormous but whose long-term average may be lower and more variable than a traditional data center’s. Forecasting that behavior requires understanding the customer’s workload pipeline, not just the nameplate capacity of the building.
What inference looks like
Inference is what happens when a trained model is actually used: answering a chatbot query, generating an image, transcribing speech, recommending a product. Inference runs continuously, serving millions of individual requests, and it is already the larger share of AI computing by total volume in many deployments.
Per request, inference is far less demanding than training. A single query might engage a fraction of a model’s parameters for a fraction of a second. But inference must be fast — users expect responses in seconds — and it must be available everywhere, all the time. That means inference infrastructure is distributed: instead of one giant cluster, inference runs in data centers spread across regions, close to users, operating at high availability around the clock.
The aggregate power draw of inference is large and growing, and its growth dynamics differ from training’s. Training demand grows in discrete jumps as new, larger models are trained on new, larger clusters. Inference demand grows more smoothly, tracking the adoption of AI-powered products — more users, more queries, more features. As AI capabilities get embedded into everyday software, inference becomes a steadily rising baseload on the grid, less visible but ultimately more persistent than training.
Inference also has a different geographic logic. Training clusters can be sited wherever power and land are cheapest, because the output — a trained model, essentially a large file of numbers — can be copied anywhere instantly. Inference benefits from being near users to minimize latency, which pushes it toward population centers and existing network hubs. This means inference load shows up in more places, in smaller increments, integrated into the existing data center footprint.
Why the distinction matters for the grid
For grid planners, training and inference create different problems.
Training creates the interconnection challenge: enormous single-site loads, arriving quickly, with unusual power-quality characteristics. The utility must build or upgrade substations, feeders, and sometimes transmission lines to serve one customer. The load may be lumpy over time. The facility may need specialized treatment for its fast-ramping behavior. And the timeline is compressed — the customer wants power in two to three years, while infrastructure takes five to ten.
Inference creates the forecasting challenge: thousands of smaller additions, spread across territories, growing steadily. No single interconnection request is dramatic, but the aggregate adds up. Traditional forecasting methods, which extrapolate from historical trends, can miss this kind of growth because it has no historical precedent. Utilities are having to develop new forecasting approaches that model technology adoption directly rather than extrapolating past load.
The two also interact with the generation mix differently. Training’s flat, continuous, location-flexible profile makes it a natural match for around-the-clock clean power — nuclear, geothermal, or firmed renewables — and for the “bring your own power” trend of on-site generation. Inference’s distributed, latency-sensitive profile is harder to serve with dedicated generation and more likely to draw from the general grid mix.
The efficiency question
Both phases are getting more efficient per unit of computation, but the efficiency gains are being outpaced by growth in total computation — the classic rebound effect. Each generation of AI accelerator delivers more calculations per watt than the last. Model architectures are being optimized to need fewer computations per query. Data center cooling keeps improving.
Yet total power draw keeps rising, because the efficiency gains make AI cheaper to deploy, which expands its use. A model that needs half the energy per query but serves ten times as many queries uses five times the energy. This dynamic — efficiency enabling growth — has characterized computing for decades, and there is no sign it is abating for AI.
For illustration, consider a hypothetical inference deployment serving a popular consumer application. If each query consumes roughly 1 watt-hour of data center energy (an illustrative figure, not a measured one), a billion queries per day consumes 1 gigawatt-hour per day — the output of a 40-megawatt power plant running around the clock. Scale that across dozens of widely used AI features, and the aggregate becomes a meaningful share of grid load, built query by query.
What this means for demand forecasts
Forecasters now try to model training and inference separately, because they grow for different reasons and show up in different places. Training forecasts depend on the capital expenditure plans of a handful of large technology companies and the cadence of new model generations. Inference forecasts depend on product adoption curves — how quickly AI features reach users and how intensively they get used.
Both are uncertain, and the uncertainty compounds. If models get much more efficient, training clusters may grow more slowly — but cheaper inference could explode adoption. If AI capabilities plateau, training demand could level off while inference continues its steady climb. The scenarios diverge widely, which is why responsible forecasts present ranges rather than point estimates, and why utilities are building flexibility into their planning — procuring generation that can serve either profile, and structuring contracts that accommodate uncertainty.
Power quality: the hidden grid challenge of training
The fast, synchronized power swings of AI training deserve a closer look, because they represent a genuinely new phenomenon for grid operators. Traditional large loads — factories, mines, smelters — change their demand over minutes or hours. AI training clusters can swing tens of megawatts in seconds, repeatedly, for the duration of a training run. These are not gradual ramps; they are sharp, high-frequency fluctuations.
The concern is resonance and voltage flicker. Rapid load swings can cause local voltage to oscillate, which affects power quality for neighboring customers and stresses utility voltage-regulation equipment — capacitor banks, transformer tap changers — that was designed for slower duty cycles. In extreme cases, large synchronized swings could interact with the electromechanical dynamics of the transmission system itself, though this remains a subject of active study rather than observed crisis.
The industry’s response is emerging on two fronts. Facility designers are adding fast-acting power electronics — STATCOMs, dynamic VAR compensators, and large-scale battery systems — that absorb the swings before they reach the grid, presenting the utility with a smooth, well-behaved load. Grid operators, meanwhile, are developing interconnection requirements specific to large AI loads: ramp-rate limits, power-factor standards, and ride-through requirements that keep the facility stable during grid disturbances. Expect these technical standards to tighten as training clusters grow — the physics scales with size.
The inference buildout inside existing facilities
One underappreciated aspect of inference growth is that much of it happens inside data centers that already exist. Unlike training, which often requires purpose-built campuses with extreme power density, inference can run on conventional server infrastructure, in colocation facilities, and at the network edge — small deployments in telecom facilities, office buildings, and retail locations.
This has two implications. First, inference growth is partially invisible to the interconnection queue: it shows up as rising utilization of existing capacity rather than as new large-load requests. A colocation facility contracted for 20 megawatts might have historically drawn 12; as inference workloads fill it, the draw creeps toward the contracted maximum without any new interconnection filing. Forecasters who track only the queue miss this intensive-margin growth.
Second, edge inference distributes AI’s power draw into the fabric of the existing grid — thousands of small additions rather than a few giant ones. Each is trivial; the aggregate is not. Planning for it requires metering, data collection, and forecasting methods tuned to the distribution system, not just the transmission queue.
The practical takeaway: when you see a headline number for “AI electricity demand,” ask which phase it describes. A forecast dominated by training tells a story about giant campuses and interconnection queues. A forecast dominated by inference tells a story about steady, distributed growth woven into the existing grid. The real future contains both — and the grid has to be ready for each.