The industry frames inference as a chip-and-cloud problem, but the binding constraint is electrical, and legacy data centers were never designed for it
Industry reporting indicates that a significant share of US data center projects are being delayed or canceled due to power constraints. Not because GPUs are unavailable, not because networking fabrics cannot handle the throughput, but because the physical electricity needed to run inference at the densities the market now demands simply isn’t there. The dominant framing treats AI inference as a problem defined by whoever makes the chips and whoever runs the cloud regions, with power architecture mentioned, if at all, as a background utility assumption. But every GPU allocation and every inference endpoint runs on electrons before it runs on silicon. The gap between what legacy electrical infrastructure can deliver and what inference-density deployments actually require determines whether capacity is real or theoretical.
Inference Demands a Different Electrical Foundation
Most of the data center inventory operating today was designed around average rack densities of 5 to 15 kilowatts, a range that served general-purpose cloud and enterprise workloads well for more than a decade. Inference changes the math. AI-optimized racks push toward hundreds of kilowatts each, and forecasts suggest AI could account for 35 to 50 percent of total data center power consumption by 2030. Training workloads tend toward large, bursty clusters that ramp up and down around discrete jobs, while inference runs continuously across many endpoints with latency sensitivity that penalizes any interruption in sustained power delivery.
The industry is increasingly recognizing this. Cisco has said AI inference is becoming a networking problem, and AMD’s leadership has signaled that the GPU-to-CPU ratio in AI systems will shrink as inference and agentic workloads increase CPU demand. Both observations are correct and reinforce the consensus that inference demands a full-stack architectural rethink. Neither addresses the layer beneath them all. Networking cannot function without power, and CPUs and GPUs cannot run without sustained, high-density electrical delivery. The rethink must start at the electrical layer, or it doesn’t start at all.
Retrofit Cannot Close This Gap
Much of the industry has defaulted to retrofitting existing facilities, upgrading power distribution in buildings originally designed for a different era of compute. Large power transformers now carry lead times that often exceed 120 weeks, new transmission permitting routinely takes many years, and interconnection queues in power-constrained markets have shifted from an inconvenience to a competitive threat for AI providers. Industry networking specialists argue that without a network engineered as part of the computing system, you have built a server farm, not an AI factory. The same logic applies even more strongly at the electrical layer; without power architecture engineered specifically for inference-density operation, you have a building with GPUs in it, not deployable inference capacity.
Enterprises and AI-native companies seeking colocation or edge inference capacity face a gap the hyperscalers aren’t closing. Hyperscale operators build for their own internal workloads with custom electrical designs unavailable to external tenants. Meanwhile, the colocation market largely offers legacy infrastructure with incremental density upgrades that cannot support sustained operation at 75 kilowatts per rack, let alone 200 or more. The capacity inference actually needs sits between those two poles and requires purpose-built power architecture from the ground up.
What Power-First Inference Infrastructure Requires
Building for inference density means treating power architecture as a first-order design decision rather than a procurement line item. Power sourcing has to be modular and multi-source, because waiting for a single grid interconnection to deliver full facility capacity means waiting years while inference demand grows month over month. Facilities that can combine grid power with on-site generation from natural gas, hydrogen fuel cells, or other sources through a unified power conditioning layer can begin operating at initial capacity within months and scale incrementally as demand grows. Industry projections suggest that a growing share of data centers will rely on on-site power by 2030, because the grid alone cannot keep pace.
Power delivery must also be co-designed with cooling for inference-specific thermal characteristics, because sustained heat at high density is a different engineering problem than intermittent peaks across lower-density racks. The relevant metric for evaluating infrastructure readiness isn’t time to power, but what I think of as time to token. That is the interval between committing capital to a site and producing the first useful inference output. Every month a facility sits waiting for grid interconnection is a month of capital deployed with nothing to show for it, and in an environment where competitors are already serving inference traffic, that delay compounds as market-share loss on top of financial drag. Patrick Moorhead has framed AI infrastructure as requiring balance across compute, memory, storage, and networking, warning that misalignment in any one layer undermines results [6]. Power is the layer most prone to misalignment right now because it operates on the longest timelines and, until recently, has drawn the least architectural attention.
For anyone evaluating where to place inference workloads, the better question to ask any facility is not how many GPUs it can house, but whether its electrical architecture was designed for sustained, high-density inference from day one. Those answers will separate providers that deliver inference at scale over the next several years from those still waiting on grid interconnections.
Yuval Bachar is Founder and CEO of ECL, where he leads development of modular, power-flexible data center infrastructure for AI and high-density compute workloads. Before founding ECL, he spent decades working on data center architecture and infrastructure innovation. Connect with him on LinkedIn at www.linkedin.com/in/yuvalbachar.
