Infrastructure capacity in the AI era: why organisations need to adapt

For much of the last decade, infrastructure capacity has been relatively abundant and affordable. Organisations could scale workloads, increase virtual machine sizes and deploy new services without giving too much thought to resource efficiency. 

But that environment is changing quickly, and the rapid adoption of AI is creating an unprecedented demand for infrastructure resources across the industry. While GPUs often receive the most attention, the reality is that pressure is building across multiple component categories, including memory, CPUs, GPUs and enterprise storage. 

Rather than a short-term disruption, this appears to be a longer-term shift in market dynamics that will require organisations to rethink how they design, consume and manage infrastructure. 

AI Is driving a new wave of demand 

The growth of AI workloads is fundamentally different from previous infrastructure cycles. Memory, in particular, is becoming one of the most valuable resources in modern computing. New generations of AI platforms require significantly greater amounts of high-bandwidth memory (HBM) than previous architectures, and demand continues to accelerate as AI models become larger and more sophisticated. This trend extends beyond memory, with increasing demand for processors, accelerators and storage infrastructure. It’s important to note that this is more than a temporary spike in consumption. 

Unlike the shortages experienced during the COVID-19 pandemic, which were primarily driven by supply-side disruption, today's constraints are largely demand-driven. Organisations across every industry are investing in AI capabilities, and many analysts believe enterprise adoption remains in its early stages. 

As AI moves from experimentation into production environments, demand for infrastructure resources is expected to continue growing.While manufacturers are increasing capacity, significant additional supply is not expected to become available immediately. New fabrication plants, production lines and supporting ecosystems take years rather than months to build and operationalise. 

Even the cloud has its limits 

Public cloud platforms have often been viewed as a way to avoid infrastructure capacity challenges. However, cloud providers ultimately rely on the same physical resources as everyone else. 

Recent reports of capacity constraints within certain Azure regions demonstrate that strong demand can create challenges even at hyperscale. Microsoft has highlighted that Azure is supported by a global network of regions and that it continuously monitors demand and allocates resources to maintain service availability for customers.  

While the specific causes of reported UK South capacity pressures have not been publicly confirmed, industry discussions have suggested that growing demand for AI services may be contributing to wider datacentre resource pressures. Microsoft continues to provide customers with flexibility through alternative Azure regions where appropriate.  

The broader lesson is that no infrastructure model is entirely insulated from supply and demand dynamics. 

A familiar evolution 

There are parallels with the storage market 10–15 years ago. At that time, high-performance storage was often the most expensive and constrained element of infrastructure projects. In response, the industry developed technologies such as deduplication, compression, thin provisioning and auto-tiering to maximise efficiency and reduce waste. 

Over time, these capabilities evolved from premium differentiators into standard features. A similar shift may now be emerging for memory and compute resources. Many applications were designed during a period when memory was relatively cheap and plentiful. As a result, organisations often tolerated oversized virtual machines, underutilised workloads and inefficient resource consumption because the financial impact was minimal. 

Today's market is creating stronger incentives to optimise. The vendors and organisations that can extract more value from existing resources are likely to be better positioned as demand continues to grow. 

Five actions organisations should take now 

Rather than waiting for market conditions to improve, organisations should focus on increasing the efficiency of their existing environments. 

So, what could your organisation action right now?  

• Employ right-size workloads to eliminate over-provisioning and reallocate resources more effectively. 
• Use assessment and observability tools to identify underutilised infrastructure. 
• Optimise workload placement across on-premises, cloud and SaaS environments. 
• Review application strategies and identify opportunities to improve memory and compute efficiency. 
• Improve forecasting and capacity planning to avoid future bottlenecks. 

These actions can help extend existing investments while reducing exposure to ongoing capacity pressures. 

The cost of delaying decisions 

Although organisations should remain disciplined in their investment decisions, waiting for prices to fall or capacity constraints to disappear may prove costly. Most indicators suggest that infrastructure demand has not yet peaked. AI adoption continues to accelerate, while additional manufacturing capacity will take time to come online. 

As a result, infrastructure planning should increasingly focus on resilience, optimisation and long-term forecasting rather than assumptions of abundant, low-cost capacity. The organisations that adapt quickest will be those that treat today's environment not as a temporary shortage, but as a long-term reset in how infrastructure resources are valued, consumed and managed. 

If you’d like to find out more about how Softcat can support your organisation, please click here.