AI is moving fast, but the infrastructure behind it is still catching up. There’s no shortage of investment, with the AI market expected to reach $2.4 trillion by 2032. At the same time, however, many organizations are running into the same old challenges. Around 44% say IT infrastructure is still the biggest barrier to scaling their AI initiatives, and most have already experienced performance issues in the past year. Confidence isn’t where it needs to be either, with nearly four in five organizations unsure whether their current setup will meet AI demands over the next few years. What’s becoming clear is that AI success is no longer just about raw compute or the capability of models, but how well network infrastructure supports it.
I recently sat down with William Sigmund, Director of Sales Engineering at phoenixNAP, to explore this in more detail in a webinar titled Benefits of Peering for AI Workloads. We wanted to get a clearer picture of how organizations are approaching AI infrastructure today, where the pressure points really are, and how connectivity choices are starting to shape performance in practice. Through a series of live polls and discussion with attendees, one consistent theme emerged: Businesses aren’t short on ambition when it comes to AI, but many are still figuring out how to build the kind of infrastructure that can actually support it at scale.
What our pulse poll revealed
One of the first things we asked attendees was how they’re currently running their AI workloads. Unsurprisingly, most indicated they are operating in hybrid or multi-cloud environments. That lines up closely with what we’re seeing across the industry more broadly, where according to A10 Networks’ State of
AI Infrastructure Report 2025, around 42% of organizations combine on-prem infrastructure with cloud, and a further 35% lean heavily on public cloud environments. That makes sense. Latency-sensitive workloads tend to stay closer to home, while less time-critical processing is pushed into the cloud. But what stood out was the lack of confidence behind it. A large majority of organizations still aren’t convinced their current infrastructure will hold up as their AI demands grow. Performance came through as the number one pressure point in our discussions.
When we asked about the biggest challenges, responses clustered around compute limitations, software constraints, and consistently, the network. That’s not surprising when you look at the wider industry data. The same report revealed that bandwidth issues are rising sharply, with 59% of organizations reporting problems in 2025, up from 43% the year before. Latency concerns have followed a similar path, increasing from 32% to 53% over the same period. Even areas that aren’t always front of mind, like storage, are starting to surface, with 41% planning upgrades. What’s also interesting is how evenly split the audience was on peering. Roughly half are already using it in some form, while the other half either aren’t or aren’t sure. That gap tells its own story. Many organizations recognize the challenges in front of them, but not all are using every option available to address them. More on that later.
Why AI changes the infrastructure conversation
One of the most important points we covered in the session is that AI isn’t a single, uniform workload. It operates in two very different modes, and each places its own demands on infrastructure. On the training side, the priority is moving large volumes of data into models as efficiently as possible. Bandwidth matters here more than anything else. The data can come from anywhere, and while speed is important, a few extra milliseconds won’t make or break the process. Here, it’s all about scale and throughput.
Inference is a very different picture though. This is where AI moves from development and training into real-world use, responding to inputs, supporting real-time decisions, and in some cases interacting continuously with users or systems where every millisecond matters. Whether it’s a vehicle responding to changing conditions, a sensor-based system making real-time adjustments, or a user expecting an instant answer to a query, delays quickly become noticeable, disruptive, and potentially life threatening. That’s where the network starts to take on a much more central role. The path data takes, the number of hops involved, and the distance it travels all begin to directly influence how well AI performs. Looking back at our poll results, as well as the wider data showing bandwidth issues and latency concerns rising, and the picture becomes clearer still. Organizations aren’t just dealing with more data, they’re supporting more dynamic, distributed, and time-sensitive workloads than ever before.
Where peering fits in a distributed AI environment
This is where the conversation naturally turns to connectivity, and more specifically, to peering. At its simplest, peering is about creating a more direct path between networks. Instead of sending traffic through multiple intermediary providers, you’re connecting straight to the networks and services you rely on. Fewer hops, shorter routes, and more control over how your data moves. That has a direct impact on performance, especially for latency-sensitive AI workloads where every millisecond counts, and at a time when more than half of organizations are already reporting bandwidth and latency challenges.
What stood out was how evenly split the audience was on peering itself. Roughly half are already using it in some form, while the other half either aren’t or aren’t sure. That means many organizations are still relying on traditional connectivity models that weren’t designed for the kind of distributed, performance-sensitive workloads AI is driving.
But it’s not just about speed. Peering also gives organizations greater visibility and control over their traffic, which strengthens security and simplifies troubleshooting. It improves resilience too, since it sits alongside existing transit connections and creates additional paths through the network. And from a cost perspective, it can significantly reduce reliance on bandwidth-based transit pricing models. When you put that into the context of AI, especially in hybrid and multi-cloud environments, the value begins to take shape. Data, models, and compute resources rarely sit in one place anymore – they’re spread across clouds, data centers, and edge locations. During the webinar, we had one question about moving datasets from Azure to GPU resources, and the answer came right back to direct connectivity. With access to cloud on-ramps and interconnection platforms like DE-CIX, organizations can move data more efficiently between environments and avoid unnecessary detours. That’s increasingly important as AI architectures become more distributed and more dependent on consistent, predictable performance.
AI’s next bottleneck won’t be compute
What came through clearly in the discussion is that AI adoption isn’t slowing down, but infrastructure readiness is still uneven. Hybrid environments are now the norm, performance challenges are becoming more visible, and concerns around latency and bandwidth are growing as workloads evolve. At the same time, there’s still a noticeable gap in how organizations approach connectivity. Some are already using peering to improve performance and control, while others are still relying heavily on traditional models that weren’t designed with these kinds of workloads in mind. When nearly 80% of organizations don’t fully trust their infrastructure to support AI demands over the next few years, it raises a bigger question about how prepared the industry really is for what comes next.
Looking ahead, the limiting factor for AI won’t just be access to compute or the sophistication of models. It will come down to how efficiently data can move between them. As AI becomes more distributed, spanning clouds, data centers, and edge environments, the network becomes part of the application itself. Organizations that recognize this new environment and invest accordingly will be better positioned to scale, adapt, and deliver consistent performance. Those that don’t may find that even the most advanced AI strategies are held back by the invisible infrastructure underneath them.