AI is on the brink of its biggest shift yet: from training models to delivering real-time intelligence everywhere. As inference demand continues to explode, neoclouds have a rare opportunity – if they rethink connectivity, embrace edge infrastructure, and get closer to users, Dr. Thomas King, CTO of DE-CIX, shows how they can seize the moment and shape the AI landscape of 2027 and beyond.
Believe it or not, today only 16% of people around the world use AI, according to Microsoft’s 2025 AI Diffusion Report. Of course, that’s a global number, and in some countries, the percentage is much higher. The growth in usage over the past three years has been significant. A research report into AI usage across 14 OECD countries found that an average of almost 40% of those populations were active users of gen AI in late 2025, with adoption highest among 18-35 years olds. But this growth will pale in significance compared to the wave of AI inference that will crash onto our shores in 2027.
Before long, every device and every application will be enhanced with AI. Meaning not only the devices used by every man, woman and child – their laptops, tablets, smartphones, and wearables – but also the vast ecosystem of IoT, from the smart home to the dark factory, to autonomous vehicles, logistics, and agricultural and healthcare use cases. With Morgan Stanley Research estimating US $2.9 trillion in global CAPEX on data centers between 2025 and 2028, the current construction boom is clearly showing us the trajectory of AI’s future reach.
And while much of the hype until now has been centered around building hyperscalers for AI training, the coming wave of AI in applications and in the real world will be AI inference. McKinsey forecasts that inference will be the dominant workload in AI data centers by 2030, while Precedence Research forecasts that the global market size of AI Inference as a Service will grow at more than 25% CAGR to a volume of close to US $200 billion by 2035.
A game-changer for neoclouds
This new wave of AI inference changes the game for GPU as a Service providers like neoclouds. While AI training can be centralized in remote locations where power and land are plentiful and data can be trucked in, inference requires real-time interaction with end users, be they devices, individual end users, or enterprise customers. Proximity and low latency will become paramount so that inference can create the expected user experience.
We have previously seen a similar demand for low latency networks with real-time video content, and we know that scaled content distribution using edge computing provides the answer. So, it’s clear that edge computing will become vital to achieving the performance of inference at scale. And connectivity will again be the enabler. Content delivery networks (CDNs) – like Akamai, CloudFlare, Fastly, and G-Core Labs – will enter the inference market, applying the lessons they have learned in content distribution. Some already have, and are providing additional competition for neoclouds, on top of the hyperscalers.
For neoclouds to ensure that they can get a piece of the action in the coming inference wave, they need to start thinking like network operators.
Until now, connectivity has played a subordinate role in neocloud architecture. Omdia Informa TechTarget research shows that more than 50% of neoclouds do not yet use peering exchanges and 20% still rely on a single transit provider, risking a single point of failure. The Omdia analysis shows that one of the few aggregation points for neoclouds so far is DE-CIX Frankfurt, the largest Internet, Cloud, and AI Exchange in Europe and one of the largest in the world – which alone carries 10% of the world’s neocloud IX port capacity.
While the major hyperscalers and CDNs all peer at IXs around the globe, only a handful of neoclouds do: providers like CoreWeave and Nebius, for example, are available across multiple locations in Europe and the US on the DE-CIX platform. For the others, their connectivity choices pose the risk of their networks becoming the bottleneck to performance, efficiency, and revenue growth as the inference wave builds.
The peering imperative for neoclouds
Why is peering an important part of the neocloud future? Connectivity using IXs enables neoclouds to find the networks they need for their own AI inference service provision: not only the sources of essential customer data (often stored in the hyperscalers), but also access to individuals and corporate users. Because strong IXs have the cloud players and eyeball networks that neoclouds need. In addition, by building their own on-ramps to Cloud and AI Exchange platforms, they make it easier for their enterprise customers to connect to them securely and privately.
Direct connectivity to relevant networks – instead of relying on IP transit – significantly reduces the latency between data sources, AI models, and users. Connecting to exchanges like DE-CIX’s helps neoclouds to control their network environment – they can see how and where their traffic is flowing and take the shortest routes. They can ensure compliance with regional regulation, maintain sovereignty over their data, and ensure reliable, predictable, and low-latency data flows, rather than depending on an opaque, “best-effort” Internet experience.
The power of low latency
Network latency is important for a variety of reasons when it comes to AI. Firstly, high latency to the AI infrastructure implies that GPUs are lying idle while they wait – whether they are waiting for a user prompt or for relevant data. This leads to underutilization and cost inefficiency. Added to this, latency compounds on itself. Additional milliseconds quickly add up if the network connectivity is not optimized. What’s more, the adoption of agentic AI by organizations is placing further pressure on existing architecture. With agents autonomously accessing data, consulting models, and interacting with other agents at machine-speeds, the demand for both ultra-low-latency and software defined networks increase massively.
Using local IXs, neoclouds can dramatically reduce their end-to-end latency. Let me give you an example: Runloop gives us a typical network latency to AI clusters of 50-200 milliseconds (ms). But for inference, you don’t need a cluster, you need an edge facility in each city or region where the users are located. Using the DE-CIX IXs, latency can be brought down to less than one millisecond within a metro area. Take DE-CIX New York, for instance, the largest IX in the Northeast of the US and the 3rd largest in North America. Its geographical scope covers not only NYC, but extends further into Long Island, Brooklyn, and New Jersey, offering <1 ms connectivity throughout this whole region. So, a neocloud can have its inference GPUs set up in a colocation facility anywhere within this metro area and serve the entire population of the region lightning fast – at least, as fast as their own infrastructure can generate tokens. Network latency to the user ceases to be a concern.
Why peering is becoming a technological imperative.
NVIDIA tells us that “the core challenge of AI inference is balancing latency, cost, and throughput,” and that “by favoring one, you may need to trade off maximum value in another.” Using IXs takes care of a portion of the latency involved, reducing its impact on the equation. By bringing down the Time to First Token (TTFT) and the end-to-end latency to AI compute, providers can increase the number of tokens generated per second. And with Gartner predicting a 90% drop in the costs associated with the operation of LLMs by 2030, neoclouds can ensure that they don’t just catch the wave of AI inference, but that they can ride it with confidence. Peering is thus becoming a technological imperative.
But there are other reasons to take control of your connectivity. One of these is protecting the network against threats like Distributed Denial of Service (DDoS) attacks. With IP transit as the sole connectivity infrastructure, neoclouds are at the mercy of DDoS attacks, which will take out the entire transit line, leaving GPUs idling and customers frustrated. With peering, networks can protect themselves against DDoS attacks. If one of the connected networks suffers an attack, it can be isolated and contained, protecting connectivity to all other networks and keeping GPUs in operation.
Making waves with edge infrastructure
Therefore, Neoclouds need to become well peered and use edge infrastructure in many locations – not just in network-dense locations like Frankfurt, London, or New York, but everywhere where their customers are operating. From Barcelona (Spain) to Rio de Janeiro (Brazil), from Dubai (UAE) to Dallas (US), from Bangalore (India) to Stockholm (Sweden), it’s a simple formula: the lower the network latency, the more neoclouds can earn, and the more disruptive their impact on the market will be.
Internet Exchange operators like DE-CIX – which is evolving into a Network as a Service provider for Cloud and AI connectivity – provide the answer for how neoclouds can evolve their networks and business cases to make waves in the market. According to JLL, inference will overtake training workloads long before 2030. By 2027, the data center specialist anticipates that inference workloads will have already caught up, and by 2030 nearly three quarters (72%) of all AI workloads will be inference based. And the market is big enough for hyperscalers, CDNs and neoclouds all to capture their share: ABI Research anticipates that, by 2030, neoclouds can triple their revenues from inference, alone accounting for US $150 billion annually – but that will only work if they get their connectivity right.
