A look behind the scenes: How DE-CIX keeps data flowing
How do we keep the platforms running and ready for the future, while also making sure our customers have access to new services? There is a lot of work happening behind the scenes: from product development – like the new Cloud ROUTER – to infrastructure projects such as the EVPN rollout, new routers, and introduction of 100G LR.
Here we shed some light on what happens behind the scenes and how the latest-generation hardware helps us keep the platforms running and offer our customers reliable peering and interconnection services.
New interconnection services vs. platform maintenance
We need to ensure that, on the physical platform side, we meet customer demand in terms of data traffic. And new hardware enables higher data transmission rates. Previously, the demand was always ahead of what hardware suppliers could deliver, but this has changed in the last five years with the introduction of Nokia's new FP5 network processors for example, designed to support 800 Gigabit Ethernet (GE).
At the moment, one big project is the decommissioning of the XRS-40 routers which we introduced back in 2013 when the Apollon platform was born. These big wardrobe-sized routers can process 64 terabits per second (Tbps) of data. The new devices manage 216 Tbps – more than three times the capacity of the older hardware. We are also currently introducing the new 100G LR optics, which can transfer 100 gigabits per second (Gbps) of data via a single laser beam.
It is also important to offer new connectivity services to our customers so that they can benefit from the latest technology. Many of our interconnection services are virtual – existing as configurations on the routers. As the services are separated from the physical platform, we can quickly roll them out to our platforms in different locations.
We have a dedicated research and product management team responsible of further developing existing services and introducing new ones. Our newest product, the Cloud ROUTER, was launched at the beginning of the year.
Always maintaining free capacity for traffic growth
At big DE-CIX hubs like Frankfurt, the traffic grows 20 to 30% per year. Newer locations, such as Madrid, have higher growth figures, sometimes over 100% per year. And the lockdown during the pandemic made the traffic jump 30% in one week.
We can handle this growth by always maintaining appropriate redundancies and capacities in our network. The capacities are expanded as soon as 63.5% of the existing capacity has been reached. The remaining 36.5% free capacity is needed to create redundancy, and to ensure that we always have enough free capacity for traffic growth.
This needs to be planned properly – we always plan around 12 months in advance – as it takes about three months for the hardware to get to us when we order it. And then we need to install it, test it, and put it into operation.
Minimal downtime for customers
When we change something on the network – like replacing a router – our main priority is that the downtime for the customers is minimal. We carry out very specific risk management to give our team the opportunity to prepare for all eventualities. Then, should an incident arise, we know exactly what to do.
We swap routers at night when the customers are least affected. This includes reconnecting hundreds, sometimes thousands, of fiber optic cables, and takes time. We have up to three maintenance windows per week where we move between 30 and 120 of the fiber optic cables.
You can never dial the risk down to zero, but it really is a very rare exception that we cannot restore a service to a customer. For example, a cable might not work even though we had tested it beforehand, so we make sure we can put it back the way it was before. And we of course always have spare ones at hand. During the EVPN project, we prepared rollback scenarios. If the new configuration didn’t work, we could have restored the old configuration within a few seconds and restored the old state.
When it comes to our infrastructure, our main concern – and the need that always comes first – is that the interconnection services are available for our customers.