Data Center Acceleration

Do you remember the data center of the past? And by past, I mean 20 to 25 years ago, when there was this huge, almost philosophical debate about complex instruction set computer (CISC) and reduced instruction set computer (RISC) architectures, and between large symmetric multi-processing (SMP) servers and mainframes and smaller systems. There were even fights over some esoteric system designs. All of this was happening before there were co-processors, ASICs and other fancy accelerators to speed data access and optimize complex operations.

You might think we are past the fighting, now that data centers have largely aligned around commoditized x86 (ahem, CISC) CPUs, small two-socket servers, and a general standardization of the components that make up the modern data center. But the reality is that an increasing number of companies are rethinking the data center in ways that remind me of the ideological tussles of the past, while introducing new paradigms and innovations based on recent technology advancements.

The Limits of Ordinary Data Centers

Intel CPUs today are amazingly powerful. They can boast up to 112 cores and an incredible number of instructions and features to manage every type of workload—the latest Intel CPUs can handle specialized machine learning activities with aplomb. But there’s a catch, and the entire industry is working to find solutions.

When you look at current x86-based server designs, the first thing that comes to my mind is “Jack of all trades, master of none.” These servers offer a balanced approach that works well for most applications but simply aren’t designed for the specialized workloads that are emerging. Big data analytics, machine learning/artificial intelligence (ML/AI), Internet of Things, and other high-demand workloads are changing the shape and focus of data centers. For some enterprises, these specialized workloads are already more important than the workaday business applications that most x86-based servers were designed to address.

Yes, many companies are running most of these new applications in the cloud, but the fundamental concept remains. Cloud providers long ago changed the way they think about their server architectures. Isn’t it time you did, too?

CPUs and GPUs and Accelerators, Oh My!

As we think about cost, power, efficiency, and optimization in a modern data center, we immediately find that traditional x86 architectures don’t work anymore. Don’t believe me? Examples are everywhere.

Consider ARM CPUs (RISC!), which are less powerful on a single core basis than their x86 counterparts, but consume a fraction of the energy and can be more densely packed in the same rack space. When you consider that most modern applications are highly parallelized and organized as microservices, suddenly ARM becomes a very attractive option. No, you won’t run SAP on it, but ARM servers can run practically everything else. Good examples of this type of server design can be found from Bamboo Systems or with Amazon Graviton instances.

At the same time, single-core CPU performance is becoming less relevant now that GPUs have been deployed for specialized tasks. GPU-enabled platforms have prompted a rebalancing of system designs, addressing the uniquely data-hungry nature of these processors.

In addition to new network interfaces, we have seen developed new and efficient protocols to access data, such as NVMe-oF.

The problem is that the overhead required to make network communications secure and efficient can easily clog a CPU. For this reason, we are seeing a new generation of network accelerators that offload demanding tasks from the CPU. Examples of these implementations include Pensando, which offers impressive performance without impacting CPU workload and optimizes the movement, compression, and encryption of large amounts of data. Here is an introduction to Pensando from a recent Cloud Field Day. And again, major cloud providers are implementing similar solutions in their data centers.

This story is not fully told. Storage is following a similar trend. NVMe-oF has simplified, parallelized, and reduced the data path, improving overall latency, while data protection, encryption, compression, and other operations are offloaded to storage controllers designed to build virtual arrays distributed across multiple servers without impacting CPU or memory. Nebulon offers an example of this approach, which I am scheduled to present at Storage Field Day 20. Another example is Diamanti with their HCI solution for Kubernetes which leverages accelerators for both storage and networking.

Closing the Circle

Is this a radical approach to data center design? Large cloud providers have been remaking their data centers for years now, and large enterprises are starting to follow suit. The fact is, if you are using the cloud for your IT, you are already engaging these new data center models in one way or the other.

I’ve written before on this topic, especially about ARM CPUs and their role in the data center. This time is different. The software is mature, the ecosystem of off-the-shelf solutions is growing, and everybody is looking for ways to make IT more cost-conscious. Are you ready?

from Gigaom https://gigaom.com/2020/07/16/data-center-acceleration/