Featured in this episode of Tech News of the Week
Courtesy of The Register comes a recent white paper from French multinational company Schneider Electric, no relation to multinational ass-hat Rob Schneider. The paper takes a closer look at the coming power demands for AI workloads, and boy do they love them some energy.
We’ve been through this before with cryptocurrency, which also ate up watts on massive rigs full of GPUs for highly dubious purposes. The GPUs remain as does the dubiousness of said workloads. But I digress.
AI workloads generally need two things, power-hungry GPUs to train models and high-speed networks to coordinate training runs between said GPUs. And when I say high-speed, I’m talking greater than 900GB/s. In fact, the goal is to keep GPUs as close together as possible with minimum latency, which leads to extremely high rack density.
How dense? Consider the average rack runs 10-20kW at the high end. These servers are running 10kW per server.
Schneider posits that power delivery to the racks will need to be scaled up accordingly, and overall power requirements of data centers will spike as well. What about all the heat built up by these GPUs? Liquid cooling is the natural answer, which is a nightmare to set up and maintain, but with all the money being poured into AI right now, where there’s a billion, there’s a way.