Featured in this episode of Tech News of the Week
NVIDIA has kind of a hammerlock on the AI supercomputer hardware space, with their Hundred-X-Thousand dollar H100 GPU. That setup is totally old news now, with the announcement of the H200. The new GPU, combined with HBM3e memory, adds up to at least a 2x inference speed on the llama 2 LLM.
The performance numbers that are associated with these boxes are staggering. I mean, if things like 141GB of memory and 4.8TB of memory bandwidth mean anything to you. So that’s cool. But NVIDIA also released the GH200 “superchip” which is intended for AI AND HPC. The GH is intended to be used in research based supercomputers such as HPE’s Cray supercomputers (yes those are still a thing in name at least) that will scale up to “tens of thousands” of superchip nodes.
2024 will see the beginning of installation on the JUPITER supercomputer in Germany which will use nearly 24,000 of these chips. And Infiniband! Infiniband is still a thing! Neat. The goal of GH200’s in JUPITER is “scientific breakthroughs” around climate in particular, which is different than the regular H200’s which will be used for cynical money grabs by AI-based side hustle grifters. So, like, potato-potato.