The effectiveness of NVIDIA’s new products in the Green500 supercomputer rating was called into question due to AMD chips and… NVIDIA itself

Although NVIDIA accelerators are considered to be among the most power-hungry in their class, supercomputers based on the company’s chips still dominate the world’s Green500 ranking of the energy efficiency of their respective machines. However, the company faces strong competition from AMD and is not always ready to compete even with its own products, The Register reports.

At first glance, the leadership of NVIDIA-based projects is undeniable. Eight out of ten supercomputers included in the “Top 10” energy-efficient machines are built on NVIDIA chips, five of them on 1000-watt GH200 hybrid superaccelerators, which are very popular among users of HPC solutions.

In the latest Green500 rating, the first and second most energy efficient systems are built on their basis – JEDI (EuroHPC) and Romeo-2025 (Romeo HPC Center). In the High-Performance Linpack benchmark, they demonstrated performance of 72.7 Gflops/W and 70.9 Gflops/W, respectively (FP64).

Image source: Jakub Żerdzicki/unsplash.com

The systems are almost identical and are built on Eviden’s BullSequana XH3000 platform. The GH200 solution also accounts for the fourth, sixth and seventh positions in the ranking: Isambard-AI Phase 1 (68.8 Gflops/W), Jupiter Exascale Transition Instrument (67.9 Gflops/W) and Helios (66.9 Gflops/W). Systems with tested NVIDIA H100 take fifth, eighth and ninth places – these are Capella, Henri and HoreKa-Teal.

Nevertheless, there are doubts that NVIDIA solutions will continue to reign supreme in the rankings. Grace-Blackwell solutions are already on the way in the form of GB200 (2.7 kW) and GB200 NVL4 (5.4 kW). New products do not always provide maximum performance per watt of energy.

From the A100 in 2020 to the H100 in 2022, performance (FP64) has skyrocketed by about 3.5x, but compared to Blackwell’s 1.2kW platform, the 700W H100 is actually faster in FP64 mode. In fact, only vector math improved for FP64, where the new products were 32% more productive.

In other words, although NVIDIA today boasts a high position in the Green500 rating, the solution based on AMD MI300A accelerators has already taken third place (Adastra 2). MI300A was announced a little less than a year ago, the solution received a 24-core CPU and six CDNA-3 chiplets in a single APU module equipped with up to 128 GB of HBM3 memory, as well as a customizable TDP level of 550-760 W. Moreover, the system is 1.8 times faster than the NVIDIA H100 (at least on paper).

Built by HPE Cray using EX255a blade servers, the Adastra 2 supercomputer delivers 69 Gflops/W of performance. Tenth place is also occupied by a machine based on MI300A – RZAdams from the Livermore National Laboratory (62.8 Gflops/W).

All systems in the top ten of the Green500 rating already significantly exceed the energy efficiency target of 50 Gflops/W. This figure is necessary to achieve exascale computing while limiting power consumption to 20 megawatts.

The problem is that less powerful systems are significantly more efficient: JEDI consumes only 67 kW, and the highest performing GH200 machine in the Top500 ranking – Alps of the Swiss National Supercomputing Center – provides 434 Pflops in the HPL benchmark, consuming 7.1 MW – that’s only 14- I am one of the most energy efficient machines with 61 Gflops/W.

The same problem with Adastra 2: the computer consumes even less than JEDI – 37 kW. If 69 Gflops/W could be maintained at scale, it would only take 25.2 MW to deliver El Capitan’s 1,742 eflops of performance. Meanwhile, the latter requires about 29.6 MW to achieve its record figures.