Microsoft introduced the AMD Instinct MI300C accelerator – a special version of Epyc with 128 GB HBM3 memory

At the Ignite 2024 conference, Microsoft Azure presented the first processor of the Epyc family, equipped with Zen 4 cores and HBM3 memory – the product was designated AMD Instinct MI300C, and at the moment it is available only in the Microsoft cloud infrastructure, namely in Azure HBv5 instances.

Image source: microsoft.com

Last Sunday, when the head of AMD’s data center division attended the presentation of the fastest supercomputer, El Capitan, the company mentioned the MI300C as one of the models in the Instinct line, but did not provide details then. The AMD Instinct MI300 family of accelerators is designed for the high-performance computing and artificial intelligence sectors.

They have a chiplet layout, the basis of which is four adjacent I/O Dies, on which GPU/XCD or central processor chiplets are placed. If the MI300A accelerator configuration includes six XCDs and three eight-core CCDs, then the MI300C has XCD units replaced with CCDs – accordingly, the total number of Zen 4 cores is not 24, but 96. There are practically no other differences; There is an HBM3 memory unit with a capacity of 128 GB. Essentially, it turns out that the AMD MI300C accelerator is an Epyc processor with HBM3.

In practice, the amount of available processor resources depends on its usage scenarios. In the Microsoft Azure HBv5 virtual machine, they work in four units, with only 88 cores available for each processor; Four processors installed on the board with a maximum clock frequency of 4 GHz provide 352 cores. Memory bandwidth is 6.9 TB/s with a capacity of 400–450 GB (HBM3), you can install up to 9 GB of memory per core. SMT (Simultaneous Multithreading) feature is disabled; There is only one virtual machine per server.

Nvidia Quantum-2 InfiniBand interconnect with a speed of 800 Gbit/s is supported – 200 Gbit/s per accelerator; VMSS Flex allows you to scale MPI workloads to hundreds of thousands of processor cores with access to HBM memory; There is an Azure Boost 2 network adapter that provides network speeds of up to 160 Gbps. Customers are also offered 14 TB of local NVMe SSD storage with speeds of up to 50 GB/s and write speeds of up to 30 GB/s.

It is emphasized that AMD Instinct MI300C will remain available exclusively in Microsoft Azure for the foreseeable future.