Terabyte GPUs: Panmnesia demonstrated CXL memory for AI accelerators

Panmnesia has been designing CXL DRAM pools for quite some time: in 2023, it demonstrated a system that leaves behind all RDMA-based solutions and provides access to 6 TB of RAM. But large amounts of memory today, in an era of increasingly complex AI models, are needed not only and not so much by processors, but by accelerators, which are a priori deprived of the ability to upgrade on-board RAM. At CES 2025, the company demonstrated a solution to this problem.

According to Panmnesia developers, performance when training large-scale AI models depends precisely on the volume of on-board memory of the accelerators: instead of tens of gigabytes, terabytes are required, and installing additional accelerators can be too expensive, given that the computing power will be redundant.

Source here and below: Panmnesia

The CXL system demonstrated at the exhibition is built on the latest Panmnesia controller with support for CXL 3.1. In bidirectional mode, access latency was less than 100 ns and is approximately 80 ns.

The key to success here lies in the proprietary implementation of CXL 3.1, including the software part, thanks to which GPUs can access a shared memory pool using the same load/store instructions as when accessing on-board HBM or GDDR.

However, the technology requires a proprietary CXL Root Complex controller on board the GPU, one of the most important parts of which is the HDM decoder, which is responsible for managing the memory address space (host physical address, HPA), so already released accelerators will not be able to work directly with the Panmnesia system.

However, the technology looks promising. It has already attracted attention from AI companies as a potential way to reduce the cost of data center infrastructure.

admin

Share
Published by
admin

Recent Posts

Telegram now offers group calls for 100 people with end-to-end encryption — in test mode for now

The beta version of Telegram for Android (11.9.1) has a group calling feature that allows…

3 minutes ago

Newly awakened black hole reveals ‘wild’ appetite that stuns scientists

A supermassive black hole in the galaxy SDSS1335+0728, located 300 million light years from Earth,…

1 hour ago

The developer of laser-cooled chips made a lot of promises and is now looking for help from scientists

Details about the technology of the young company Maxwell Labs, which proposes to cool chips…

7 hours ago

Asus Unveils Budget X870 MAX Gaming WiFi7 Motherboard for Ryzen 9000

Asus has introduced the X870 MAX Gaming WiFi7 motherboard. The new product is designed for…

7 hours ago

Asus Unveils World’s First 610Hz Gaming Monitor — ROG XG248Q5G-P for $1,100

Asus has introduced two gaming monitors — ROG XG248Q5G-P and ROG XG32UCG. The first is…

7 hours ago