AMD Reveals Fleet of Chips for Heavy AI Workloads

AMD announced the upcoming release of its most powerful AI chips to date, the Instinct MI325X accelerators, on Thursday.

“Our goal is to drive an open industry standard AI ecosystem so that everyone can add their innovation on top,” said Lisa Su, AMD chair and CEO, at the company’s Advancing AI 2024 presentation in San Francisco.

The 5th generation Epyc processor positions AMD as an underdog contender to NVIDIA’s Blackwell in the AI market. During the same presentation, AMD also unveiled several novel products, including a new server CPU designed for enterprise, AI, and cloud applications.

AMD Instinct MI325X accelerators add capacity to AI infrastructure

AMD Instinct MI325X accelerators speed up foundation model training, fine-tuning, and inferencing — the processes involved in today’s rapidly-proliferating generative AI — and feature 256GB of HBM3E supporting 6.0TB/s. AMD’s CDNA 4 architecture enables the new line.

The capacity and bandwidth of these accelerators out-perform the major competitor, the NVIDIA H200, AMD claims. The tech company also says that the Instinct MI325X accelerators can hasten inference performance on the Mistral 7B AI by 1.3x, on Llama 3.1 70B by 1.2x, and on Mistra’s Mixtral 8x7B by 1.4X when compared with the H200.

AMD primarily targets hyperscalers with this product. In particular, hyperscalers want to expand their AI-capable hardware in data centers and power heavy-duty cloud infrastructure.

The Instinct MI325X is scheduled to go on sale in the last quarter of 2024. In the first quarter of 2025, they’ll appear in devices from Dell Technologies, Eviden, Gigabyte, Hewlett Packard Enterprise, Lenovo, and Supermicro. Following that, AMD will continue to expand its MI350 series, with 288GB Instinct MI350 series accelerators expected in the second half of 2025.

The 5th Gen AMD Epyc server CPU includes up to 192 cores

The latest generation of AMD’s Epyc processors, code-named “Turin,” also debuted in San Francisco, featuring Its Zen 2 Core architecture. AMD Epyc 9005 Series processors come in myriad configurations — with core counts from eight to 192 — and speed up GPU processing for AI workloads. AMD’s main competitor in this area is Intel’s Xeon 8592+ CPU-based servers.

The performance density is a key advantage, AMD said. Higher-capacity GPUs make it possible to use an estimated 71% less power and about 87% fewer servers in a data center, the company said. AMD provides a disclaimer noting that environmental factors involve many assumptions if not applied to a specific use case and location.

SEE: Security researchers found some fraudsters profit with the help of AI-generated video that can trick facial recognition software.

All Epyc 9005 Series processors were released on Thursday. Cisco, Dell, Hewlett Packard Enterprise, Lenovo, Supermicro, and major ODMs and cloud service providers support the new line of chips.

“With the new AMD Instinct accelerators, EPYC processors and AMD Pensando networking engines, the continued growth of our open software ecosystem, and the ability to tie this all together into optimized AI infrastructure, AMD underscores the critical expertise to build and deploy world class AI solutions,” said Forrest Norrod, executive vice president and general manager, Data Center Solutions Business Group, AMD, in a press release.

Two new products cover front- and back-end tech for AI networking

For AI networking in hyperscale environments, AMD developed the Pensando Salina DPU (front end) and the Pensando Pollara 400 NIC (back end). The former handles data transfer, delivering data to an AI cluster securely and at speed. The latter, a NIC or network interface card, manages data transfer between accelerators and clusters using a Ultra Ethernet Consortium-approved design. It is the industry’s first AI NIC to do so, AMD said. The DPU supports 400G throughput.

The broader goal of this technology is to enable more organizations to run generative AI on devices, in data centers, or in the cloud.

Both the AMD Pensando Salina DPU and AMD Pensando Pollara 400 NIC will be generally available in the first half of 2025, AMD expects.

Coming soon: The Ryzen Pro 300 Series laptops for commercial use

OEMs will begin shipping laptops with AMD’s Ryzen Pro 300 series processors later in 2024. First revealed in June, the Ryzen Pro 300 series is a key component of AI PCs. In particular, they help Microsoft’s effort to put Copilot+ AI features forward in its current and upcoming commercial devices.

“Microsoft’s partnership with AMD and the integration of Ryzen AI PRO processors into Copilot+ PCs demonstrate our joint focus on delivering impactful AI-driven experiences for our customers,” said Pavan Davuluri, corporate vice president, Windows+ Devices, Microsoft, in a press release.

Lenovo built its ThinkPad T14s Gen 6 AMD around the Ryzen AI PRO 300 Series processors. Luca Rossi, president, Lenovo Intelligent Devices Group, talked up the chips in the press release, saying, “This device offers outstanding AI computing power, enhanced security, and exceptional battery life, providing professionals with the tools they need to maximize productivity and efficiency.”

TechRepublic covered AMD’s Advancing AI event remotely.