The AMD MI350 series details are announced, with 35 times more efficient than previous generations

AMD revealed the latest Instinct MI350 GPU series at Hot Chips 2025, equipped with the new CDNA 4 architecture, which targets generative AI and HPC.

With the rapid expansion of large language models (LLMs) scales, GPUs require larger memory and faster frequency widths, and the MI350 is born to understand this problem. The MI350 uses t-electric 3nm (N3P) and 6nm (N6) FinFET processes, and uses COWOS-S advanced packaging technology, and is combined with 3D multi-grain packaging to combine high-performance cores with cost-effective I/O grains.

(Source: AMD, the same below)

HBM capacity is 288GB, and the inference and training efficiency are comprehensively improved

The memory part is the biggest highlight, with 8 HBM3E stacks, each with up to 12 layers (12-Hi), 36GB per unit, 288GB total capacity, about 1.5 times more than the NVIDIA B200, and up to 8TB/s in frequency, which can provide excellent data processing performance whether it is reasoning or training.

FP8 computing power has broken 80 PFLOPs, and the inference efficiency has increased by 35 times.

In terms of computing performance, AMD pointed out that the MI355X under the MI350 series supports FP8 computing about 80 PFLOPs, FP16/BF16 can also reach 40PFLOPs, and new low-precision formats of MXFP6 and MXFP4 are added, which specifically optimizes LLM inference. The chip integrates 256 MB Infinity Cache, plus 5.5TB/s Infinity Fabric dual-directional bandwidth, achieving high-speed interconnection. The overall performance is comparable to that of the previous generation MI300, and the reasoning ability is up to 35 times higher.

Air-cooled vs. Liquid cooling

As for the heat dissipation part, the MI350 series offers two configuration versions, among which the MI350X uses an air-cooling design, with a power consumption of 1000W, and a maximum time of 2.2GHz; while the MI355X uses a liquid cooling solution, with a power consumption increased to 1400W, and a time of 2.4GHz, which is mainly a large entry data center market.

AMD predicts that the MI350 series will enter the market through partners in the third quarter of 2025. The next generation MI400 series is under development and plans to be launched in 2026.

AMD’s Instinct MI350 GPU Is A AI-Hardware Powerhouse: 3nm 3D Chiplet Based on CDNA 4, 185 Billion Transistors, 1400W TBP, Over 4000B LLM Support With Massive 288GB Memory