MLPerf Inference v6.1: AMD’s 512-GPU MI355X Cluster Sets Scale Record

AI inference benchmarking just got a clearer scoreboard. On September 16, 2026, MLCommons published MLPerf Inference v6.1—a participation record with 30 organizations and hundreds of datacenter and edge results, plus new RAG and edge-agent tests. Independent synthesis from topcpu highlights the headline fight: extreme-scale AMD clusters versus NVIDIA’s mainstream Blackwell lead—and the first public Vera Rubin preview numbers.

AMD’s 512-GPU MI355X sets a scale record

AMD submitted a cluster of 512 Instinct MI355X GPUs—the largest system ever entered in MLPerf Inference. At that extreme scale it led aggregate offline/server throughput on large-model tests: roughly 5.75 million tokens/s on GPT-OSS-120B offline and about 2.9 million tokens/s on DeepSeek-R1 offline. At 72 GPUs, AMD cited ~95% scaling efficiency on GPT-OSS-120B.

Blackwell still owns “normal” rack sizes

NVIDIA’s GB300 configurations kept the lead in the configurations most buyers actually compare—8-GPU and 72-GPU groups—including Llama 2 70B. In an 8-GPU Llama 3.1 8B matchup, GB300 was reported up to 17% faster than a same-size MI355X node. A 288-GPU (four NVL72 racks) submission showed ~99% scaling efficiency.

First Vera Rubin preview

v6.1 also includes the first peer-reviewed Vera Rubin VR200 preview on DeepSeek-R1 (72- and 36-GPU). The 72-GPU Rubin config was reported up to 95% faster than same-size GB300; NVIDIA materials have floated up to ~3.7× NVL72 throughput versus GB300 in highlight claims—treat preview category results as directional, not a shipping product SLA.

Software is the quiet multiplier

Vendors stressed stack gains on unchanged silicon: AMD ROCm iteration on MI355X, NVIDIA claiming up to ~1.6× vs v6.0 via software, Intel Arc Pro B70 and Xeon 6 showing double-digit to 2.4× gains from compilers and runtimes. That pattern matters more for IT buyers than a single leaderboard screenshot.

Rocks take

Read MLPerf as a configuration-specific snapshot, not a shopping cart. AMD wins the “how big can we go” story; NVIDIA still dominates the rack sizes enterprises quote; Rubin is a preview signal. Cross-check the official MLCommons release before repeating vendor blog numbers.

How to use these results

Match model (DeepSeek-R1, GPT-OSS-120B, Llama), scenario (offline vs server), and GPU count to your planned deployment. Ignore 512-GPU glory if you buy 8-GPU nodes.

FAQ

When were results published? September 16, 2026 by MLCommons.

Who “won”? Depends on scale: AMD at 512 GPUs; NVIDIA Blackwell in common 8/72-GPU configs; Rubin is preview-only.