NVIDIA publishes first inference results for its Vera Rubin rack

NVIDIA has published preview results for its new Vera Rubin NVL72 rack as part of the MLPerf Inference v6.1 round released today. The company picked two of the suite's most demanding workloads: DeepSeek-R1 and Qwen3-VL.

On Qwen3-VL, NVIDIA reports up to 3.7x higher throughput than GB300 NVL72 across the offline, server and interactive scenarios, using vLLM with the open source NVIDIA Dynamo inference framework. On DeepSeek-R1, running the TensorRT-LLM library, throughput is up to 2.5x higher. According to the company, each Vera Rubin rack therefore produces more tokens and serves more users while lowering cost per token.

Where the gains come from

NVIDIA attributes the results to hardware and software codesign. Vera Rubin's enhanced Tensor Cores and Transformer Engine speed up both the prefill and decode stages, while NVFP4 precision shrinks the memory footprint of model weights, attention and the KV cache. The submissions leaned on disaggregated serving, splitting prefill from decode, plus large-scale expert parallelism for mixture-of-experts layers. Inside the rack, sixth-generation NVLink and NVLink Switch provide what NVIDIA describes as 10x higher packet rates and 3x lower latency than off-the-shelf Ethernet.

On scaling, the DeepSeek-R1 submission went from a single GB300 NVL72 rack (72 GPUs) to four racks (288 GPUs) at 99% scaling efficiency in the offline scenario. GB300 NVL72 also hit 0.65 720p videos per second on the WAN 2.2 text-to-video benchmark, at 5.7 seconds per video.

Software work shows up too: GB300 NVL72 improved up to 1.6x on Qwen3-VL compared with v6.0. NVIDIA also submitted Jetson AGX Thor results on the new Edge-Agentic benchmark with Qwen3.6-27B. Nineteen partners took part, eight of them on multi-node Blackwell NVL72 systems, and Nebius submitted Vera Rubin NVL72 preview results as well.

Full details are in the NVIDIA blog post.