vLLM: DeepSeek‑V4.1‑Flash on Vera Rubin NVL72 gets 58× GB300 throughput per GPU at 387 tokens/s per user

Corrected Wednesday, 18 November 2026: The 58× figure is throughput per GPU at 387 tokens/s per user, not the speed each user gets; added the benchmark, the P90 and batch size of the single‑user figure, that the HBM pool is experimental, and that the rack was early access hardware.

vLLM reports that on one early access NVIDIA Vera Rubin NVL72 rack, DeepSeek‑V4.1‑Flash serves 1.7M tokens/s per GPU while each user gets 387 tokens/s, 58× the throughput of GB300 NVL72 at that speed.

Who it affects Teams running or planning to run DeepSeek‑V4.1‑Flash with vLLM on NVIDIA Vera Rubin NVL72 racks, or using vLLM with Mooncake KV and Dynamo routing, can use these optimizations to increase agentic serving throughput and lower latency.

Source: vllm.ai ·