vllm-omni/recipes/MiniMaxAI/MiniMax-H3-RTX-PRO-6000.md

MiniMax-H3 on RTX PRO 6000 Blackwell GPUs

This recipe runs MiniMax-H3 in BF16 on 96 GiB RTX PRO 6000 Blackwell GPUs. The additional HBM over the RTX PRO 5000 profile removes the four-GPU minimum: two GPUs are enough to keep the model resident with TP2. Four and eight GPUs raise the Ulysses degree to shard the attention sequence further. CPU offload and distributed layerwise offload are not required in any of these configurations.

Validated on:

• Host: YLX Y762
• GPUs: 8 × RTX PRO 6000 Blackwell (96 GiB)
• Device order: default (CUDA_VISIBLE_DEVICES not set)
• Driver / CUDA: 580.105.08 / 13.0
• Container image: vllm/vllm-omni:minimax-h3
• Workload: T2VA, 1344×768, duration=5.0, 50 steps, flow_shift=12, seed=1101

Capacity requirements

Resource	Two GPUs	Four GPUs	Eight GPUs
GPU HBM	96 GiB per GPU	96 GiB per GPU	96 GiB per GPU
Checkpoint storage	135 GiB per partition	135 GiB per partition	135 GiB per partition
Available system RAM	200 GiB minimum	200 GiB minimum	200 GiB minimum
Recommended system RAM	384 GiB	384 GiB	384 GiB
Measured peak HBM per GPU	77.49 GiB	66.44 GiB	61.07 GiB

FL2VA and Ref2VA are separate checkpoint partitions. Start one server at a time on a host sized for the minimum system-memory requirement, or pass --task-type fl2va or --task-type ref2va to download and load a single partition. The two-server layout described in the eight-GPU section doubles both the storage and the host-memory requirement.

Parallelism summary

Configuration	TP	Ulysses	Text-encoder TP	VAE patch parallel
Two GPUs	2	1	2	2
Four GPUs	2	2	4	4
Eight GPUs	2	4	8	8
Eight GPUs, headroom variant	4	2	8	8

Per-GPU weight residency is set by the tensor-parallel degree alone. Raising the Ulysses degree lowers activation memory and per-step latency but leaves weight residency unchanged. Ring parallelism stays at 1, and H3 is CFG-distilled, so --cfg-parallel-size must also remain 1.

PCIe topology and GPU order

RTX PRO 6000 Blackwell does not provide NVLink. Before starting the server, inspect the host topology:

nvidia-smi topo -m
nvidia-smi nvlink -s

Tensor-parallel groups are consecutive ranks. Ulysses groups are strided by the tensor-parallel degree. Confirm this grouping against the server log before tuning the device order on a new host.

For the two-GPU configuration, pick a single PXB pair on one NUMA node. For the four-GPU configuration, pick two PXB pairs on the same node and order them so that each Ulysses group lands inside one pair. If physical GPUs (0,1) and (2,3) are the two local pairs, the order is CUDA_VISIBLE_DEVICES=0,2,1,3. Do not copy these IDs blindly: reproduce the same relationship on the target host.

Device-order pinning and NUMA binding are tuning knobs, not prerequisites. The measurements in this recipe were taken without either, so any gain from them is additional to the numbers reported below and is host-specific.

Two-GPU serving configuration

Two 96 GiB GPUs hold the BF16 model with TP2 alone. There is no Ulysses group, so the DiT attention sequence is not sharded and activation memory is higher than in the four-GPU profile at the same output shape. Re-measure peak memory before increasing the output shape, the reference-input length, or concurrency.

export MODEL_ROOT=/path/to/MiniMax-H3
export MODEL="${MODEL_ROOT}/FL2VA"
export PORT=8000

VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve "${MODEL}" \
  --omni \
  --host 0.0.0.0 \
  --port "${PORT}" \
  --trust-remote-code \
  --num-gpus 2 \
  --tensor-parallel-size 2 \
  --usp 1 \
  --ring 1 \
  --text-encoder-tp-size 2 \
  --vae-patch-parallel-size 2 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend CUDNN_ATTN

Four-GPU serving configuration

Four GPUs keep TP2 for the weight shard and add Ulysses2, which shards the attention sequence and lowers per-GPU activation memory. Text-encoder TP4 and VAE patch parallelism 4 spread the Qwen3-VL encoder and tiled decode across all four GPUs. TP1 is not an option on this card: an unsharded BF16 partition does not fit in 96 GiB.

export MODEL_ROOT=/path/to/MiniMax-H3
export MODEL="${MODEL_ROOT}/FL2VA"
export PORT=8000

VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve "${MODEL}" \
  --omni \
  --host 0.0.0.0 \
  --port "${PORT}" \
  --trust-remote-code \
  --num-gpus 4 \
  --tensor-parallel-size 2 \
  --usp 2 \
  --ring 1 \
  --text-encoder-tp-size 4 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend CUDNN_ATTN

Eight-GPU serving configuration

Eight GPUs extend the Ulysses degree to 4 while keeping TP2. This does not lower per-GPU weight residency relative to the four-GPU profile; it lowers activation memory and per-step latency by sharding the attention sequence four ways.

export MODEL_ROOT=/path/to/MiniMax-H3
export MODEL="${MODEL_ROOT}/FL2VA"
export PORT=8000

VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve "${MODEL}" \
  --omni \
  --host 0.0.0.0 \
  --port "${PORT}" \
  --trust-remote-code \
  --num-gpus 8 \
  --tensor-parallel-size 2 \
  --usp 4 \
  --ring 1 \
  --text-encoder-tp-size 8 \
  --vae-patch-parallel-size 8 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend CUDNN_ATTN

Rank ordering across two sockets

With TP2 and Ulysses4 the two Ulysses groups are ranks (0,2,4,6) and (1,3,5,7), and the four tensor-parallel pairs are (0,1), (2,3), (4,5), and (6,7). On a dual-socket host these two collectives cannot both be socket-local. If you choose to pin the device order, the trade-off is:

Device order	Socket-local collective	Crosses sockets
0,4,1,5,2,6,3,7	Ulysses all-to-all	TP all-reduce
0,1,2,3,4,5,6,7	TP all-reduce	Ulysses all-to-all

The first order assumes physical GPUs 0-3 are on NUMA node 0 and 4-7 on node 1. Reproduce that relationship on the target host rather than copying the IDs. The reported measurements use the default order and no NUMA binding; the two orders were not compared on this host.

Headroom variant: TP4 with Ulysses2

Raising the tensor-parallel degree to 4 halves per-GPU weight residency and leaves room for long Ref2VA references, larger output shapes, or concurrency greater than one. It also adds a four-way all-reduce per DiT block over a link without NVLink. Use it when the TP2 profile runs out of headroom, not as the default. Replace the parallel flags above with:

  --tensor-parallel-size 4 \
  --usp 2 \

Two independent four-GPU servers

An eight-GPU host can instead run one FL2VA server and one Ref2VA server side by side, each pinned to its own NUMA node with the four-GPU configuration. This removes every cross-socket collective and lifts the one-server-at-a-time restriction, at the cost of doubling the requirements to 400 GiB available system RAM and 270 GiB of model storage.

Use CUDA_VISIBLE_DEVICES=0,2,1,3, --cpunodebind=0 --membind=0, and --port 8091 for the FL2VA server. Use CUDA_VISIBLE_DEVICES=4,6,5,7, --cpunodebind=1 --membind=1, and --port 8092 for the Ref2VA server. This layout is described for completeness and was not part of the measured runs.

Shared serving notes

Do not add --enforce-eager for a performance run. Warm the server before measuring so regional compilation is outside the measured request. On this host the first request after startup ran 19% slower than the steady state (107.58 s versus 90.48 s on eight GPUs); two warmup requests were enough to converge.

For Ref2VA on a single-server layout, stop the FL2VA server and restart the same command with MODEL="${MODEL_ROOT}/Ref2VA".

Attention backend

RTX PRO 6000 Blackwell is an SM120 part, so the datacenter-Blackwell TRTLLM_ATTN default does not apply here. Every configuration above selects cuDNN BF16 attention explicitly. This keeps the recipe independent of platform-default backend changes and does not depend on the experimental SM120 kernel.

Optional: online FP8

--quantization fp8 quantizes eligible DiT linears at load time and is compatible with tensor parallelism and VAE tiling. Use it when the resident BF16 peak leaves too little headroom for long Ref2VA references, a larger output shape, or concurrency greater than one. It cannot be combined with layerwise offload.

Target-hardware validation

T2VA, 1344×768, duration=5.0, fps=24, 50 steps, flow_shift=12, seed=1101, BF16, CUDNN_ATTN, tiled VAE, one request at a time. Servers were started with the commands above — default device order, no NUMA binding — plus --enable-diffusion-pipeline-profiler. Two warmup requests preceded each measured request.

Measurement	Two GPUs	Four GPUs	Eight GPUs
Topology	TP2 × USP1	TP2 × USP2	TP2 × USP4
Text encode	0.040 s	0.044 s	0.035 s
Denoise, 50 steps	278.55 s	168.73 s	87.90 s
Per step	5.571 s	3.375 s	1.758 s
VAE decode	5.396 s	2.791 s	1.798 s
Client E2E	284.76 s	172.32 s	90.48 s
Peak HBM per GPU	77.49 GiB	66.44 GiB	61.07 GiB
Headroom below 96 GiB	18.5 GiB	29.6 GiB	34.9 GiB

Stage times are read from the X-Stage-Durations response header of /v1/videos/sync. Peak memory is the maximum of nvidia-smi --query-gpu=memory.used sampled at 1 Hz across every device for the duration of the measured request. Per step is denoise wall time divided by the 50 requested steps. Stage times sum to roughly 0.75 s less than end-to-end in all three configurations; queueing, result transfer, and MP4 muxing sit outside the profiled stages.

Scaling behaviour

Denoise speedup is not uniform across the sweep: 1.65× from two to four GPUs (83% efficiency) but 1.92× from four to eight (96%). Two GPUs run Ulysses1, so there is no sequence parallelism at all; going to four GPUs pays the one-time cost of introducing the all-to-all, while going from four to eight only widens an all-to-all that already exists. Expect the second doubling to pay off better than the first on this topology.

VAE decode behaves in the opposite direction — 1.93× from two to four GPUs but only 1.55× from four to eight — as patch parallelism produces smaller tiles and tile-boundary overhead grows.

Text encoding is 0.04 s in every configuration and is not worth optimising; denoise is 96-98% of end-to-end throughout.

Per-GPU memory model

Peak memory falls by 11.05 GiB from two to four GPUs and by a further 5.37 GiB from four to eight — halving each time the Ulysses degree doubles, because all three configurations shard the DiT with TP2 and hold identical weights per GPU. Fitting the first two points gives:

peak HBM per GPU ≈ 55.4 GiB + 22.1 GiB / ulysses_degree

That fit predicts 60.9 GiB at Ulysses4 against 61.07 GiB measured, a 0.25% error. The practical consequence is a floor near 55 GiB per GPU: adding GPUs past eight will not reduce the per-GPU requirement further. To serve on smaller cards, raise the tensor-parallel degree instead.

Re-measure memory for longer reference inputs, concurrency greater than one, or a different output shape.

Not yet measured

Ref2VA latency and memory have not been measured on this host. Ref2VA drives the Qwen3-VL presentation from tens of tokens for text-only prompts to several thousand with image or video references, so T2VA numbers do not transfer to it. The eight-GPU device-order comparison in the section above was also not run.

T2VA request example

t2va requires an explicit aspect_ratio even when width and height are supplied.

export API_URL="http://127.0.0.1:${PORT}/v1/videos/sync"

curl -sS --max-time 1800 -X POST "${API_URL}" \
  -F 'prompt=At night, three cats march into a bedroom playing tiny brass instruments, then abruptly file out, with synchronized room ambience.' \
  -F 'width=1344' \
  -F 'height=768' \
  -F 'aspect_ratio=16:9' \
  -F 'fps=24' \
  -F 'num_inference_steps=50' \
  -F 'flow_shift=12' \
  -F 'seed=1101' \
  -F 'extra_params={"task":"t2va","duration":5.0,"audio_flow_shift":3.0}' \
  -o t2va.mp4

To collect the stage breakdown and peak memory for a request, add --enable-diffusion-pipeline-profiler to the server command and -D headers.txt to the curl invocation, then read X-Stage-Durations and X-Peak-Memory-MB from the saved headers.

Ref2VA request example

ref2va defaults to a 16:9 aspect ratio, so aspect_ratio is optional here.

curl -X POST "http://127.0.0.1:${PORT}/v1/videos/sync" \
  --fail-with-body -w '\nHTTP %{http_code}\n' \
  --max-time 1200 \
  -F "input_reference=@/root/hand.jpg;type=image/jpeg" \
  -F "audio_reference=</root/audio_ref.json" \
  -F "prompt=2D动画融合在一起的影像。夕阳余晖残留在窗边，生活感十足的小厨房里有旧木桌、洗到一半的马克杯、起雾的玻璃瓶、悬挂的抹布。画面带有智能手机单手拍摄的手抖、近距离对焦的犹豫、逆光曝光波动。要像在家中慌忙拍下某个不可思议事件的自然质感，不要广告影像的精心整理。声音只用厨房环境声与手绘生物柔和的电子音、小小的叫声。" \
  -F 'width=1344' -F 'height=768' -F 'fps=24' \
  -F 'num_inference_steps=60' -F 'flow_shift=12' -F 'seed=1101' \
  -F 'extra_params={"task":"ref2va","duration":8,"audio_flow_shift":3.0}' \
  -o /root/out_ref2va.mp4