vllm-omni/recipes/MiniMaxAI/MiniMax-H3-RTX-PRO-6000.md

# MiniMax-H3 on RTX PRO 6000 Blackwell GPUs

This recipe runs MiniMax-H3 in BF16 on 96 GiB RTX PRO 6000 Blackwell GPUs.
The additional HBM over the RTX PRO 5000 profile removes the four-GPU
minimum: two GPUs are enough to keep the model resident with TP2. Four and
eight GPUs raise the Ulysses degree to shard the attention sequence further.
CPU offload and distributed layerwise offload are not required in any of
these configurations.

Validated on:
- Host: YLX Y762
- GPUs: 8 × RTX PRO 6000 Blackwell (96 GiB)
- Device order: default (`CUDA_VISIBLE_DEVICES` not set)
- Driver / CUDA: 580.105.08 / 13.0
- Container image: `vllm/vllm-omni:minimax-h3`
- Workload: T2VA, 1344×768, `duration=5.0`, 50 steps, `flow_shift=12`, `seed=1101`

## Capacity requirements

| Resource | Two GPUs | Four GPUs | Eight GPUs |
| --- | ---: | ---: | ---: |
| GPU HBM | 96 GiB per GPU | 96 GiB per GPU | 96 GiB per GPU |
| Checkpoint storage | 135 GiB per partition | 135 GiB per partition | 135 GiB per partition |
| Available system RAM | 200 GiB minimum | 200 GiB minimum | 200 GiB minimum |
| Recommended system RAM | 384 GiB | 384 GiB | 384 GiB |
| Measured peak HBM per GPU | 77.49 GiB | 66.44 GiB | 61.07 GiB |

`FL2VA` and `Ref2VA` are separate checkpoint partitions. Start one server at
a time on a host sized for the minimum system-memory requirement, or pass
`--task-type fl2va` or `--task-type ref2va` to download and load a single
partition. The two-server layout described in the eight-GPU section doubles
both the storage and the host-memory requirement.

## Parallelism summary

| Configuration | TP | Ulysses | Text-encoder TP | VAE patch parallel |
| --- | ---: | ---: | ---: | ---: |
| Two GPUs | 2 | 1 | 2 | 2 |
| Four GPUs | 2 | 2 | 4 | 4 |
| Eight GPUs | 2 | 4 | 8 | 8 |
| Eight GPUs, headroom variant | 4 | 2 | 8 | 8 |

Per-GPU weight residency is set by the tensor-parallel degree alone. Raising
the Ulysses degree lowers activation memory and per-step latency but leaves
weight residency unchanged. Ring parallelism stays at 1, and H3 is
CFG-distilled, so `--cfg-parallel-size` must also remain 1.

## PCIe topology and GPU order

RTX PRO 6000 Blackwell does not provide NVLink. Before starting the server,
inspect the host topology:

```bash
nvidia-smi topo -m
nvidia-smi nvlink -s
```

Tensor-parallel groups are consecutive ranks. Ulysses groups are strided by
the tensor-parallel degree. Confirm this grouping against the server log
before tuning the device order on a new host.

For the two-GPU configuration, pick a single `PXB` pair on one NUMA node.
For the four-GPU configuration, pick two `PXB` pairs on the same node and
order them so that each Ulysses group lands inside one pair. If physical
GPUs `(0,1)` and `(2,3)` are the two local pairs, the order is
`CUDA_VISIBLE_DEVICES=0,2,1,3`. Do not copy these IDs blindly: reproduce the
same relationship on the target host.

Device-order pinning and NUMA binding are tuning knobs, not prerequisites.
The measurements in this recipe were taken without either, so any gain from
them is additional to the numbers reported below and is host-specific.

## Two-GPU serving configuration

Two 96 GiB GPUs hold the BF16 model with TP2 alone. There is no Ulysses
group, so the DiT attention sequence is not sharded and activation memory is
higher than in the four-GPU profile at the same output shape. Re-measure
peak memory before increasing the output shape, the reference-input length,
or concurrency.

```bash
export MODEL_ROOT=/path/to/MiniMax-H3
export MODEL="${MODEL_ROOT}/FL2VA"
export PORT=8000

VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve "${MODEL}" \
  --omni \
  --host 0.0.0.0 \
  --port "${PORT}" \
  --trust-remote-code \
  --num-gpus 2 \
  --tensor-parallel-size 2 \
  --usp 1 \
  --ring 1 \
  --text-encoder-tp-size 2 \
  --vae-patch-parallel-size 2 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend CUDNN_ATTN
```

## Four-GPU serving configuration

Four GPUs keep TP2 for the weight shard and add Ulysses2, which shards the
attention sequence and lowers per-GPU activation memory. Text-encoder TP4
and VAE patch parallelism 4 spread the Qwen3-VL encoder and tiled decode
across all four GPUs. TP1 is not an option on this card: an unsharded BF16
partition does not fit in 96 GiB.

```bash
export MODEL_ROOT=/path/to/MiniMax-H3
export MODEL="${MODEL_ROOT}/FL2VA"
export PORT=8000

VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve "${MODEL}" \
  --omni \
  --host 0.0.0.0 \
  --port "${PORT}" \
  --trust-remote-code \
  --num-gpus 4 \
  --tensor-parallel-size 2 \
  --usp 2 \
  --ring 1 \
  --text-encoder-tp-size 4 \
  --vae-patch-parallel-size 4 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend CUDNN_ATTN
```

## Eight-GPU serving configuration

Eight GPUs extend the Ulysses degree to 4 while keeping TP2. This does not
lower per-GPU weight residency relative to the four-GPU profile; it lowers
activation memory and per-step latency by sharding the attention sequence
four ways.

```bash
export MODEL_ROOT=/path/to/MiniMax-H3
export MODEL="${MODEL_ROOT}/FL2VA"
export PORT=8000

VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800 \
vllm serve "${MODEL}" \
  --omni \
  --host 0.0.0.0 \
  --port "${PORT}" \
  --trust-remote-code \
  --num-gpus 8 \
  --tensor-parallel-size 2 \
  --usp 4 \
  --ring 1 \
  --text-encoder-tp-size 8 \
  --vae-patch-parallel-size 8 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --diffusion-attention-backend CUDNN_ATTN
```

### Rank ordering across two sockets

With TP2 and Ulysses4 the two Ulysses groups are ranks `(0,2,4,6)` and
`(1,3,5,7)`, and the four tensor-parallel pairs are `(0,1)`, `(2,3)`,
`(4,5)`, and `(6,7)`. On a dual-socket host these two collectives cannot
both be socket-local. If you choose to pin the device order, the trade-off
is:

| Device order | Socket-local collective | Crosses sockets |
| --- | --- | --- |
| `0,4,1,5,2,6,3,7` | Ulysses all-to-all | TP all-reduce |
| `0,1,2,3,4,5,6,7` | TP all-reduce | Ulysses all-to-all |

The first order assumes physical GPUs `0-3` are on NUMA node 0 and `4-7` on
node 1. Reproduce that relationship on the target host rather than copying
the IDs. The reported measurements use the default order and no NUMA
binding; the two orders were not compared on this host.

### Headroom variant: TP4 with Ulysses2

Raising the tensor-parallel degree to 4 halves per-GPU weight residency and
leaves room for long Ref2VA references, larger output shapes, or
concurrency greater than one. It also adds a four-way all-reduce per DiT
block over a link without NVLink. Use it when the TP2 profile runs out of
headroom, not as the default. Replace the parallel flags above with:

```bash
  --tensor-parallel-size 4 \
  --usp 2 \
```

### Two independent four-GPU servers

An eight-GPU host can instead run one FL2VA server and one Ref2VA server
side by side, each pinned to its own NUMA node with the four-GPU
configuration. This removes every cross-socket collective and lifts the
one-server-at-a-time restriction, at the cost of doubling the requirements
to 400 GiB available system RAM and 270 GiB of model storage.

Use `CUDA_VISIBLE_DEVICES=0,2,1,3`, `--cpunodebind=0 --membind=0`, and
`--port 8091` for the FL2VA server. Use `CUDA_VISIBLE_DEVICES=4,6,5,7`,
`--cpunodebind=1 --membind=1`, and `--port 8092` for the Ref2VA server.
This layout is described for completeness and was not part of the measured
runs.

## Shared serving notes

Do not add `--enforce-eager` for a performance run. Warm the server before
measuring so regional compilation is outside the measured request. On this
host the first request after startup ran 19% slower than the steady state
(107.58 s versus 90.48 s on eight GPUs); two warmup requests were enough to
converge.

For Ref2VA on a single-server layout, stop the FL2VA server and restart the
same command with `MODEL="${MODEL_ROOT}/Ref2VA"`.

## Attention backend

RTX PRO 6000 Blackwell is an SM120 part, so the datacenter-Blackwell
`TRTLLM_ATTN` default does not apply here. Every configuration above selects
cuDNN BF16 attention explicitly. This keeps the recipe independent of
platform-default backend changes and does not depend on the experimental
SM120 kernel.

## Optional: online FP8

`--quantization fp8` quantizes eligible DiT linears at load time and is
compatible with tensor parallelism and VAE tiling. Use it when the resident
BF16 peak leaves too little headroom for long Ref2VA references, a larger
output shape, or concurrency greater than one. It cannot be combined with
layerwise offload.

## Target-hardware validation

T2VA, 1344×768, `duration=5.0`, `fps=24`, 50 steps, `flow_shift=12`,
`seed=1101`, BF16, `CUDNN_ATTN`, tiled VAE, one request at a time. Servers
were started with the commands above — default device order, no NUMA
binding — plus `--enable-diffusion-pipeline-profiler`. Two warmup requests
preceded each measured request.

| Measurement | Two GPUs | Four GPUs | Eight GPUs |
| --- | ---: | ---: | ---: |
| Topology | TP2 × USP1 | TP2 × USP2 | TP2 × USP4 |
| Text encode | 0.040 s | 0.044 s | 0.035 s |
| Denoise, 50 steps | 278.55 s | 168.73 s | 87.90 s |
| Per step | 5.571 s | 3.375 s | 1.758 s |
| VAE decode | 5.396 s | 2.791 s | 1.798 s |
| Client E2E | 284.76 s | 172.32 s | 90.48 s |
| Peak HBM per GPU | 77.49 GiB | 66.44 GiB | 61.07 GiB |
| Headroom below 96 GiB | 18.5 GiB | 29.6 GiB | 34.9 GiB |

Stage times are read from the `X-Stage-Durations` response header of
`/v1/videos/sync`. Peak memory is the maximum of `nvidia-smi
--query-gpu=memory.used` sampled at 1 Hz across every device for the
duration of the measured request. Per step is denoise wall time divided by
the 50 requested steps. Stage times sum to roughly 0.75 s less than
end-to-end in all three configurations; queueing, result transfer, and MP4
muxing sit outside the profiled stages.

### Scaling behaviour

Denoise speedup is not uniform across the sweep: 1.65× from two to four
GPUs (83% efficiency) but 1.92× from four to eight (96%). Two GPUs run
Ulysses1, so there is no sequence parallelism at all; going to four GPUs
pays the one-time cost of introducing the all-to-all, while going from four
to eight only widens an all-to-all that already exists. Expect the second
doubling to pay off better than the first on this topology.

VAE decode behaves in the opposite direction — 1.93× from two to four GPUs
but only 1.55× from four to eight — as patch parallelism produces smaller
tiles and tile-boundary overhead grows.

Text encoding is 0.04 s in every configuration and is not worth optimising;
denoise is 96-98% of end-to-end throughout.

### Per-GPU memory model

Peak memory falls by 11.05 GiB from two to four GPUs and by a further
5.37 GiB from four to eight — halving each time the Ulysses degree doubles,
because all three configurations shard the DiT with TP2 and hold identical
weights per GPU. Fitting the first two points gives:

```
peak HBM per GPU ≈ 55.4 GiB + 22.1 GiB / ulysses_degree
```

That fit predicts 60.9 GiB at Ulysses4 against 61.07 GiB measured, a 0.25%
error. The practical consequence is a floor near 55 GiB per GPU: adding
GPUs past eight will not reduce the per-GPU requirement further. To serve
on smaller cards, raise the tensor-parallel degree instead.

Re-measure memory for longer reference inputs, concurrency greater than
one, or a different output shape.

### Not yet measured

Ref2VA latency and memory have not been measured on this host. Ref2VA
drives the Qwen3-VL presentation from tens of tokens for text-only prompts
to several thousand with image or video references, so T2VA numbers do not
transfer to it. The eight-GPU device-order comparison in the section above
was also not run.

## T2VA request example

`t2va` requires an explicit `aspect_ratio` even when `width` and `height`
are supplied.

```bash
export API_URL="http://127.0.0.1:${PORT}/v1/videos/sync"

curl -sS --max-time 1800 -X POST "${API_URL}" \
  -F 'prompt=At night, three cats march into a bedroom playing tiny brass instruments, then abruptly file out, with synchronized room ambience.' \
  -F 'width=1344' \
  -F 'height=768' \
  -F 'aspect_ratio=16:9' \
  -F 'fps=24' \
  -F 'num_inference_steps=50' \
  -F 'flow_shift=12' \
  -F 'seed=1101' \
  -F 'extra_params={"task":"t2va","duration":5.0,"audio_flow_shift":3.0}' \
  -o t2va.mp4
```

To collect the stage breakdown and peak memory for a request, add
`--enable-diffusion-pipeline-profiler` to the server command and `-D
headers.txt` to the curl invocation, then read `X-Stage-Durations` and
`X-Peak-Memory-MB` from the saved headers.

## Ref2VA request example

`ref2va` defaults to a 16:9 aspect ratio, so `aspect_ratio` is optional
here.

```bash
curl -X POST "http://127.0.0.1:${PORT}/v1/videos/sync" \
  --fail-with-body -w '\nHTTP %{http_code}\n' \
  --max-time 1200 \
  -F "input_reference=@/root/hand.jpg;type=image/jpeg" \
  -F "audio_reference=</root/audio_ref.json" \
  -F "prompt=2D动画融合在一起的影像。夕阳余晖残留在窗边，生活感十足的小厨房里有旧木桌、洗到一半的马克杯、起雾的玻璃瓶、悬挂的抹布。画面带有智能手机单手拍摄的手抖、近距离对焦的犹豫、逆光曝光波动。要像在家中慌忙拍下某个不可思议事件的自然质感，不要广告影像的精心整理。声音只用厨房环境声与手绘生物柔和的电子音、小小的叫声。" \
  -F 'width=1344' -F 'height=768' -F 'fps=24' \
  -F 'num_inference_steps=60' -F 'flow_shift=12' -F 'seed=1101' \
  -F 'extra_params={"task":"ref2va","duration":8,"audio_flow_shift":3.0}' \
  -o /root/out_ref2va.mp4
```