vllm-omni/recipes/MiniMaxAI/MiniMax-H3-Spark-GB10.md

MiniMax-H3 on DGX Spark (GB10)

This recipe uses online FP8 weight quantization, tiled VAE decode, and a single resident partition. GB10 is a unified-memory platform, so unlike the discrete-GPU recipes it uses no offload of any kind — see the capacity note below.

Validated on:

• Host: DGX Spark (GB10), aarch64
• GPUs: 1, or 2 hosts with one unified-memory GPU each
• Driver: 580.173.02 for the two-host qualification
• vLLM: 0.26.0
• vLLM-Omni: main at e1aa6eae75c460cd1893bc320546e81e66973831
• One-host workloads: T2VA and Ref2VA, 960x576, duration=8.0, 50 steps
• Two-host workload: T2VA, 1344x768, duration=5.0 and duration=10.0, 50 steps, text-encoder TP2, and DiT USP2
• Common sampling settings: flow_shift=12, seed=1101

The commands and measurements below describe this pinned revision. Newer versions have changed the scope of --quantization fp8; see Online FP8 quantization for the current component selection and offload support. Re-qualify memory, output, and latency on GB10 before comparing results across revisions.

Capacity requirements

Resource	DGX Spark (GB10)
Unified memory	128 GB LPDDR5X (~121 GiB usable)
GPUs	1
Checkpoint storage	135 GiB per partition
Measured peak allocator high-water, T2VA (FP8)	97.7 GiB
Measured peak allocator high-water, Ref2VA (FP8)	102.8 GiB
Measured peak per rank, two-host T2VA 5 s / 10 s (FP8)	75.45 / 87.17 GiB

FL2VA and Ref2VA are separate 135 GiB checkpoint partitions. Start one server at a time.

On GB10 the CPU and GPU share one physical memory pool, which changes the capacity math relative to the RTX and datacenter recipes:

• Do not use --enable-distributed-layerwise-offload. DLO stages rank-local weights into pinned host memory, which on GB10 is the same pool the weights are already in. Peak usage roughly doubles and the process is killed by the OOM killer (Exit code: -9) shortly after Enabling offloader backend.
• Do not use --enable-cpu-offload. Moving weights from "VRAM" to "host RAM" frees nothing here.
• --quantization fp8 is mandatory. A BF16 partition is 135 GiB and does not fit in 121 GiB. At the validated revision, online FP8 quantizes the DiT only (62 GiB to ~31 GiB); the Qwen3-VL text encoder and both VAEs stay BF16. Current versions also quantize eligible text-decoder linears by default, so the same flag does not imply the same quantization scope across revisions.

With ~97.7 GiB of the pool held by the allocator at peak, T2VA leaves roughly 23 GiB of headroom for the OS, page cache, and any other process on the box. Ref2VA is tighter on both counts. The allocator high-water rises to 102.8 GiB, and the allocator figure understates true pool usage.

Sampling /proc/meminfo at 1 Hz across a 50-step Ref2VA request on a host with MemTotal: 127600524 kB (121.7 GiB), MemAvailable started at 8.1 GiB, fell to a trough of 7.4 GiB, and recovered to 8.9 GiB. At the trough the pool held roughly 114.3 GiB — 11.4 GiB more than the 102.8 GiB the allocator reports, which is the CUDA context, non-PyTorch allocations, and page cache the header does not count.

Treat this platform as strictly single-tenant for Ref2VA. There is no room for a second process, and adding reference images, reference videos, or a larger output shape will run into the OOM killer.

One GB10: 960x576, 8 seconds

CUDA_VISIBLE_DEVICES=0 \
FLASHINFER_DISABLE_VERSION_CHECK=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=7200 \
vllm serve /path/to/MiniMax-H3/FL2VA \
  --omni --trust-remote-code --host 0.0.0.0 --port 8000 \
  --init-timeout 3600 \
  --num-gpus 1 --tensor-parallel-size 1 --text-encoder-tp-size 1 \
  --usp 1 --ring 1 --vae-patch-parallel-size 1 \
  --vae-parallel-mode tile --vae-use-tiling \
  --quantization fp8 --enforce-eager \
  --diffusion-attention-backend CUDNN_ATTN

Wait for Application startup complete, then run a short smoke test before committing to a full 50-step request.

t2va requires an explicit aspect_ratio even when width and height are supplied; omitting it fails the request with OmniClientError: t2va requires an explicit aspect_ratio.

curl --fail-with-body -sS -X POST http://127.0.0.1:8000/v1/videos/sync \
  -F 'prompt=At night, three cats march into a bedroom playing tiny brass instruments, then abruptly file out, with synchronized room ambience.' \
  -F 'width=960' \
  -F 'height=576' \
  -F 'aspect_ratio=16:9' \
  -F 'fps=24' \
  -F 'num_inference_steps=10' \
  -F 'flow_shift=12' \
  -F 'seed=1101' \
  -F 'extra_params={"task":"t2va","duration":4.0,"audio_flow_shift":3.0}' \
  -o h3-gb10-smoke.mp4

The full-quality request uses 50 steps:

curl --fail-with-body -sS -X POST http://127.0.0.1:8000/v1/videos/sync \
  -F 'prompt=At night, three cats march into a bedroom playing tiny brass instruments, then abruptly file out, with synchronized room ambience.' \
  -F 'width=960' \
  -F 'height=576' \
  -F 'aspect_ratio=16:9' \
  -F 'fps=24' \
  -F 'num_inference_steps=50' \
  -F 'flow_shift=12' \
  -F 'seed=1101' \
  -F 'extra_params={"task":"t2va","duration":8.0,"audio_flow_shift":3.0}' \
  -o h3-gb10-t2va.mp4

ffprobe -v error -show_entries \
  stream=index,codec_name,width,height,r_frame_rate,sample_rate,channels \
  -of json h3-gb10-t2va.mp4

Both requests return 200 OK with an MP4 body: H.264 video at the requested geometry plus a 32 kHz stereo AAC track.

One GB10: Ref2VA, 960x576, 8 seconds

FL2VA and Ref2VA are separate 135 GiB partitions and only one fits at a time. Stop the FL2VA server before starting this one.

CUDA_VISIBLE_DEVICES=0 \
FLASHINFER_DISABLE_VERSION_CHECK=1 \
VLLM_WORKER_MULTIPROC_METHOD=spawn \
VLLM_OMNI_VIDEO_SYNC_TIMEOUT=14400 \
vllm serve /path/to/MiniMax-H3/Ref2VA \
  --omni --trust-remote-code --host 0.0.0.0 --port 8000 \
  --init-timeout 3600 \
  --num-gpus 1 --tensor-parallel-size 1 --text-encoder-tp-size 1 \
  --usp 1 --ring 1 --vae-patch-parallel-size 1 \
  --vae-parallel-mode tile --vae-use-tiling \
  --quantization fp8 --enforce-eager \
  --diffusion-attention-backend CUDNN_ATTN

The 7200 s timeout used for T2VA is too short here: a 50-step Ref2VA request takes over an hour on this platform. Use 14400 s.

Unlike t2va, ref2va defaults to a 16:9 output ratio, so aspect_ratio can be omitted when width and height are supplied.

curl --fail-with-body -sS -X POST http://127.0.0.1:8000/v1/videos/sync \
  -D ref2va-headers.txt \
  -w 'client_e2e_s=%{time_total}\n' \
  --max-time 14400 \
  -F 'input_reference=@/path/to/reference.jpg;type=image/jpeg' \
  -F 'prompt=A hand rests on a wooden table in warm evening light, then slowly opens and turns toward the camera, with quiet room ambience.' \
  -F 'width=960' \
  -F 'height=576' \
  -F 'fps=24' \
  -F 'num_inference_steps=50' \
  -F 'flow_shift=12' \
  -F 'seed=1101' \
  -F 'extra_params={"task":"ref2va","duration":8.0,"audio_flow_shift":3.0}' \
  -o h3-gb10-ref2va.mp4

The response is 200 OK with an MP4 body: H.264 at the requested geometry plus a 32 kHz stereo AAC track. ffprobe reports 8.032 s for duration=8.0.

Two GB10 hosts: T2VA at 1344x768

The two-host qualification uses one GB10 GPU per DGX Spark, connected through a single ConnectX-7 RoCE interface. The FL2VA checkpoint and software revision are identical on both hosts. The distributed configuration is:

• world size 2;
• text-encoder tensor parallel size 2;
• DiT Ulysses sequence parallel size 2 and ring degree 1;
• VAE patch parallel size 1;
• online FP8, eager execution, cuDNN attention, and tiled VAE decode.

The current diffusion multiprocess executor starts all ranks on one host and sets MASTER_ADDR=localhost. These measurements therefore used an external env:// launcher with one rank on each host and a benchmark-only adaptation that maps both hosts' local worker to cuda:0. This qualifies the two-rank MiniMax-H3 pipeline and its output, but it is not yet a supported cross-host vllm serve command. Do not infer that the single-host launch command above works unchanged across two machines.

One 5-second, 2-step request warmed both workers before the measured runs. The reported pipeline elapsed time covers distributed model execution through GPU completion; MP4 encoding happened afterward and is not included.

Duration	Steps	Text encode	Denoise	Per step	VAE decode	Pipeline elapsed	Max peak per rank
5 s	50	0.21 s	1570.19 s	31.40 s	79.45 s	1649.87 s (27 min 30 s)	75.45 GiB
10 s	50	0.15 s	4448.12 s	88.96 s	160.71 s	4609.00 s (76 min 49 s)	87.17 GiB

Both ranks reported stage times and allocator peaks within measurement noise. The generated outputs were H.264 at 1344x768 and 24 FPS with 32 kHz stereo AAC: 124 frames (5.167 s) and 243 frames (10.125 s). Both files decoded successfully, and neither run encountered an OOM or NCCL error. These are single qualification runs (n=1), not throughput guarantees.

Measured latency

Single GB10, FP8, eager, cuDNN attention, tiled VAE, one request at a time:

Workload	Steps	Text encode	Denoise	Per step	VAE decode	MP4 mux	Client E2E	Peak memory
T2VA, 4 s, smoke shape	10	—	—	8.68 s	—	0.49 s	95.9 s	—
T2VA, 8 s, 960x576	50	0.25 s	2088 s	42.61 s	70 s	4.84 s	2169.4 s (36 min 9 s)	97.7 GiB
Ref2VA, 8 s, 960x576, one image (3 runs)	10	8.43-10.94 s	665.5-748.8 s	66.55-74.88 s	67.9-69.8 s	3.63 s	770.3-861.1 s	102.8 GiB
Ref2VA, 8 s, 960x576, one image	50	9.66 s	4039.6 s	80.79 s	69.4 s	3.63 s	4157.0 s (69 min 17 s)	102.8 GiB

How these were measured:

• Stage times come from --enable-diffusion-pipeline-profiler, read from the X-Stage-Durations response header of /v1/videos/sync. The keys are MiniMaxH3Pipeline.encode_prompt (text encode), MiniMaxH3Pipeline.diffuse (denoise), and MiniMaxH3Pipeline.decode (VAE decode).
• MP4 mux is not a header field. It is the Video response encoding (MP4 bytes) line in the server log.
• Peak memory is the X-Peak-Memory-MB header, which reports the CUDA allocator high-water mark (torch.cuda.max_memory_reserved) for the request. It excludes the CUDA context and non-PyTorch allocations, so true pool usage is somewhat higher. nvidia-smi --query-gpu=memory.used is not usable on GB10 — unified memory has no discrete VRAM to report and the field returns [N/A]. See the capacity section for a /proc/meminfo cross-check.
• For T2VA, text encode and VAE decode were measured on a separate 10-step run at the same 960x576 / 8 s geometry. Both are independent of step count, so they transfer to the 50-step row unchanged. The Ref2VA MP4 mux figure transfers the same way, from the 10-step run at identical output geometry and duration.
• Ref2VA peak memory is byte-identical between the 10-step and 50-step runs (X-Peak-Memory-MB: 105318.000 in both), confirming that peak is set by output geometry and reference input rather than step count.
• Stage times do not sum exactly to end-to-end. Queueing, result transfer, and engine bookkeeping sit outside the profiled stages.

Denoise dominates at 96% of end-to-end, so step count and geometry are the only knobs that materially change the wall clock on this platform.

Ref2VA costs 1.92x the wall clock of T2VA at the same geometry and step count: 4157.0 s versus 2169.4 s. Text encoding shows the most dramatic ratio — 9.66 s versus 0.25 s, 39x — but at 0.2% of end-to-end it is not what makes Ref2VA slow. The cost is in denoise. A single reference image expands the Qwen3-VL presentation to 6883 tokens (logged as MiniMax H3 ref2va Qwen presentation: 6883 tokens, 1 reference images), and those tokens sit in the attention context of every denoise step. Per-step time rises from 42.61 s to 80.79 s, and that 1.90x multiplies across all 50 steps. VAE decode is unchanged at ~69 s, as expected for an identical output shape.

Repeated identical 10-step T2VA requests varied by up to 23% in denoise time (397 s to 490 s) on a machine that had already been under sustained load. Treat these as single-run measurements on one machine, not a throughput guarantee, and expect the slower end of that range for back-to-back jobs.

Ref2VA per-step time is load-dependent in the same way. On an idle machine the first 10-step request ran at 66.55 s per step. The 50-step request that followed occupied the box for 69 consecutive minutes and averaged 80.79 s per step. Two further 10-step requests taken immediately afterwards settled at 74.88 s and 74.31 s — 12.5% slower than the idle run, and within 0.8% of each other, so this is a stable thermal state rather than measurement noise.

Plan for 81 s per step when sizing a full 50-step Ref2VA job; 66 s per step is only reachable on a cold machine and does not hold for the length of one request.

Ref2VA text encode converged downward across the four runs (10.94 s, 9.66 s, 8.52 s, 8.43 s) as first-touch costs fell away; treat ~8.5 s as steady state. VAE decode was stable at 67.9-69.8 s throughout.

Set VLLM_OMNI_VIDEO_SYNC_TIMEOUT well above the expected request time — the default 1800 s is shorter than a 50-step run on this platform, and 7200 s is shorter than a 50-step Ref2VA run.

Known limitations

• With one GPU there is no Ulysses, ring, TP, or VAE patch parallelism. Pass --usp 1 --ring 1 --vae-patch-parallel-size 1 explicitly if a profile or script would otherwise supply higher degrees.
• The two-host results require the external launch adaptation described above; cross-host diffusion worker launch is not exposed by the documented server CLI.
• --enforce-eager is used because regional compile adds startup time without a parallelism win on a single rank.
• t2va rejects requests without an explicit aspect_ratio. When width and height are both supplied they take precedence and aspect_ratio only has to name one of the supported ratios.
• Sustained load degrades throughput. See the variance notes above.
• Only one partition is resident. For Ref2VA, stop the FL2VA server and restart with /path/to/MiniMax-H3/Ref2VA. Ref2VA with one reference image is measured above; reference videos and additional images push the token count higher and were not measured. With MemAvailable bottoming out at 7.4 GiB during a single-image request, there is very little room to grow — re-measure before adding references or raising the output shape.
• Ref2VA requires a vLLM-Omni build newer than v0.26.0. See the version note under Validated on: FP8 weight loading and image-only Ref2VA are both broken on release/v0.26.0. The suggested version is v0.26.1.
• At the validated revision, online FP8 with layerwise offload produced a weight stride the Cutlass FP8 kernel rejected. Current H3 supports online FP8 with layerwise offload and both DLO transfer paths; see the linked FP8 section above. This does not change the GB10 capacity guidance: offload uses the same unified-memory pool and is not enabled in this recipe.