vllm-omni/recipes/MiniMaxAI/MiniMax-H3-NPU.md

MiniMax H3 on Ascend NPU

Joint video and audio generation with text, first/last keyframes, and mixed image/video/audio references — Ascend NPU deployment guide for Atlas 800I A2 and Atlas 800I A3

Summary

• Vendor: MiniMaxAI
• Model: MiniMaxAI/MiniMax-H3
• Tasks: T2VA, FL2VA, and Ref2VA
• Mode: OpenAI-compatible /v1/videos HTTP serving
• Hardware: Atlas 800I A2 / Atlas 800I A3, 8x NPU
• Maintainer: Community

This recipe adapts MiniMax-H3.md for Ascend NPU environments. Differences from the GPU path:

• Runs on a CPU-only (aarch64) PyTorch build with torch_npu; no CUDA runtime is present.
• Audio loading does not require TorchCodec (whose aarch64 wheels are built against CUDA torch and fail to load on CPU-only builds). vLLM-Omni automatically falls back to soundfile / ffmpeg for wav/mp3/m4a/mp4 audio inputs.

For the four-card Ascend 950PR / 950DT route, see MiniMax-H3-NPU-950PR.md.

Prerequisites

Checkpoint

Download the weights once; vllm serve accepts the repository ID or a local path and resolves the nested components automatically:

export MODEL=/path/to/MiniMax-H3
hf download MiniMaxAI/MiniMax-H3 --local-dir "${MODEL}"

The checkpoint directory layout is:

MiniMax-H3/
├── FL2VA/          # shared by T2VA/FL2VA tasks
├── Ref2VA/         # Ref2VA task weights
└── model_index.json

Environment

The recommended deployment uses the official vLLM-Omni NPU images, which ship the matching vLLM / vLLM-Ascend / vLLM-Omni stack (see NPU installation). Pick the image matching the hardware — quay.io/ascend/vllm-omni:v0.29.0 for Atlas 800I A2, quay.io/ascend/vllm-omni:v0.29.0-a3 for Atlas 800I A3 — and check the tag list for newer releases:

export IMAGE=quay.io/ascend/vllm-omni:v0.29.0-a3
export CONTAINER_NAME=h3

docker run -it -u root --name ${CONTAINER_NAME} \
  --privileged=true \
  --shm-size=2000g \
  --net=host \
  --device /dev/davinci0 \
  --device /dev/davinci1 \
  --device /dev/davinci2 \
  --device /dev/davinci3 \
  --device /dev/davinci4 \
  --device /dev/davinci5 \
  --device /dev/davinci6 \
  --device /dev/davinci7 \
  --device /dev/davinci_manager \
  --device /dev/devmm_svm \
  --device /dev/hisi_hdc \
  -v /home:/home \
  -v /usr/local/dcmi:/usr/local/dcmi \
  -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
  -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
  -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
  -v /etc/ascend_install.info:/etc/ascend_install.info \
  -v /root/.cache:/root/.cache \
  ${IMAGE} \
  /bin/bash

Inside the container:

• MindIE-SD provides the Ascend-optimized fused operators (adalayernorm, etc.), the LaserAttention kernel, and the RainFusion block-sparse attention kernel. If the image does not ship it, build it from source:

git clone https://gitcode.com/Ascend/MindIE-SD.git
cd MindIE-SD
python setup.py bdist_wheel
pip install dist/mindiesd-*.whl

• To track the latest vLLM-Omni, install it from a checkout:

git clone https://github.com/vllm-project/vllm-omni.git
cd vllm-omni
VLLM_OMNI_TARGET_DEVICE=npu pip install -e . --no-build-isolation

• ffmpeg and ffprobe must be available on PATH (reference-video preparation and MP4 output). Reference-video decoding uses decord when available and falls back to PyAV otherwise; decord is required only for the Ref2VA workflow and can be built with:

apt update
apt install -y ffmpeg pkg-config libavcodec-dev libavformat-dev \
  libavutil-dev libswscale-dev libavfilter-dev libavdevice-dev
git clone --depth 1 --recursive https://github.com/dmlc/decord.git
cd decord
mkdir -p build && cd build
cmake .. -DUSE_CUDA=0 -DDECODE_FFMPEG=1 -DCMAKE_BUILD_TYPE=Release
make
cd ../python
pip3 install . --no-build-isolation

Start a server

T2VA and FL2VA share one set of weights; the Ref2VA weights differ. Select the task partition at startup with --task-type (fl2va or ref2va). When unset, only the T2VA/FL2VA weights are loaded. All remaining flags are task-agnostic.

Recommended lossless configuration (Atlas 800I A2 / A3)

Lossless means fused operators plus the high-performance FLASH_ATTN backend only. The configuration uses 8-card USP, 8-card text-encoder TP, distributed layerwise offload (DLO), and VAE tile parallelism:

export PORT=9098
export MODEL=/path/to/MiniMax-H3
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export VLLM_OMNI_VIDEO_SYNC_TIMEOUT=4000
export PYTHONDONTWRITEBYTECODE=1
export HF_HUB_OFFLINE=1
export TRANSFORMERS_OFFLINE=1
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
export MINDIE_SD_FA_TYPE="ascend_laser_attention"
export HCCL_NPU_SOCKET_PORT_RANGE="auto"

vllm serve "${MODEL}" \
  --omni \
  --host 0.0.0.0 \
  --port "${PORT}" \
  --trust-remote-code \
  --task-type fl2va \
  --num-gpus 8 \
  --usp 8 \
  --ring 1 \
  --text-encoder-tp-size 8 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --vae-patch-parallel-size 8 \
  --enable-distributed-layerwise-offload \
  --enable-diffusion-pipeline-profiler \
  --diffusion-attention-backend FLASH_ATTN

Notes:

• Do not add --enforce-eager. The first request includes regional compilation; warm the server once before measuring steady-state latency.
• H3 is CFG-distilled, so --cfg-parallel-size must remain 1.
• Keep --ring 1 when switching to RainFusion attention (below): the block-sparse kernel ranks key blocks over the whole sequence, so ring parallelism would split away the keys it needs. Scale with --usp instead.
• DLO offloads DiT layers with parameters gathered across the parallel group instead of replicated per rank, which is what fits the combined 768P service into 64 GB HBM per device (Atlas 800I A3). Host (CPU) memory usage is high with this option.

Recommended lossy configuration (Atlas 800I A2 / A3)

Adds RainFusion block-sparse attention and INT8 online quantization on top of the lossless configuration. Validated for T2VA and Ref2VA; see Benchmarks:

vllm serve "${MODEL}" \
  --omni \
  --host 0.0.0.0 \
  --port "${PORT}" \
  --trust-remote-code \
  --task-type fl2va \
  --num-gpus 8 \
  --usp 8 \
  --ring 1 \
  --text-encoder-tp-size 8 \
  --vae-parallel-mode tile \
  --vae-use-tiling \
  --vae-patch-parallel-size 8 \
  --enable-distributed-layerwise-offload \
  --enable-diffusion-pipeline-profiler \
  --diffusion-quantization-config '{"transformer":{"method":"int8"}}' \
  --diffusion-attention-config '{"default": {"backend": "RAINFUSION_ATTN",
      "block_sparse": {"sparsity": 0.8, "start_step": 12}}}'

Online INT8 quantization can be combined with distributed layerwise offload while AllGather is enabled. Keep AllGather enabled for SP>1; use --dlo-no-use-allgather only when collective communication is unavailable.

Optional optimizations

The following sections list only the increment or replacement relative to the recommended lossless configuration; environment variables and the remaining flags stay unchanged.

Cache-DiT

Append to enable DiT block caching with TaylorSeer extrapolation — see the Cache-DiT guide:

  --cache-backend cache_dit \
  --enable-cache-dit-summary \
  --cache-config '{"Fn_compute_blocks":2,"Bn_compute_blocks":1,"max_warmup_steps":4,"residual_diff_threshold":0.4,"max_continuous_cached_steps":4,"enable_taylorseer":true,"taylorseer_order":2}'

EQBSA sparse attention

EQBSA applies Q/K INT8 with V FP8 mixed quantization on top of block-sparse attention. Replace --diffusion-attention-backend FLASH_ATTN with:

  --diffusion-attention-config '{"default":{"backend":"RAINFUSION_ATTN","block_sparse":{"sparsity":0.8,"start_step":8,"end_step":13,"precision":"mix"}}}'

precision defaults to bf16; enabling EQBSA requires setting it to mix explicitly. start_step and end_step are the numbers of leading/trailing steps that fall back to dense attention — not step indices. For 1344x768, 50-step T2VA:

Configuration	start_step	end_step	Notes
Quality first (recommended)	8	13	General and complex-motion scenes
Balanced	0	13	Balanced
Speed first	0	0	Simple, low-motion scenes

When frames look discontinuous or details unstable, increase end_step first, then start_step; do not compensate by lowering sparsity alone. These pairings were validated at 1344x768 / 50 steps only; re-evaluate for other resolutions or step counts.

HTTP API examples

The request contract is identical to the GPU recipe; see MiniMax-H3.md § HTTP API examples. Use the validated 768P shapes (e.g. width=1344 height=768) on NPU. Quick start against the synchronous endpoint:

export API_URL="http://127.0.0.1:${PORT}/v1/videos/sync"

curl -sS -X POST "${API_URL}" \
  -F 'prompt=In a snowy blue-purple forest, Ori carefully walks past a sleeping giant; footsteps crunch in the snow while the creature breathes and softly snorts.' \
  -F 'width=1344' \
  -F 'height=768' \
  -F 'aspect_ratio=16:9' \
  -F 'fps=24' \
  -F 'num_inference_steps=50' \
  -F 'flow_shift=12.0' \
  -F 'audio_flow_shift=3.0' \
  -F 'seed=1101' \
  -F 'extra_params={"task":"t2va","duration":5.0}' \
  -o t2va.mp4

Switch tasks via extra_params.task (t2va / fl2va / ref2va); FL2VA and Ref2VA additionally pass input_references (first-frame image, reference video/audio). The full parameter table lives in the GPU recipe.

FlashGen 4-step online LoRA (T2VA)

The FlashGen 4-step weights are a native-layout LoRA single file (key_format=minimax-h3-native) loaded at runtime — no merge into the base checkpoint. Download them from FlashGen/Minimax-H3-4step-lora-flashgen, then on top of the recommended configuration above:

• append --lora-backend peft --lora-path "${FLASHGEN_LORA}",
• use --diffusion-attention-backend FLASH_ATTN (dense; drop the RainFusion attention config),
• issue T2VA requests with num_inference_steps=4 and the lora request field (name / path / scale) — see MiniMax-H3.md § FlashGen native LoRA.

T2VA only; FL2VA and Ref2VA are not supported, and neither --enable-cpu-offload nor --enable-layerwise-offload may be combined with it.

CPU MP4 response encoding (Atlas A2)

Non-streaming MP4 responses use one public automatic encoder. It checks the runtime frame shape, common dtype, and RGB channel-plane contiguity for every request; compatible inputs use direct planar PyAV frames, while unsupported inputs fall back to the legacy muxer before opening the PyAV container. No CLI flag, model declaration, or user configuration is required. Streaming fMP4 output remains on its existing incremental path.

Ascend-optimized components

The optimizations stacked by the recommended configurations above:

1. Ascend-friendly attention: the MindIE-SD ascend_laser_attention fused kernel selected through MINDIE_SD_FA_TYPE, carried by the FLASH_ATTN backend with a mask-free packed varlen path and K/V prefix slicing driven by the packed cu_seqlens metadata.
2. Ascend-friendly fused operators: RMSNorm, AddRMSNorm, GQA, SwiGLU, and rotary position embedding from MindIE-SD.
3. RainFusion block-sparse attention, optionally with EQBSA mixed INT8/FP8 precision — see RainFusion.
4. Online quantization: INT8, MXFP8, and MXFP4 — see online quantization.
5. Parallelism: USP, text-encoder TP, and native VAE tile patch parallelism, plus distributed layerwise offload.
6. Post-training: few-step distillation via the FlashGen online LoRA.

Benchmarks (Atlas 800I A2 / A3)

Measured with --enable-diffusion-pipeline-profiler on vLLM-Omni 0.28.0: lossless rows use the recommended lossless configuration, lossy rows the recommended lossy configuration. Memory figures are per device.

Atlas 800I A2

Task	Resolution	Frames	Duration (s)	Config	E2E (s)	DiT total (s)	DiT per-step (s)	DiT steps	Text encode (s)	Ref video encode (s)	Ref audio encode (s)	VAE decode (s)	Ref preprocess (s)	Post-process (s)	CPU MP4 (s)	Resident weights (GB)	Peak memory (GB)
t2va	1344x768	124	5	lossless	150.06	139.9	2.85	49	0.83	NA	NA	6.76	NA	0.462	1.18	2.97	24.7
t2va	1344x768	362	15	lossless	786.24	761.36	15.53	49	1.20	NA	NA	15.87	NA	1.98	2.95	2.97	36.21
t2va	1344x768	124	5	lossy	108.1	95.85	1.95	49	0.9	NA	NA	6.73	NA	0.745	1.04	2.55	22.47
t2va	1344x768	362	15	lossy	552.51	526.04	10.73	49	1.28	NA	NA	16.07	NA	1.90	3.76	2.55	35.11
ref2va	1344x768	124	5	lossless	474.89	421.68	8.6	49	2.42	13.31	0.38	6.47	3.94	0.489	1.16	3.4	23.45
ref2va	1344x768	362	15	lossless	3414.18	3336.98	68.08	49	6.48	30.85	0.59	15.8	12.59	1.99	3.08	3.4	36.28
ref2va	1344x768	124	5	lossy	380.92	326.50	6.66	49	2.47	13.48	0.38	6.36	4.68	0.591	1.22	2.55	22.57
ref2va	1344x768	362	15	lossy	2332.04	2249.63	45.89	49	6.33	35.1	0.59	15.72	11.07	1.35	2.3	2.55	35.58

Atlas 800I A3

Task	Resolution	Frames	Duration (s)	Config	E2E (s)	DiT total (s)	DiT per-step (s)	DiT steps	Text encode (s)	Ref video encode (s)	Ref audio encode (s)	VAE decode (s)	Ref preprocess (s)	Post-process (s)	CPU MP4 (s)	Resident weights (GB)	Peak memory (GB)
t2va	1344x768	124	5	lossless	114.19	105.87	2.16	49	0.48	NA	NA	6.56	NA	0.001	0.46	3.49	24.7
t2va	1344x768	362	15	lossless	599.29	580.38	11.84	49	1.83	NA	NA	14.95	NA	0.001	1.02	3.49	36.21
t2va	1344x768	124	5	lossy	84.82	77.47	1.58	49	0.57	NA	NA	6.23	NA	0.001	0.61	2.63	22.47
t2va	1344x768	362	15	lossy	401.43	383.56	7.82	49	1.72	NA	NA	14.83	NA	0.001	1.14	2.63	35.11
ref2va	1344x768	124	5	lossless	355.53	311.86	6.36	49	1.89	12.37	0.24	5.55	2.49	0.001	0.43	3.49	23.45
ref2va	1344x768	362	15	lossless	2339.56	2277.61	46.47	49	5.61	34.46	0.36	13.61	6.83	0.001	1.11	3.49	36.28
ref2va	1344x768	124	5	lossy	306.82	247.72	5.05	49	1.88	12.71	0.24	5.24	2.45	0.001	0.54	2.63	22.57
ref2va	1344x768	362	15	lossy	1695.26	1634.58	33.35	49	5.57	33.27	0.36	13.51	6.96	0.001	1.22	2.63	35.58

These numbers describe the validated shapes rather than a general throughput guarantee.

Known limitations

• Task serving is partitioned by --task-type: T2VA/FL2VA and Ref2VA load different DiTs, so switching between them requires a restart with the other partition.
• H3 currently executes one generation request per diffusion batch.
• The first regional-compile request is a warmup and should not be included in steady-state performance measurements.
• The official H3 input matrix and media limits are documented in the GPU recipe; this NPU path uses the same HTTP request contract.
• VAE patch parallelism requires size 1 or the full DiT group size and supports the H3 native tile mode only.
• The lossy configuration (RainFusion + INT8) is validated for T2VA and Ref2VA; use the lossless configuration for FL2VA or re-validate first.

Additional resources

• MiniMax-H3.md — full GPU guide
• MiniMax-H3-NPU-950PR.md — four-card Ascend 950PR / 950DT guide
• NPU installation
• RainFusion attention — block-sparse knobs and tuning
• Online quantization — INT8 / MXFP8 / MXFP4
• Cache-DiT guide
• Supported models
• Video API