[Hardware][Ascend] Add FastH3 four-step deployment to MiniMax-H3 NPU … (#7149)

Signed-off-by: kinnara <kinnara1989@gmail.com>
2026-09-27 17:34:59 +00:00
1 parent 1b97ff1188
commit 817f5d0e4d
@@ -281,6 +281,186 @@ curl -sS -X POST "${API_URL}" \
  -o t2va.mp4
 ```

+This requires the FLASH_ATTN backend and MindIE-SD. H3 automatically applies
+exact power-of-two input pre-scaling (`laser_input_scale=256`) so the
+kernel's fp16 workspace cannot overflow on outlier activations. Measured on
+the FastH3 four-step Dense configuration below, T2VA 15 s at 1344x768, this
+kernel reduced end-to-end latency from ~73 s to ~57 s (about 22%).
+
+
+## FastH3 four-step on A3
+
+[FastH3](https://haoailab.com/blogs/fasth3-preview/) is FastVideo's four-step
+DMD2 student of H3-Base; see
+[MiniMax-H3.md § FastH3 adapter](MiniMax-H3.md#fasth3-adapter) for the adapter
+contract. On A3 the adapter cannot be fused the GPU way. The GPU recipe fuses
+it at load time from `--lora-path`, but that path replicates the full model per
+rank and does not fit in 64 GB HBM. A3 needs distributed layerwise offload, and
+offload is refused with `--lora-path` because it streams weights in without
+going through the fusion (see the GPU recipe's note on why
+`--enable-distributed-layerwise-offload` fails fast with a FastH3 adapter).
+
+The A3 path therefore fuses the adapter **offline**, once, into a native-layout
+checkpoint that distributed layerwise offload can memory-map directly. This
+also lets the offloaded server start without `--lora-path`, so the fusion
+contract check that rejects offload never fires.
+
+### Prepare the fused checkpoint (one-time, offline)
+
+The fusion runs in the vLLM-Omni native namespace
+(`blocks.N.attn.qkv_proj.weight`), matching the base H3 `FL2VA` transformer
+layout. It reads the base transformer and the Dense adapter, adds the adapter's
+low-rank and full-rank deltas per shard, symlinks the unchanged components
+(text encoder, VAEs, tokenizer, processor), and writes the four-step sigma
+ladder into `model_index.json` so the pipeline samples on the release's rungs
+rather than a uniform schedule.
+
+Download the base checkpoint's `FL2VA` partition (native layout) and the Dense
+adapter:
+
+```bash
+export BASE_DIR=/path/to/MiniMax-H3
+hf download MiniMaxAI/MiniMax-H3 --include "FL2VA/*" --local-dir "${BASE_DIR}"
+
+export FASTH3_DIR=/path/to/fasth3
+hf download FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA \
+  dense-datafree/adapter_model.safetensors --local-dir "${FASTH3_DIR}"
+```
+
+> The published full checkpoint
+> `FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree` is stored in the
+> diffusers layout (`transformer_blocks.N.attn.to_q/to_k/to_v`, split QKV,
+> value-first MLP). Distributed layerwise offload memory-maps by exact runtime
+> parameter name and has no diffusers-to-native remap for H3, so that artifact
+> cannot be served directly. Fuse from the base `FL2VA` partition instead — its
+> keys already match the runtime.
+
+Run the fusion:
+
+```python
+# prepare_fasth3_dense.py — fuse base H3 FL2VA + FastH3 Dense adapter (native layout)
+import json, os, shutil, sys, time
+import torch
+from safetensors.torch import load_file, save_file
+from vllm_omni.diffusion.models.minimax_h3.fasth3 import FastH3WeightFusion
+
+BASE = os.environ["BASE_DIR"] + "/FL2VA"          # base H3 FL2VA partition (native layout)
+ADAPTER = os.environ["FASTH3_DIR"] + "/dense-datafree/adapter_model.safetensors"
+OUT = os.environ.get("FUSED_DIR", "/path/to/FastH3-Dense-Fused/FL2VA")
+
+# MiniMax-H3 architecture
+HEAD_DIM, NUM_BLOCKS, NUM_REFINER_BLOCKS = 128, 50, 2
+# FastH3 four-step sigma positions; the pipeline adds per-modality shift on top.
+BASE_SCHEDULE = [0.999, 0.749, 0.5, 0.25, 0.0]
+COMPONENTS = ["audio_vae", "video_vae", "text_encoder", "tokenizer", "processor"]
+
+fusion = FastH3WeightFusion.from_path(
+    ADAPTER, head_dim=HEAD_DIM, num_blocks=NUM_BLOCKS,
+    num_refiner_blocks=NUM_REFINER_BLOCKS)
+assert fusion is not None, "adapter not recognized as FastH3"
+
+os.makedirs(OUT + "/transformer", exist_ok=True)
+src_t = BASE + "/transformer"
+for shard in sorted(f for f in os.listdir(src_t)
+                    if f.endswith(".safetensors") and "index" not in f):
+    data = load_file(os.path.join(src_t, shard), device="cpu")
+    fused = {}
+    for k, v in data.items():
+        fv = fusion.fuse(k, v)
+        fused[k] = fv.to("cpu").to(torch.bfloat16) if fv is not v else v
+    save_file(fused, os.path.join(OUT, "transformer", shard))
+fusion.validate_fully_applied()   # every delta must have met its parameter
+
+for j in os.listdir(src_t):       # copy transformer index/config
+    if j.endswith(".json"):
+        shutil.copy2(os.path.join(src_t, j), os.path.join(OUT, "transformer", j))
+
+for c in COMPONENTS:              # symlink unchanged components
+    s = os.path.join(BASE, c)
+    if os.path.exists(s):
+        os.symlink(os.path.realpath(s), os.path.join(OUT, c))
+
+idx = json.loads(open(BASE + "/model_index.json").read())   # inject sigma ladder
+idx.setdefault("_minimax_h3", {})["base_schedule"] = BASE_SCHEDULE
+open(OUT + "/model_index.json", "w").write(json.dumps(idx, indent=4) + "\n")
+print("fused ->", OUT)
+```
+
+```bash
+export FUSED_DIR=/path/to/FastH3-Dense-Fused/FL2VA
+python prepare_fasth3_dense.py
+```
+
+### Serve the fused checkpoint
+
+Serve the fused directory with the same multi-NPU flags as the base recipe, but
+**without** `--lora-path` — the adapter is already in the weights. Keep
+distributed layerwise offload and, optionally, LaserAttention:
+
+```bash
+export ASCEND_RT_VISIBLE_DEVICES=0,1,2,3,4,5,6,7
+export PORT=9098
+export MODEL=/path/to/FastH3-Dense-Fused/FL2VA
+export VLLM_WORKER_MULTIPROC_METHOD=spawn
+export VLLM_OMNI_VIDEO_SYNC_TIMEOUT=1800
+export MINDIE_SD_FA_TYPE="ascend_laser_attention"   # optional
+
+vllm serve "${MODEL}" \
+  --omni \
+  --host 0.0.0.0 \
+  --port "${PORT}" \
+  --trust-remote-code \
+  --task-type t2va \
+  --init-timeout 1800 \
+  --stage-init-timeout 1800 \
+  --num-gpus 8 \
+  --usp 8 \
+  --ring 1 \
+  --text-encoder-tp-size 8 \
+  --enable-distributed-layerwise-offload \
+  --vae-parallel-mode tile \
+  --vae-use-tiling \
+  --vae-patch-parallel-size 8 \
+  --diffusion-attention-backend FLASH_ATTN
+```
+
+Requests must ask for `num_inference_steps=4` and `task=t2va`; FastH3 preview v1
+distills the T2VA path only. The first request includes regional compilation
+(~50 s warmup); exclude it from steady-state numbers.
+
+```bash
+curl -s -X POST "http://localhost:${PORT}/v1/videos/sync" \
+  -F 'prompt=A golden retriever running through a sunflower field at sunset' \
+  -F 'seconds=5' \
+  -F 'aspect_ratio=16:9' \
+  -F 'num_inference_steps=4' \
+  -o out.mp4
+```
+
+### Measured FastH3 four-step evidence
+
+Measured on an Atlas 800I A3 server (8x NPU) with CANN 9.0.1, PyTorch
+2.10.0+cpu, torch_npu 2.10.0.post2, the multi-NPU configuration above, T2VA at
+1344x768, one warmup excluded:
+
+| Duration | LaserAttention | End-to-end |
+| ---: | --- | ---: |
+| 4 s | on | ~12 s (768x768), ~15 s (1344x768) |
+| 8 s | on | ~26 s |
+| 15 s | on | ~57 s |
+| 5 s | off | ~19 s |
+| 10 s | off | ~41 s |
+| 15 s | off | ~73 s |
+
+These describe the validated shapes rather than a general throughput guarantee.
+
+> A native `--lora-path + --enable-distributed-layerwise-offload` path that
+> fuses during the offload memory-map (avoiding the offline step) is possible
+> through the offload backend's per-tensor transform hook, but is left as future
+> work pending upstream design discussion.
+
+
+## HTTP API examples
 Switch tasks via `extra_params.task` (`t2va` / `fl2va` / `ref2va`); FL2VA and
 Ref2VA additionally pass `input_references` (first-frame image, reference
 video/audio). The full parameter table lives in the GPU recipe.