[Feature] Support MiniMax-H3 FlashGen native LoRA with the legacy manager (#6666)
Signed-off-by: Zijun Huang <zijunhua@usc.edu> Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
This commit is contained in:
1 parent
143ccc5a2f
commit
08cdee8dc4
8 files changed
+1389
-6
No files matched your search
@@ -860,6 +860,63 @@ fixed HBM usage. Model-level and standard layerwise offload remain unsupported.
|
||||
The five requested sigma points produce the four denoiser evaluations expected
|
||||
by the Turbo artifact.
|
||||
|
||||
### FlashGen native LoRA
|
||||
|
||||
The FlashGen 4-step T2VA artifact uses the native MiniMax-H3 module layout and
|
||||
declares its distilled sigma schedule in safetensors metadata. It is published on
|
||||
[ModelScope](https://modelscope.cn/models/FlashGen/Minimax-H3-4step-lora-flashgen):
|
||||
|
||||
```text
|
||||
FlashGen/Minimax-H3-4step-lora-flashgen/minimax_h3_t2va_flashgen_4step_v1.0_768p_bf16.safetensors
|
||||
```
|
||||
|
||||
Download only that file:
|
||||
|
||||
```bash
|
||||
python -m pip install modelscope
|
||||
export FLASHGEN_DIR=/path/to/minimax-h3-flashgen-lora
|
||||
export FLASHGEN_FILE=minimax_h3_t2va_flashgen_4step_v1.0_768p_bf16.safetensors
|
||||
modelscope download FlashGen/Minimax-H3-4step-lora-flashgen \
|
||||
--local_dir "${FLASHGEN_DIR}" \
|
||||
--include "${FLASHGEN_FILE}"
|
||||
export FLASHGEN_LORA="${FLASHGEN_DIR}/${FLASHGEN_FILE}"
|
||||
```
|
||||
|
||||
Start from a non-offloaded or DLO FL2VA server command and add
|
||||
`--task-type fl2va --lora-backend peft --lora-path "${FLASHGEN_LORA}"`.
|
||||
Each request must use T2VA and the distilled interval-count contract:
|
||||
|
||||
```bash
|
||||
-F 'num_inference_steps=4' \
|
||||
-F 'extra_params={"task":"t2va","duration":5.2}' \
|
||||
-F "lora={\"name\":\"h3-flashgen-v1.0\",\"path\":\"${FLASHGEN_LORA}\",\"scale\":1.0}"
|
||||
```
|
||||
|
||||
This path rejects Ref2VA and checkpoints that already pin `base_schedule` in
|
||||
`model_index.json`. The adapter metadata carries
|
||||
`base_schedule=1.0,0.7,0.4,0.15,0.0`, so `num_inference_steps=4` means four
|
||||
denoiser evaluations, not five sigma points. Request-mode generation may omit
|
||||
the field and take the count from the adapter schedule; `--step-execution`
|
||||
requires it explicitly, because the step scheduler reads the total step count
|
||||
off the request at admission, before the adapter schedule is known.
|
||||
|
||||
DLO is supported in request-mode generation on the same terms as the Turbo
|
||||
adapter: the request-switchable LoRA A/B buffers stay resident on the
|
||||
accelerator while DLO streams only the base blocks, so budget for that
|
||||
additional fixed HBM usage. The native artifact is rank 64 over 259 target
|
||||
modules, and its packed `qkv_proj` and `fc1` layers reuse the full-input A
|
||||
tensor per slice while B carries slice-local output rows, so the resident
|
||||
footprint exceeds the on-disk payload; measure it for your parallel layout
|
||||
rather than assuming the checkpoint size. Pure Ulysses replicates the adapter
|
||||
on every rank, while DiT tensor parallelism shards the B buffers. Model-level
|
||||
and standard layerwise offload remain unsupported, and `--step-execution`
|
||||
cannot be combined with `--enable-distributed-layerwise-offload`.
|
||||
|
||||
To validate a deployment, post the same fixed-seed T2VA request twice with the
|
||||
adapter and twice without it, then compare the four output digests. The adapter
|
||||
is bound and deterministic when each pair matches internally and the two pairs
|
||||
differ from each other.
|
||||
|
||||
## Key parameters
|
||||
|
||||
| Parameter | Recommended value | Notes |
|
||||
|
||||
Reference in new issue
Block a user