[Feature][MiniMax-H3] Support the full LightX2V Turbo LoRA matrix (#7062)

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
汪志鹏andClaude Opus 5 authored and GitHub committed 2026-09-06 20:23:36 +08:00
1 parent 039808e0d9
commit af74a5a155
10 files changed
+557 -141

No files matched your search

+8 -2
View File
@@ -249,8 +249,14 @@ MiniMax-H3 supports two few-step mechanisms that must not be conflated:
- **Checkpoint-pinned schedule**: a merged release writes `base_schedule` into
`model_index.json`. Requests must pass `num_inference_steps` as the interval
count (for example `4` for `[1.0, 0.7, 0.4, 0.15, 0.0]`).
- **Runtime Turbo LoRA**: LightX2V Turbo artifacts request five sigma points and
enforce `flow_shift=6`, `audio_flow_shift=3`.
- **Runtime Turbo LoRA**: each LightX2V Turbo artifact carries its own contract,
read from its filename. A four-step artifact requests five sigma points and an
eight-step one requests nine; the 768p retrains enforce `flow_shift=6` and the
544p artifacts `flow_shift=12`. `audio_flow_shift=3` across the family. A
request that does not match the loaded artifact is rejected by name. Alpha
comes from the artifact's metadata, or 8 when it declares none. `ref2v`
artifacts require `--task-type ref2va`; `fl2v` artifacts serve T2VA and FL2VA.
The ComfyUI exports use a fused-QKV layout and are not supported.
- **Runtime native LoRA**: FlashGen-style artifacts declare
`key_format=minimax-h3-native` and embed `base_schedule` in safetensors
metadata. When active, the adapter schedule overrides the base checkpoint and
+13 -5
View File
@@ -68,16 +68,24 @@ The `/v1/videos` request schema and `extra_params.task` values (`t2va`,
## Turbo LoRA
MiniMax-H3 Turbo uses five sigma points, `flow_shift=6`, and
`audio_flow_shift=3`. Start Turbo deployments with the dedicated defaults so
requests that omit sampling controls do not inherit the 50-step base schedule:
The deploy config below carries the four-step 768p contract -- five sigma
points, `flow_shift=6`, `audio_flow_shift=3` -- so requests that omit sampling
controls do not inherit the 50-step base schedule:
```bash
vllm-omni serve MiniMaxAI/MiniMax-H3 \
--omni \
--lora-path /path/to/MiniMax-H3-Turbo \
--lora-path /path/to/minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors \
--deploy-config vllm_omni/deploy/minimax_h3_disaggregated_turbo.yaml
```
`--lora-path` must name one artifact, or a directory holding exactly one; the
Turbo repository holds several and a directory of them is rejected as
ambiguous. Serving a different artifact means overriding those defaults with
that artifact's own contract -- see the
[Turbo LoRA table](MiniMax-H3.md#turbo-lora) for every published row.
The base deployment intentionally retains 50 inference steps for non-LoRA
quality. Turbo LoRA supports T2VA and FL2VA requests, not Ref2VA.
quality. This disaggregated topology is validated on the `fl2v` artifacts,
which serve T2VA and FL2VA; the `ref2v` artifacts need a `--task-type ref2va`
server.
+70 -12
View File
@@ -869,13 +869,51 @@ balanced switch order.
### Turbo LoRA
Only the native Diffusers 4-step FL2VA/T2VA v1.0 artifact is supported:
The eight Diffusers-layout LightX2V Turbo artifacts are supported. The
filename records the contract, so the server reads the step count, task family
and flow shift from it and validates each request against the artifact that is
loaded. It does not rewrite request or deploy-config sampling values: the
request must carry that artifact's own settings, listed here, or it is
rejected.
```text
lightx2v/Minimax-h3-Turbo/minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors
```
| Artifact | Task | Forwards | `num_inference_steps` | `flow_shift` | declared `alpha` |
| --- | --- | ---: | ---: | ---: | ---: |
| `minimax_h3_fl2v_turbo_4step_v0.1.safetensors` | T2VA / FL2VA | 4 | 5 | 12 | none -> 8 |
| `minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors` | T2VA / FL2VA | 4 | 5 | 6 | 128 |
| `minimax_h3_fl2v_turbo_4step_v1.1_768p_bf16.safetensors` | T2VA / FL2VA | 4 | 5 | 6 | 128 |
| `minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors` | T2VA / FL2VA | 4 | 5 | 6 | 8 |
| `minimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors` | T2VA / FL2VA | 8 | 9 | 12 | 8 |
| `minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors` | T2VA / FL2VA | 8 | 9 | 6 | 8 |
| `minimax_h3_ref2v_turbo_4step_v0.1_bf16.safetensors` | Ref2VA | 4 | 5 | 12 | 8 |
| `minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors` | Ref2VA | 8 | 9 | 6 | 8 |
Download only that file:
`audio_flow_shift` is `3.0` across the family. Each row is the complete
published filename; use it verbatim as `TURBO_FILE` below.
Alpha needs no manual compensation: the server reads it from the artifact's
metadata, falling back to 8 with a warning for
`minimax_h3_fl2v_turbo_4step_v0.1`, the one artifact that declares none. The
request-level `scale` is a further multiplier on top of it.
> [!NOTE]
> Every artifact is rank 128 and the delta is applied at `scale * alpha /
> rank`, so the `alpha=8` rows drive at 1/16 the strength of the `alpha=128`
> rows. 8 is the default of LightX2V's reference script, which never reads the
> metadata; its documented `v0.1` command does not override that default.
Every artifact except `minimax_h3_fl2v_turbo_4step_v0.1` also ships a
`_comfyui_` export of the same weights. Those fuse Q/K/V into one projection
and are **not** supported; downloading one is refused by name. Take the
Diffusers file. The filename is the contract, so do not rename an artifact
either -- a renamed file is rejected rather than served on a guess.
FL2VA artifacts serve `t2va` and `fl2va` on any FL2VA or combined server.
Ref2VA artifacts require
`--task-type ref2va`: a combined server serves `ref2va` from a second DiT that
the adapter cannot bind to, so loading one there is refused rather than silently
running an undistilled model on the few-step schedule.
Download the artifact you want:
```bash
export TURBO_DIR=/path/to/minimax-h3-turbo
@@ -884,10 +922,19 @@ hf download lightx2v/Minimax-h3-Turbo "${TURBO_FILE}" --local-dir "${TURBO_DIR}"
export TURBO_LORA="${TURBO_DIR}/${TURBO_FILE}"
```
`--lora-path` accepts one artifact, or a directory holding exactly one.
> [!IMPORTANT]
> This changes earlier behaviour. `--lora-path /path/to/minimax-h3-turbo`
> pointing at a full clone of the Turbo repository used to select the v1.0 768p
> file implicitly; a directory holding several recognized artifacts is now
> rejected as ambiguous. Name the artifact instead:
> `--lora-path /path/to/minimax-h3-turbo/minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors`.
Start from a non-offloaded or DLO FL2VA server command and add
`--task-type fl2va --lora-backend peft --lora-path "${TURBO_LORA}"`.
`--lora-path` preloads the adapter; each request still activates it and uses
the published sampling settings:
`--lora-path` preloads the adapter; each request still activates it and must
carry that artifact's sampling settings:
```bash
-F 'num_inference_steps=5' \
@@ -896,14 +943,25 @@ the published sampling settings:
-F "lora={\"name\":\"h3-turbo-v1.0\",\"path\":\"${TURBO_LORA}\",\"scale\":1.0}"
```
For FL2VA, change `task` and add `input_reference` as shown above. The 8-step,
ComfyUI, Ref2VA, and v1.1 artifacts are not supported. This integration is
dynamic-only and does not support prefusion or LoRA composition. DLO is
Switching to another FL2VA artifact means repointing `TURBO_FILE`, which moves
both `--lora-path` and the request's `lora.path`, and carrying that row's
`num_inference_steps` and `flow_shift`: `9` and `6` for `8step_v1.0_768p`, `9`
and `12` for the 544p `8step_v1.0`. A request that does not match the loaded
artifact is rejected, so a mismatch cannot silently degrade output.
The two `ref2v` rows are not served by this FL2VA command. Start a
`--task-type ref2va` server, take a request from
[Ref2VA](#3-ref2va-image-only-imageaudio-or-mixed-references) and override
`num_inference_steps` and `flow_shift` with that row's values; those examples
already send `audio_flow_shift=3.0`.
For FL2VA, change `task` and add `input_reference` as shown above. This
integration is dynamic-only and does not support prefusion or LoRA composition. DLO is
supported by keeping the request-switchable LoRA A/B buffers resident on the
accelerator while DLO streams only the base blocks; budget for this additional
fixed HBM usage. Model-level and standard layerwise offload remain unsupported.
The five requested sigma points produce the four denoiser evaluations expected
by the Turbo artifact.
The requested sigma points always number one more than the artifact's denoiser
evaluations.
### FlashGen native LoRA
@@ -13,6 +13,8 @@ import pytest
import torch
import torch.nn as nn
from vllm_omni.diffusion.models.minimax_h3.lora import TurboSpec, parse_turbo_filename
pytestmark = [pytest.mark.core_model, pytest.mark.cpu, pytest.mark.diffusion]
_ItemT = TypeVar("_ItemT")
@@ -24,6 +26,13 @@ def _append_and_return(items: list[_ItemT], item: _ItemT, result: _ResultT) -> _
return result
def _turbo_spec(filename: str) -> TurboSpec:
"""Build the spec a loaded artifact of this name would carry."""
spec = parse_turbo_filename(filename)
assert spec is not None
return spec
def test_h3_prepares_resolved_cache_state_immediately_before_denoise():
from vllm_omni.diffusion.models.minimax_h3 import MiniMaxH3Pipeline
from vllm_omni.diffusion.request import OmniDiffusionRequest
@@ -244,9 +253,12 @@ def test_combined_task_inference_and_transformer_routing():
assert pipeline._resolve_task(None, {"image": object()}) == "fl2va"
assert pipeline._resolve_task(None, {"audio": object()}) == "ref2va"
assert pipeline._resolve_task(None, {"video": object()}) == "ref2va"
assert pipeline._resolve_task("fl2va", {"image": object()}, has_turbo_lora=True) == "fl2va"
with pytest.raises(OmniClientError, match="supports T2VA/FL2VA requests only"):
pipeline._resolve_task("ref2va", {}, has_turbo_lora=True)
fl2v_spec = _turbo_spec("minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors")
ref2v_spec = _turbo_spec("minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors")
assert pipeline._resolve_task("fl2va", {"image": object()}, turbo_spec=fl2v_spec) == "fl2va"
with pytest.raises(OmniClientError, match="serves"):
pipeline._resolve_task("ref2va", {}, turbo_spec=fl2v_spec)
assert pipeline._resolve_task("ref2va", {"image": object()}, turbo_spec=ref2v_spec) == "ref2va"
pipeline.partition = "ref2va"
pipeline.supported_tasks = frozenset({"ref2va"})
assert pipeline._resolve_task(None, {"image": object()}) == "ref2va"
@@ -303,7 +303,7 @@ def _pipeline_stub(fusion=None):
torch.nn.Module.__init__(pipeline)
pipeline.partition = "combined"
pipeline.supported_tasks = frozenset({"t2va", "fl2va", "ref2va"})
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._fasth3 = fusion
pipeline.od_config = SimpleNamespace()
pipeline.default_video_shift, pipeline.default_audio_shift = 12.0, 3.0
@@ -13,7 +13,9 @@ from vllm.lora.lora_weights import PackedLoRALayerWeights
from vllm_omni.diffusion.lora.manager import DiffusionLoRAManager
from vllm_omni.diffusion.models.minimax_h3 import lora as lora_module
from vllm_omni.diffusion.models.minimax_h3.lora import (
TurboSpec,
load_minimax_h3_turbo_lora,
parse_turbo_filename,
)
from vllm_omni.errors import OmniClientError
from vllm_omni.lora.request import LoRARequest
@@ -46,7 +48,7 @@ def _request(path) -> LoRARequest:
def _write_tiny_turbo(
path,
*,
alpha: str = "128",
alpha: str | None = "128",
key_format: str = "minimax-h3-diffusers",
omit_target: str | None = None,
shape_overrides: dict[str, tuple[int, int]] | None = None,
@@ -88,11 +90,17 @@ def _write_tiny_turbo(
)
else:
tensors[b_name] = torch.ones(overrides.get(b_name, (output_dim, rank)))
save_file(
tensors,
str(path),
metadata={"alpha": alpha, "key_format": key_format},
)
metadata = {"key_format": key_format}
if alpha is not None:
metadata["alpha"] = alpha
save_file(tensors, str(path), metadata=metadata)
def _spec(filename: str = "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors") -> TurboSpec:
"""Build the spec a loaded artifact of this name would carry."""
spec = parse_turbo_filename(filename)
assert spec is not None
return spec
def test_h3_turbo_loads_through_legacy_lora_model_and_swaps_ffn(tmp_path):
@@ -107,9 +115,12 @@ def test_h3_turbo_loads_through_legacy_lora_model_and_swaps_ffn(tmp_path):
)
assert loaded is not None
lora_model, peft_helper = loaded
lora_model, peft_helper, spec = loaded
assert peft_helper.r == 128
assert peft_helper.lora_alpha == 128
assert spec.denoise_steps == 4
assert spec.sigma_points == 5
assert spec.task_family == "fl2v"
assert len(lora_model.loras) == 312
assert "blocks.0.attn.to_q" in lora_model.loras
assert "blocks.0.mlp.fc1" in lora_model.loras
@@ -152,7 +163,8 @@ def test_legacy_manager_uses_the_h3_model_loader_without_changing_its_interface(
class _Pipeline:
def _load_diffusion_lora_adapter(self, **kwargs):
return load_minimax_h3_turbo_lora(partition="fl2va", **kwargs)
lora_model, peft_helper, _spec_unused = load_minimax_h3_turbo_lora(partition="fl2va", **kwargs)
return lora_model, peft_helper
manager = object.__new__(DiffusionLoRAManager)
manager.pipeline = _Pipeline()
@@ -168,22 +180,30 @@ def test_legacy_manager_uses_the_h3_model_loader_without_changing_its_interface(
assert "blocks.0.mlp.fc1" in lora_model.loras
def test_h3_turbo_rejects_wrong_alpha_and_ref2va(tmp_path):
wrong_alpha = tmp_path / "wrong_alpha" / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors"
wrong_alpha.parent.mkdir()
_write_tiny_turbo(wrong_alpha, alpha="8")
with pytest.raises(ValueError, match="requires alpha=128"):
load_minimax_h3_turbo_lora(
partition="fl2va",
lora_request=_request(wrong_alpha),
lora_path=wrong_alpha,
dtype=torch.float32,
)
def test_h3_turbo_honours_the_declared_alpha_and_task_family(tmp_path):
"""Alpha comes from the artifact; the family decides which server serves it."""
eight_step_alpha = tmp_path / "alpha8" / "minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors"
eight_step_alpha.parent.mkdir()
_write_tiny_turbo(eight_step_alpha, alpha="8")
loaded = load_minimax_h3_turbo_lora(
partition="fl2va",
lora_request=_request(eight_step_alpha),
lora_path=eight_step_alpha,
dtype=torch.float32,
)
assert loaded is not None
_, peft_helper, spec = loaded
# The eight-step artifact is a 1/16-strength adapter; pinning alpha to the
# four-step value would over-drive it by 16x.
assert peft_helper.lora_alpha == 8
assert spec.alpha == 8
assert spec.sigma_points == 9
valid = tmp_path / "valid" / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors"
valid.parent.mkdir()
_write_tiny_turbo(valid)
with pytest.raises(ValueError, match="supports FL2VA/T2VA only"):
with pytest.raises(ValueError, match="Ref2VA-only server cannot serve it"):
load_minimax_h3_turbo_lora(
partition="ref2va",
lora_request=_request(valid),
@@ -192,6 +212,43 @@ def test_h3_turbo_rejects_wrong_alpha_and_ref2va(tmp_path):
)
@pytest.mark.parametrize("partition", ["fl2va", "combined"])
def test_h3_turbo_ref2v_artifact_needs_a_ref2va_only_server(tmp_path, partition):
"""A combined server routes ref2va to ``transformers_ref``, which the
adapter cannot bind to: the LoRA target pattern only injects into
``transformer``. Admitting it there would run an undistilled DiT on the
artifact's few-step schedule with no error anywhere."""
path = tmp_path / "minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors"
_write_tiny_turbo(path)
with pytest.raises(ValueError, match="start the server with --task-type ref2va"):
load_minimax_h3_turbo_lora(
partition=partition,
lora_request=_request(path),
lora_path=path,
dtype=torch.float32,
)
def test_h3_turbo_ref2v_artifact_loads_on_a_ref2va_server(tmp_path):
path = tmp_path / "minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors"
_write_tiny_turbo(path)
loaded = load_minimax_h3_turbo_lora(
partition="ref2va",
lora_request=_request(path),
lora_path=path,
dtype=torch.float32,
)
assert loaded is not None
spec = loaded[2]
assert spec.task_family == "ref2v"
assert spec.supported_tasks == frozenset({"ref2va"})
assert spec.sigma_points == 9
@pytest.mark.parametrize(
"offload_mode",
[
@@ -214,6 +271,8 @@ def test_h3_turbo_rejects_offload_modes(tmp_path, offload_mode):
def test_h3_turbo_allows_distributed_layerwise_offload(monkeypatch):
"""DLO keeps the LoRA buffers resident, so it is not an unsupported mode."""
from vllm_omni.diffusion.models.minimax_h3 import MiniMaxH3Pipeline
from vllm_omni.diffusion.models.minimax_h3 import (
pipeline_minimax_h3 as pipeline_module,
@@ -222,7 +281,7 @@ def test_h3_turbo_allows_distributed_layerwise_offload(monkeypatch):
pipeline = object.__new__(MiniMaxH3Pipeline)
torch.nn.Module.__init__(pipeline)
pipeline.partition = "fl2va"
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._native_lora_adapter_ids = set()
pipeline._lora_sigma_schedules = {}
pipeline.od_config = SimpleNamespace(
@@ -231,10 +290,11 @@ def test_h3_turbo_allows_distributed_layerwise_offload(monkeypatch):
enable_distributed_layerwise_offload=True,
)
captured = {}
spec = _spec()
def load_turbo(**kwargs):
captured.update(kwargs)
return object(), object()
return object(), object(), spec
monkeypatch.setattr(pipeline_module, "load_minimax_h3_turbo_lora", load_turbo)
loaded = pipeline._load_diffusion_lora_adapter(
@@ -245,6 +305,10 @@ def test_h3_turbo_allows_distributed_layerwise_offload(monkeypatch):
assert loaded is not None
assert captured["unsupported_offload_mode"] is None
# The pipeline passes its own partition through and keeps the spec that
# decides the request contract.
assert captured["partition"] == "fl2va"
assert pipeline._turbo_lora_specs == {1: spec}
@pytest.mark.parametrize(
@@ -268,7 +332,7 @@ def test_h3_turbo_offload_compatibility_follows_dit_selection(
pipeline = object.__new__(MiniMaxH3Pipeline)
torch.nn.Module.__init__(pipeline)
pipeline.partition = "fl2va"
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._native_lora_adapter_ids = set()
pipeline._lora_sigma_schedules = {}
pipeline.od_config = SimpleNamespace(
@@ -281,10 +345,11 @@ def test_h3_turbo_offload_compatibility_follows_dit_selection(
pin_cpu_memory=True,
)
captured = {}
spec = _spec()
def load_turbo(**kwargs):
captured.update(kwargs)
return object(), object()
return object(), object(), spec
monkeypatch.setattr(pipeline_module, "load_minimax_h3_turbo_lora", load_turbo)
@@ -296,31 +361,84 @@ def test_h3_turbo_offload_compatibility_follows_dit_selection(
assert captured["unsupported_offload_mode"] == expected_mode
def test_h3_turbo_accepts_only_the_declared_v1_artifact(tmp_path):
unsupported = tmp_path / "minimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors"
_write_tiny_turbo(unsupported)
def test_h3_turbo_uses_the_reference_alpha_when_the_artifact_declares_none(tmp_path):
"""``4step_v0.1`` ships no alpha. LightX2V's reference script applies
``scale * alpha / rank`` with alpha defaulting to 8, so the fallback must be
8; alpha == rank would drive the adapter 16x too strongly."""
with pytest.raises(ValueError, match="supports only"):
path = tmp_path / "minimax_h3_fl2v_turbo_4step_v0.1.safetensors"
_write_tiny_turbo(path, alpha=None)
loaded = load_minimax_h3_turbo_lora(
partition="fl2va",
lora_request=_request(path),
lora_path=path,
dtype=torch.float32,
)
assert loaded is not None
_, peft_helper, spec = loaded
assert spec.alpha == 8
assert peft_helper.lora_alpha == 8
assert spec.alpha / spec.rank == 0.0625
def test_h3_turbo_declines_the_comfyui_export(tmp_path):
"""The ComfyUI exports fuse Q/K/V, so they are not this loader's format."""
path = tmp_path / "minimax_h3_fl2v_turbo_4step_v1.0_768p_comfyui_bf16.safetensors"
_write_tiny_turbo(path)
assert (
load_minimax_h3_turbo_lora(
partition="fl2va",
lora_request=_request(unsupported),
lora_path=unsupported,
lora_request=_request(path),
lora_path=path,
dtype=torch.float32,
)
is None
)
supported = tmp_path / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors"
_write_tiny_turbo(supported)
# Directory resolution selects the declared artifact even when another
# v1.0 checkpoint is present beside it.
assert (
def test_h3_turbo_accepts_every_published_artifact_name(tmp_path):
"""Each published filename loads and carries its own sampler contract."""
for index, (name, steps, shift) in enumerate(
(
("minimax_h3_fl2v_turbo_4step_v0.1.safetensors", 4, 12.0),
("minimax_h3_fl2v_turbo_4step_v1.1_768p_bf16.safetensors", 4, 6.0),
("minimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors", 8, 12.0),
("minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors", 8, 6.0),
)
):
directory = tmp_path / f"artifact{index}"
directory.mkdir()
path = directory / name
_write_tiny_turbo(path)
loaded = load_minimax_h3_turbo_lora(
partition="fl2va",
lora_request=_request(path),
lora_path=path,
dtype=torch.float32,
)
assert loaded is not None, name
spec = loaded[2]
assert spec.denoise_steps == steps
assert spec.sigma_points == steps + 1
assert spec.video_shift == shift
def test_h3_turbo_directory_with_several_artifacts_is_ambiguous(tmp_path):
_write_tiny_turbo(tmp_path / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors")
_write_tiny_turbo(tmp_path / "minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors")
with pytest.raises(ValueError, match="point --lora-path at one file"):
load_minimax_h3_turbo_lora(
partition="fl2va",
lora_request=_request(tmp_path),
lora_path=tmp_path,
dtype=torch.float32,
)
is not None
)
def test_h3_turbo_rejects_a_truncated_declared_artifact(tmp_path):
@@ -352,7 +470,7 @@ def test_non_h3_checkpoint_falls_back_to_the_generic_peft_loader(tmp_path):
declared = tmp_path / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors"
_write_tiny_turbo(declared, key_format="other")
with pytest.raises(ValueError, match="requires safetensors metadata"):
with pytest.raises(ValueError, match="expected 'minimax-h3-diffusers'"):
load_minimax_h3_turbo_lora(
partition="fl2va",
lora_request=_request(declared),
@@ -371,7 +489,7 @@ def test_only_an_active_recognized_turbo_adapter_restricts_ref2va(monkeypatch):
torch.nn.Module.__init__(pipeline)
pipeline.partition = "combined"
pipeline.supported_tasks = frozenset({"t2va", "fl2va", "ref2va"})
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._native_lora_adapter_ids = set()
pipeline._lora_sigma_schedules = {}
request = _request("generic-peft")
@@ -388,14 +506,14 @@ def test_only_an_active_recognized_turbo_adapter_restricts_ref2va(monkeypatch):
return pipeline._resolve_task(
"ref2va",
{},
has_turbo_lora=pipeline._has_active_turbo_lora(sampling),
turbo_spec=pipeline._active_turbo_spec(sampling),
)
recognized = (object(), object())
recognized = (object(), object(), _spec())
monkeypatch.setattr(pipeline_module, "load_minimax_h3_turbo_lora", lambda **_: recognized)
assert load() is recognized
assert load() == recognized[:2]
assert resolve(0.0) == "ref2va"
with pytest.raises(OmniClientError, match="supports T2VA/FL2VA requests only"):
with pytest.raises(OmniClientError, match="serves"):
resolve(1.0)
# Simulate manager eviction followed by a generic PEFT adapter reusing the
@@ -410,7 +528,7 @@ def test_h3_turbo_requires_all_loaded_targets_to_bind():
pipeline = object.__new__(MiniMaxH3Pipeline)
torch.nn.Module.__init__(pipeline)
pipeline._turbo_lora_adapter_ids = {1}
pipeline._turbo_lora_specs = {1: _spec()}
lora_model = SimpleNamespace(id=1, loras={"to_q": object(), "to_k": object()})
pipeline._validate_diffusion_lora_binding(
@@ -448,7 +566,7 @@ def test_h3_turbo_rejects_unsupported_sampling(num_inference_steps, extra_args,
)
with pytest.raises(OmniClientError, match=error):
pipeline._validate_turbo_sampling(sampling)
pipeline._validate_turbo_sampling(sampling, _spec())
def test_h3_turbo_accepts_five_sigma_points_for_four_nfe():
@@ -461,5 +579,6 @@ def test_h3_turbo_accepts_five_sigma_points_for_four_nfe():
SimpleNamespace(
num_inference_steps=5,
extra_args={"flow_shift": 6.0, "audio_flow_shift": 3.0},
)
),
_spec(),
)
@@ -309,7 +309,7 @@ def test_pipeline_native_schedule_and_task_validation(monkeypatch):
torch.nn.Module.__init__(pipeline)
pipeline.partition = "fl2va"
pipeline.supported_tasks = frozenset({"t2va", "fl2va", "ref2va"})
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._native_lora_adapter_ids = {7}
pipeline._lora_sigma_schedules = {7: DMD2SigmaSchedule.from_positions([1.0, 0.7, 0.4, 0.15, 0.0])}
pipeline._base_schedule_by_partition = {"fl2va": None}
@@ -330,7 +330,7 @@ def test_pipeline_native_schedule_and_task_validation(monkeypatch):
pipeline._resolve_task(
"fl2va",
{},
has_turbo_lora=False,
turbo_spec=None,
has_native_lora=True,
)
@@ -418,7 +418,7 @@ def test_pipeline_replaces_native_classification_after_reload(monkeypatch, tmp_p
enable_layerwise_offload=False,
enable_distributed_layerwise_offload=False,
)
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._native_lora_adapter_ids = set()
pipeline._lora_sigma_schedules = {}
@@ -472,7 +472,7 @@ def test_h3_native_allows_distributed_layerwise_offload(monkeypatch):
enable_layerwise_offload=False,
enable_distributed_layerwise_offload=True,
)
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._native_lora_adapter_ids = set()
pipeline._lora_sigma_schedules = {}
captured: dict[str, object] = {}
@@ -612,7 +612,7 @@ def test_legacy_manager_uses_native_loader(tmp_path):
enable_layerwise_offload=False,
enable_distributed_layerwise_offload=False,
)
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._native_lora_adapter_ids = set()
pipeline._lora_sigma_schedules = {}
return pipeline._load_diffusion_lora_adapter(**kwargs)
@@ -634,7 +634,7 @@ def test_pipeline_schedule_inactive_when_scale_zero():
pipeline = object.__new__(MiniMaxH3Pipeline)
torch.nn.Module.__init__(pipeline)
pipeline.partition = "fl2va"
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._native_lora_adapter_ids = {7}
pipeline._lora_sigma_schedules = {7: DMD2SigmaSchedule.from_positions([1.0, 0.7, 0.4, 0.15, 0.0])}
pipeline._base_schedule_by_partition = {"fl2va": None}
@@ -664,7 +664,7 @@ def test_pipeline_schedule_falls_back_after_eviction(monkeypatch, tmp_path):
enable_layerwise_offload=False,
enable_distributed_layerwise_offload=False,
)
pipeline._turbo_lora_adapter_ids = set()
pipeline._turbo_lora_specs = {}
pipeline._native_lora_adapter_ids = set()
pipeline._lora_sigma_schedules = {}
pipeline._base_schedule_by_partition = {"fl2va": None}
@@ -0,0 +1,86 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Contract coverage for the LightX2V MiniMax-H3 Turbo artifact matrix."""
from __future__ import annotations
import pytest
from vllm_omni.diffusion.models.minimax_h3.lora import (
_TURBO_AUDIO_SHIFT,
_TURBO_VIDEO_SHIFT_544P,
_TURBO_VIDEO_SHIFT_768P,
TurboSpec,
parse_turbo_filename,
)
pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, pytest.mark.cpu]
# The eight Diffusers-layout artifacts published at lightx2v/Minimax-h3-Turbo,
# with the contract each one implies. The ComfyUI export of an artifact is a
# fused-QKV re-packing of the same weights and is not served.
PUBLISHED = [
("minimax_h3_fl2v_turbo_4step_v0.1.safetensors", "fl2v", 4, _TURBO_VIDEO_SHIFT_544P),
("minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors", "fl2v", 4, _TURBO_VIDEO_SHIFT_768P),
("minimax_h3_fl2v_turbo_4step_v1.1_768p_bf16.safetensors", "fl2v", 4, _TURBO_VIDEO_SHIFT_768P),
("minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", "fl2v", 4, _TURBO_VIDEO_SHIFT_768P),
("minimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors", "fl2v", 8, _TURBO_VIDEO_SHIFT_544P),
("minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors", "fl2v", 8, _TURBO_VIDEO_SHIFT_768P),
("minimax_h3_ref2v_turbo_4step_v0.1_bf16.safetensors", "ref2v", 4, _TURBO_VIDEO_SHIFT_544P),
("minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors", "ref2v", 8, _TURBO_VIDEO_SHIFT_768P),
]
@pytest.mark.parametrize(("filename", "task", "steps", "video_shift"), PUBLISHED)
def test_every_published_artifact_is_recognised(filename: str, task: str, steps: int, video_shift: float) -> None:
spec = parse_turbo_filename(filename)
assert spec is not None, filename
assert spec.filename == filename
assert spec.task_family == task
assert spec.denoise_steps == steps
assert spec.video_shift == video_shift
assert spec.audio_shift == _TURBO_AUDIO_SHIFT
# Until the loader reads metadata, alpha is LightX2V's reference default.
assert spec.rank == 128
assert spec.alpha == 8.0
def test_all_eight_artifacts_are_covered() -> None:
assert len(PUBLISHED) == 8
assert len({name for name, *_ in PUBLISHED}) == 8
@pytest.mark.parametrize(
"filename",
[
"adapter_model.safetensors",
"minimax_h3_fl2v_turbo_v1.0_768p_bf16.safetensors",
"minimax_h3_t2v_turbo_4step_v1.0.safetensors",
"minimax_h3_fl2v_turbo_0step_v1.0.safetensors",
# The ComfyUI exports are a different tensor layout, not a variant of
# the supported one; the loader refuses them by name.
"minimax_h3_fl2v_turbo_4step_v1.0_768p_comfyui_bf16.safetensors",
"minimax_h3_ref2v_turbo_8step_v1.0_768p_comfyui_bf16.safetensors",
],
)
def test_unrelated_names_are_rejected(filename: str) -> None:
assert parse_turbo_filename(filename) is None
def _spec(filename: str) -> TurboSpec:
spec = parse_turbo_filename(filename)
assert spec is not None
return spec
def test_sigma_points_follow_the_step_count() -> None:
assert _spec("minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors").sigma_points == 5
assert _spec("minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors").sigma_points == 9
def test_task_family_decides_supported_tasks() -> None:
fl2v = _spec("minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors")
ref2v = _spec("minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors")
assert fl2v.supported_tasks == frozenset({"t2va", "fl2va"})
assert ref2v.supported_tasks == frozenset({"ref2va"})
+145 -30
View File
@@ -1,20 +1,25 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Loader for the LightX2V Turbo MiniMax-H3 LoRA.
"""Loader for the LightX2V Turbo MiniMax-H3 LoRA family.
Keyed by ``key_format=minimax-h3-diffusers``. The native FlashGen contract lives
in ``.npu.lora``; the two artifacts differ in rank, target naming and QKV
layout, so each owns its own parsing and packing.
The published artifacts differ in task family, step count, training resolution
and LoRA alpha, so :class:`TurboSpec` carries the contract of the file actually
loaded and the pipeline validates each request against it. Only the
Diffusers-PEFT exports are served. The native FlashGen contract lives in
``.npu.lora``.
"""
from __future__ import annotations
import math
from dataclasses import dataclass, replace
from pathlib import Path
import regex as re
import torch
from safetensors import safe_open
from vllm.logger import init_logger
from vllm.lora.lora_model import LoRAModel
from vllm.lora.lora_weights import PackedLoRALayerWeights
from vllm.lora.peft_helper import PEFTHelper
@@ -22,12 +27,79 @@ from vllm.model_executor.models.utils import WeightsMapper
from vllm_omni.lora.request import LoRARequest
logger = init_logger(__name__)
_TURBO_RANK = 128
_TURBO_ALPHA = 128
_TURBO_HIDDEN_SIZE = 5376
_TURBO_ATTENTION_INNER_SIZE = 7168
_TURBO_FFN_HIDDEN_SIZE = 14336
_TURBO_FILENAME = "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors"
# ``minimax_h3_<task>_turbo_<n>step_v<major.minor>[_768p][_bf16]``. The name is
# the only place the sampler contract is recorded.
_TURBO_NAME_RE = re.compile(
r"^minimax_h3_(?P<task>fl2v|ref2v)_turbo_(?P<steps>\d+)step"
r"_v\d+\.\d+(?P<res>_768p)?(?:_bf16)?\.safetensors$"
)
# The 768p retrains moved to a shorter video flow shift; the 544p artifacts keep
# the base model's.
_TURBO_VIDEO_SHIFT_768P = 6.0
_TURBO_VIDEO_SHIFT_544P = 12.0
_TURBO_AUDIO_SHIFT = 3.0
# Only ``4step_v0.1`` declares no alpha. LightX2V's reference script never reads
# the metadata: it applies ``scale * alpha / rank`` with ``--lora-alpha``
# defaulting to 8, and its documented v0.1 command does not override that. So an
# artifact that declares nothing is driven at 8, not at its rank.
_TURBO_DEFAULT_ALPHA = 8.0
@dataclass(frozen=True)
class TurboSpec:
"""The sampler contract one Turbo artifact was distilled for."""
filename: str
task_family: str
"""``fl2v`` (serves t2va and fl2va) or ``ref2v`` (serves ref2va)."""
denoise_steps: int
video_shift: float
audio_shift: float
rank: int
alpha: float
@property
def sigma_points(self) -> int:
"""Sigma points the API contract expects: one more than the forwards."""
return self.denoise_steps + 1
@property
def supported_tasks(self) -> frozenset[str]:
if self.task_family == "ref2v":
return frozenset({"ref2va"})
return frozenset({"t2va", "fl2va"})
def parse_turbo_filename(name: str) -> TurboSpec | None:
"""Return the contract a Turbo filename encodes, or ``None``.
``alpha`` is the rank-matched default; the loader replaces it with the
artifact's declared value.
"""
match = _TURBO_NAME_RE.match(name)
if match is None:
return None
steps = int(match.group("steps"))
if steps <= 0:
return None
return TurboSpec(
filename=name,
task_family=match.group("task"),
denoise_steps=steps,
video_shift=_TURBO_VIDEO_SHIFT_768P if match.group("res") else _TURBO_VIDEO_SHIFT_544P,
audio_shift=_TURBO_AUDIO_SHIFT,
rank=_TURBO_RANK,
alpha=_TURBO_DEFAULT_ALPHA,
)
_LORA_A_SUFFIX = ".lora_A.default.weight"
_LORA_B_SUFFIX = ".lora_B.default.weight"
_TURBO_TARGETS = frozenset({"to_q", "to_k", "to_v", "out_proj", "fc1", "fc2"})
@@ -81,11 +153,20 @@ def _select_turbo_file(artifact_path: str | Path) -> Path | None:
if not path.is_dir():
return None
candidate = path / _TURBO_FILENAME
return candidate if candidate.is_file() else None
candidates = sorted(child for child in path.glob("*.safetensors") if parse_turbo_filename(child.name))
if len(candidates) == 1:
return candidates[0]
if len(candidates) > 1:
raise ValueError(
f"{path} holds {len(candidates)} MiniMax-H3 Turbo artifacts "
f"({[c.name for c in candidates[:4]]}); point --lora-path at one file."
)
return None
def _validate_and_convert_tensors(checkpoint) -> dict[str, torch.Tensor]:
"""Validate a Diffusers-layout Turbo tensor set and pack it for the manager."""
tensors: dict[str, torch.Tensor] = {}
pairs: dict[str, set[str]] = {}
raw_targets: set[str] = set()
@@ -134,7 +215,7 @@ def _validate_and_convert_tensors(checkpoint) -> dict[str, torch.Tensor]:
unexpected = sorted(raw_targets - _TURBO_EXPECTED_RAW_TARGETS)
if missing or unexpected:
raise ValueError(
"MiniMax-H3 Turbo target set does not match the supported v1.0 artifact: "
"MiniMax-H3 Turbo target set does not match the published artifact layout: "
f"missing={len(missing)} {missing[:5]}, unexpected={len(unexpected)} {unexpected[:5]}"
)
return tensors
@@ -164,39 +245,61 @@ def load_minimax_h3_turbo_lora(
lora_path: str | Path,
dtype: torch.dtype,
unsupported_offload_mode: str | None = None,
) -> tuple[LoRAModel, PEFTHelper] | None:
"""Load the published LightX2V Turbo v1.0 through the legacy manager."""
) -> tuple[LoRAModel, PEFTHelper, TurboSpec] | None:
"""Load any published LightX2V Turbo artifact through the legacy manager."""
lora_file = _select_turbo_file(lora_path)
if lora_file is None:
return None
spec = parse_turbo_filename(lora_file.name)
if spec is None:
# Not a Turbo artifact by name; leave it to the next loader.
return None
with safe_open(lora_file, framework="pt", device="cpu") as checkpoint:
# Three published artifacts declare no key_format, so absence is not an
# error; a declaration naming a different format is.
metadata = checkpoint.metadata() or {}
if metadata.get("key_format") != "minimax-h3-diffusers":
if lora_file.name == _TURBO_FILENAME:
raise ValueError(
"MiniMax-H3 Turbo v1.0 requires safetensors metadata key_format='minimax-h3-diffusers'"
)
return None
if lora_file.name != _TURBO_FILENAME:
raise ValueError(f"MiniMax-H3 Turbo supports only {_TURBO_FILENAME!r}, got {lora_file.name!r}")
declared_format = metadata.get("key_format")
if declared_format not in (None, "minimax-h3-diffusers"):
raise ValueError(
f"{lora_file.name} declares key_format={declared_format!r}, expected 'minimax-h3-diffusers'"
)
raw_alpha = metadata.get("alpha")
try:
alpha = float(raw_alpha) if raw_alpha is not None else math.nan
except ValueError as exc:
raise ValueError(f"MiniMax-H3 Turbo alpha must be numeric, got {raw_alpha!r}") from exc
if alpha != _TURBO_ALPHA:
raise ValueError(f"MiniMax-H3 Turbo v1.0 requires alpha={_TURBO_ALPHA}, got {raw_alpha!r}")
if partition == "ref2va":
raise ValueError("MiniMax-H3 Turbo LoRA supports FL2VA/T2VA only")
if raw_alpha is None:
logger.warning(
"MiniMax-H3 Turbo artifact %s declares no alpha; using the LightX2V reference "
"default alpha=%g (scale %g). Override with the request-level LoRA scale.",
spec.filename,
spec.alpha,
spec.alpha / spec.rank,
)
else:
try:
spec = replace(spec, alpha=float(raw_alpha))
except (TypeError, ValueError) as exc:
raise ValueError(f"MiniMax-H3 Turbo alpha must be numeric, got {raw_alpha!r}") from exc
if not math.isfinite(spec.alpha) or spec.alpha <= 0:
raise ValueError(f"MiniMax-H3 Turbo alpha must be a positive number, got {raw_alpha!r}")
if partition == "ref2va" and spec.task_family == "fl2v":
raise ValueError(f"{spec.filename} is an FL2VA/T2VA Turbo artifact; a Ref2VA-only server cannot serve it.")
# ``combined`` serves ref2va from ``transformers_ref``, but the LoRA
# target pattern only injects into ``transformer``: the adapter would
# bind to the stack that never runs.
if spec.task_family == "ref2v" and partition != "ref2va":
raise ValueError(
f"{spec.filename} is a Ref2VA Turbo artifact; start the server with --task-type ref2va "
f"(task_type={partition!r} serves ref2va from a DiT the adapter cannot bind to)"
)
if unsupported_offload_mode is not None:
raise ValueError(f"MiniMax-H3 Turbo dynamic LoRA does not support {unsupported_offload_mode}")
tensors = _validate_and_convert_tensors(checkpoint)
peft_helper = PEFTHelper.from_dict(
{
"r": _TURBO_RANK,
"lora_alpha": _TURBO_ALPHA,
"r": spec.rank,
"lora_alpha": spec.alpha,
"target_modules": _TURBO_TARGET_PATTERN,
}
)
@@ -209,4 +312,16 @@ def load_minimax_h3_turbo_lora(
weights_mapper=_TURBO_WEIGHTS_MAPPER,
)
_pack_h3_turbo_fc1(lora_model)
return lora_model, peft_helper
logger.info(
"MiniMax-H3 Turbo adapter %s: task=%s, %d denoiser forwards (%d sigma points), "
"rank=%d alpha=%g, flow_shift=%g/%g",
spec.filename,
spec.task_family,
spec.denoise_steps,
spec.sigma_points,
spec.rank,
spec.alpha,
spec.video_shift,
spec.audio_shift,
)
return lora_model, peft_helper, spec
@@ -125,7 +125,7 @@ from .denoise_loop import (
)
from .encoder import MiniMaxH3Qwen3VLEncoder
from .fasth3 import FastH3WeightFusion, resolve_fasth3_fusion
from .lora import load_minimax_h3_turbo_lora
from .lora import TurboSpec, load_minimax_h3_turbo_lora
from .minimax_h3_transformer import (
MiniMaxH3Attention,
MiniMaxH3DiTModel,
@@ -188,9 +188,6 @@ MINIMAX_H3_MAX_REFERENCE_IMAGE_BYTES = 30 * 1024 * 1024
MINIMAX_H3_REFERENCE_IMAGE_FORMATS = frozenset({"jpeg", "png", "webp", "heic", "heif"})
MINIMAX_H3_MIN_OUTPUT_SECONDS = 4.0
MINIMAX_H3_MAX_OUTPUT_SECONDS = 15.0
MINIMAX_H3_TURBO_SIGMA_POINTS = 5
MINIMAX_H3_TURBO_VIDEO_SHIFT = 6.0
MINIMAX_H3_TURBO_AUDIO_SHIFT = 3.0
MINIMAX_H3_DOWNLOAD_PATTERNS = [
"FL2VA/**",
"Ref2VA/model_index.json",
@@ -706,7 +703,7 @@ class MiniMaxH3Pipeline(
) -> tuple[LoRAModel, PEFTHelper] | None:
# A cache eviction may be followed by a different adapter reusing the
# same client-supplied ID. Every real load replaces the classification.
self._turbo_lora_adapter_ids.discard(lora_request.lora_int_id)
self._turbo_lora_specs.pop(lora_request.lora_int_id, None)
self._native_lora_adapter_ids.discard(lora_request.lora_int_id)
self._lora_sigma_schedules.pop(lora_request.lora_int_id, None)
od_config = getattr(self, "od_config", None)
@@ -726,8 +723,9 @@ class MiniMaxH3Pipeline(
unsupported_offload_mode=" or ".join(offload_modes) or None,
)
if loaded is not None:
self._turbo_lora_adapter_ids.add(lora_request.lora_int_id)
return loaded
lora_model, peft_helper, turbo_spec = loaded
self._turbo_lora_specs[lora_request.lora_int_id] = turbo_spec
return lora_model, peft_helper
# Selection is by the artifact's safetensors ``key_format``, not by the
# running platform: the native loader is checkpoint-format parsing with
@@ -753,7 +751,7 @@ class MiniMaxH3Pipeline(
lora_model: LoRAModel,
bound_lora_names: frozenset[str],
) -> None:
if lora_model.id in self._turbo_lora_adapter_ids:
if lora_model.id in self._turbo_lora_specs:
missing = sorted(set(lora_model.loras) - bound_lora_names)
if missing:
raise ValueError(
@@ -770,14 +768,6 @@ class MiniMaxH3Pipeline(
f"bound={len(bound_lora_names)}/{len(lora_model.loras)}, missing={missing[:5]}"
)
def _has_active_turbo_lora(self, sampling: Any) -> bool:
lora_request = sampling.lora_request
return (
lora_request is not None
and not math.isclose(0.0, float(sampling.lora_scale))
and lora_request.lora_int_id in self._turbo_lora_adapter_ids
)
def _has_active_native_lora(self, sampling: Any) -> bool:
lora_request = sampling.lora_request
return (
@@ -837,27 +827,46 @@ class MiniMaxH3Pipeline(
return adapter_schedule
return self._base_schedule_for_task(task)
def _validate_turbo_sampling(self, sampling: Any) -> None:
def _active_turbo_spec(self, sampling: Any) -> TurboSpec | None:
"""Return the spec of the Turbo adapter this request actually applies.
A recognized adapter at scale 0 contributes nothing, so it neither
constrains the task nor imposes its sampler contract.
"""
lora_request = sampling.lora_request
if lora_request is None or math.isclose(0.0, float(sampling.lora_scale)):
return None
return self._turbo_lora_specs.get(lora_request.lora_int_id)
def _validate_turbo_sampling(self, sampling: Any, spec: TurboSpec) -> None:
"""Hold a request to the contract of the artifact that is loaded.
Sigma-point count and both flow shifts vary across the Turbo family, so
each is checked against the adapter's own spec rather than a single
published configuration.
"""
extra = sampling.extra_args or {}
sigma_points = sampling.num_inference_steps
if sigma_points != MINIMAX_H3_TURBO_SIGMA_POINTS:
if sigma_points != spec.sigma_points:
raise OmniClientError(
"MiniMax-H3 Turbo requires num_inference_steps=5 (five sigma points produce four denoiser evaluations)"
f"{spec.filename} is a {spec.denoise_steps}-step artifact and requires "
f"num_inference_steps={spec.sigma_points} "
f"({spec.sigma_points} sigma points produce {spec.denoise_steps} denoiser evaluations)"
)
try:
video_shift = float(extra.get("flow_shift", self.default_video_shift))
except (TypeError, ValueError) as exc:
raise OmniClientError(f"MiniMax-H3 Turbo requires flow_shift={MINIMAX_H3_TURBO_VIDEO_SHIFT:g}") from exc
if not math.isclose(video_shift, MINIMAX_H3_TURBO_VIDEO_SHIFT):
raise OmniClientError(f"MiniMax-H3 Turbo requires flow_shift={MINIMAX_H3_TURBO_VIDEO_SHIFT:g}")
raise OmniClientError(f"{spec.filename} requires flow_shift={spec.video_shift:g}") from exc
if not math.isclose(video_shift, spec.video_shift):
raise OmniClientError(f"{spec.filename} requires flow_shift={spec.video_shift:g}")
try:
audio_shift = float(extra.get("audio_flow_shift", self.default_audio_shift))
except (TypeError, ValueError) as exc:
raise OmniClientError(
f"MiniMax-H3 Turbo requires audio_flow_shift={MINIMAX_H3_TURBO_AUDIO_SHIFT:g}"
) from exc
if not math.isclose(audio_shift, MINIMAX_H3_TURBO_AUDIO_SHIFT):
raise OmniClientError(f"MiniMax-H3 Turbo requires audio_flow_shift={MINIMAX_H3_TURBO_AUDIO_SHIFT:g}")
raise OmniClientError(f"{spec.filename} requires audio_flow_shift={spec.audio_shift:g}") from exc
if not math.isclose(audio_shift, spec.audio_shift):
raise OmniClientError(f"{spec.filename} requires audio_flow_shift={spec.audio_shift:g}")
def adopt_cache_dit_backend(self, backend: CacheDiTBackend) -> None:
"""Adopt runner-installed generic Cache-DiT for request transitions."""
@@ -890,7 +899,7 @@ class MiniMaxH3Pipeline(
getattr(od_config, "task_type", None),
str(od_config.model),
)
self._turbo_lora_adapter_ids: set[int] = set()
self._turbo_lora_specs: dict[int, TurboSpec] = {}
self._native_lora_adapter_ids: set[int] = set()
self._lora_sigma_schedules: dict[int, DMD2SigmaSchedule] = {}
model_root = _resolve_minimax_h3_model_root(
@@ -1177,7 +1186,7 @@ class MiniMaxH3Pipeline(
requested: str | None,
multi_modal_data: dict[str, Any],
*,
has_turbo_lora: bool = False,
turbo_spec: TurboSpec | None = None,
has_native_lora: bool = False,
) -> str:
if requested is None:
@@ -1196,8 +1205,11 @@ class MiniMaxH3Pipeline(
raise OmniClientError(
f"checkpoint partition {self.partition!r} supports {sorted(self.supported_tasks)}, got task={task!r}"
)
if task == "ref2va" and has_turbo_lora:
raise OmniClientError("MiniMax-H3 Turbo LoRA supports T2VA/FL2VA requests only")
if turbo_spec is not None and task not in turbo_spec.supported_tasks:
raise OmniClientError(
f"{turbo_spec.filename} is a {turbo_spec.task_family} Turbo artifact and serves "
f"{sorted(turbo_spec.supported_tasks)}, got task={task!r}"
)
if has_native_lora and task != "t2va":
raise OmniClientError("MiniMax-H3 native LoRA supports T2VA requests only")
if self._fasth3 is not None:
@@ -2216,16 +2228,16 @@ class MiniMaxH3Pipeline(
quality = sampling.quality
logger.debug("MiniMax H3 request quality=%s", quality)
extra = sampling.extra_args or {}
has_turbo_lora = self._has_active_turbo_lora(sampling)
turbo_spec = self._active_turbo_spec(sampling)
has_native_lora = self._has_active_native_lora(sampling)
task = self._resolve_task(
extra.get("task"),
multi_modal_data,
has_turbo_lora=has_turbo_lora,
turbo_spec=turbo_spec,
has_native_lora=has_native_lora,
)
if has_turbo_lora:
self._validate_turbo_sampling(sampling)
if turbo_spec is not None:
self._validate_turbo_sampling(sampling, turbo_spec)
if has_native_lora:
self._validate_native_sampling(sampling, task=task)
if self._fasth3 is not None: