diff --git a/docs/user_guide/diffusion/lora.md b/docs/user_guide/diffusion/lora.md index 3c8582af0..15817db34 100644 --- a/docs/user_guide/diffusion/lora.md +++ b/docs/user_guide/diffusion/lora.md @@ -249,8 +249,14 @@ MiniMax-H3 supports two few-step mechanisms that must not be conflated: - **Checkpoint-pinned schedule**: a merged release writes `base_schedule` into `model_index.json`. Requests must pass `num_inference_steps` as the interval count (for example `4` for `[1.0, 0.7, 0.4, 0.15, 0.0]`). -- **Runtime Turbo LoRA**: LightX2V Turbo artifacts request five sigma points and - enforce `flow_shift=6`, `audio_flow_shift=3`. +- **Runtime Turbo LoRA**: each LightX2V Turbo artifact carries its own contract, + read from its filename. A four-step artifact requests five sigma points and an + eight-step one requests nine; the 768p retrains enforce `flow_shift=6` and the + 544p artifacts `flow_shift=12`. `audio_flow_shift=3` across the family. A + request that does not match the loaded artifact is rejected by name. Alpha + comes from the artifact's metadata, or 8 when it declares none. `ref2v` + artifacts require `--task-type ref2va`; `fl2v` artifacts serve T2VA and FL2VA. + The ComfyUI exports use a fused-QKV layout and are not supported. - **Runtime native LoRA**: FlashGen-style artifacts declare `key_format=minimax-h3-native` and embed `base_schedule` in safetensors metadata. When active, the adapter schedule overrides the base checkpoint and diff --git a/recipes/MiniMaxAI/MiniMax-H3-Disaggregated.md b/recipes/MiniMaxAI/MiniMax-H3-Disaggregated.md index f4b1e35ae..62f744ee5 100644 --- a/recipes/MiniMaxAI/MiniMax-H3-Disaggregated.md +++ b/recipes/MiniMaxAI/MiniMax-H3-Disaggregated.md @@ -68,16 +68,24 @@ The `/v1/videos` request schema and `extra_params.task` values (`t2va`, ## Turbo LoRA -MiniMax-H3 Turbo uses five sigma points, `flow_shift=6`, and -`audio_flow_shift=3`. Start Turbo deployments with the dedicated defaults so -requests that omit sampling controls do not inherit the 50-step base schedule: +The deploy config below carries the four-step 768p contract -- five sigma +points, `flow_shift=6`, `audio_flow_shift=3` -- so requests that omit sampling +controls do not inherit the 50-step base schedule: ```bash vllm-omni serve MiniMaxAI/MiniMax-H3 \ --omni \ - --lora-path /path/to/MiniMax-H3-Turbo \ + --lora-path /path/to/minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors \ --deploy-config vllm_omni/deploy/minimax_h3_disaggregated_turbo.yaml ``` +`--lora-path` must name one artifact, or a directory holding exactly one; the +Turbo repository holds several and a directory of them is rejected as +ambiguous. Serving a different artifact means overriding those defaults with +that artifact's own contract -- see the +[Turbo LoRA table](MiniMax-H3.md#turbo-lora) for every published row. + The base deployment intentionally retains 50 inference steps for non-LoRA -quality. Turbo LoRA supports T2VA and FL2VA requests, not Ref2VA. +quality. This disaggregated topology is validated on the `fl2v` artifacts, +which serve T2VA and FL2VA; the `ref2v` artifacts need a `--task-type ref2va` +server. diff --git a/recipes/MiniMaxAI/MiniMax-H3.md b/recipes/MiniMaxAI/MiniMax-H3.md index 2f705812b..812ed6f30 100644 --- a/recipes/MiniMaxAI/MiniMax-H3.md +++ b/recipes/MiniMaxAI/MiniMax-H3.md @@ -869,13 +869,51 @@ balanced switch order. ### Turbo LoRA -Only the native Diffusers 4-step FL2VA/T2VA v1.0 artifact is supported: +The eight Diffusers-layout LightX2V Turbo artifacts are supported. The +filename records the contract, so the server reads the step count, task family +and flow shift from it and validates each request against the artifact that is +loaded. It does not rewrite request or deploy-config sampling values: the +request must carry that artifact's own settings, listed here, or it is +rejected. -```text -lightx2v/Minimax-h3-Turbo/minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors -``` +| Artifact | Task | Forwards | `num_inference_steps` | `flow_shift` | declared `alpha` | +| --- | --- | ---: | ---: | ---: | ---: | +| `minimax_h3_fl2v_turbo_4step_v0.1.safetensors` | T2VA / FL2VA | 4 | 5 | 12 | none -> 8 | +| `minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors` | T2VA / FL2VA | 4 | 5 | 6 | 128 | +| `minimax_h3_fl2v_turbo_4step_v1.1_768p_bf16.safetensors` | T2VA / FL2VA | 4 | 5 | 6 | 128 | +| `minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors` | T2VA / FL2VA | 4 | 5 | 6 | 8 | +| `minimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors` | T2VA / FL2VA | 8 | 9 | 12 | 8 | +| `minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors` | T2VA / FL2VA | 8 | 9 | 6 | 8 | +| `minimax_h3_ref2v_turbo_4step_v0.1_bf16.safetensors` | Ref2VA | 4 | 5 | 12 | 8 | +| `minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors` | Ref2VA | 8 | 9 | 6 | 8 | -Download only that file: +`audio_flow_shift` is `3.0` across the family. Each row is the complete +published filename; use it verbatim as `TURBO_FILE` below. + +Alpha needs no manual compensation: the server reads it from the artifact's +metadata, falling back to 8 with a warning for +`minimax_h3_fl2v_turbo_4step_v0.1`, the one artifact that declares none. The +request-level `scale` is a further multiplier on top of it. + +> [!NOTE] +> Every artifact is rank 128 and the delta is applied at `scale * alpha / +> rank`, so the `alpha=8` rows drive at 1/16 the strength of the `alpha=128` +> rows. 8 is the default of LightX2V's reference script, which never reads the +> metadata; its documented `v0.1` command does not override that default. + +Every artifact except `minimax_h3_fl2v_turbo_4step_v0.1` also ships a +`_comfyui_` export of the same weights. Those fuse Q/K/V into one projection +and are **not** supported; downloading one is refused by name. Take the +Diffusers file. The filename is the contract, so do not rename an artifact +either -- a renamed file is rejected rather than served on a guess. + +FL2VA artifacts serve `t2va` and `fl2va` on any FL2VA or combined server. +Ref2VA artifacts require +`--task-type ref2va`: a combined server serves `ref2va` from a second DiT that +the adapter cannot bind to, so loading one there is refused rather than silently +running an undistilled model on the few-step schedule. + +Download the artifact you want: ```bash export TURBO_DIR=/path/to/minimax-h3-turbo @@ -884,10 +922,19 @@ hf download lightx2v/Minimax-h3-Turbo "${TURBO_FILE}" --local-dir "${TURBO_DIR}" export TURBO_LORA="${TURBO_DIR}/${TURBO_FILE}" ``` +`--lora-path` accepts one artifact, or a directory holding exactly one. + +> [!IMPORTANT] +> This changes earlier behaviour. `--lora-path /path/to/minimax-h3-turbo` +> pointing at a full clone of the Turbo repository used to select the v1.0 768p +> file implicitly; a directory holding several recognized artifacts is now +> rejected as ambiguous. Name the artifact instead: +> `--lora-path /path/to/minimax-h3-turbo/minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors`. + Start from a non-offloaded or DLO FL2VA server command and add `--task-type fl2va --lora-backend peft --lora-path "${TURBO_LORA}"`. -`--lora-path` preloads the adapter; each request still activates it and uses -the published sampling settings: +`--lora-path` preloads the adapter; each request still activates it and must +carry that artifact's sampling settings: ```bash -F 'num_inference_steps=5' \ @@ -896,14 +943,25 @@ the published sampling settings: -F "lora={\"name\":\"h3-turbo-v1.0\",\"path\":\"${TURBO_LORA}\",\"scale\":1.0}" ``` -For FL2VA, change `task` and add `input_reference` as shown above. The 8-step, -ComfyUI, Ref2VA, and v1.1 artifacts are not supported. This integration is -dynamic-only and does not support prefusion or LoRA composition. DLO is +Switching to another FL2VA artifact means repointing `TURBO_FILE`, which moves +both `--lora-path` and the request's `lora.path`, and carrying that row's +`num_inference_steps` and `flow_shift`: `9` and `6` for `8step_v1.0_768p`, `9` +and `12` for the 544p `8step_v1.0`. A request that does not match the loaded +artifact is rejected, so a mismatch cannot silently degrade output. + +The two `ref2v` rows are not served by this FL2VA command. Start a +`--task-type ref2va` server, take a request from +[Ref2VA](#3-ref2va-image-only-imageaudio-or-mixed-references) and override +`num_inference_steps` and `flow_shift` with that row's values; those examples +already send `audio_flow_shift=3.0`. + +For FL2VA, change `task` and add `input_reference` as shown above. This +integration is dynamic-only and does not support prefusion or LoRA composition. DLO is supported by keeping the request-switchable LoRA A/B buffers resident on the accelerator while DLO streams only the base blocks; budget for this additional fixed HBM usage. Model-level and standard layerwise offload remain unsupported. -The five requested sigma points produce the four denoiser evaluations expected -by the Turbo artifact. +The requested sigma points always number one more than the artifact's denoiser +evaluations. ### FlashGen native LoRA diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_contract.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_contract.py index 791c8d79d..b11473498 100644 --- a/tests/diffusion/models/minimax_h3/test_minimax_h3_contract.py +++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_contract.py @@ -13,6 +13,8 @@ import pytest import torch import torch.nn as nn +from vllm_omni.diffusion.models.minimax_h3.lora import TurboSpec, parse_turbo_filename + pytestmark = [pytest.mark.core_model, pytest.mark.cpu, pytest.mark.diffusion] _ItemT = TypeVar("_ItemT") @@ -24,6 +26,13 @@ def _append_and_return(items: list[_ItemT], item: _ItemT, result: _ResultT) -> _ return result +def _turbo_spec(filename: str) -> TurboSpec: + """Build the spec a loaded artifact of this name would carry.""" + spec = parse_turbo_filename(filename) + assert spec is not None + return spec + + def test_h3_prepares_resolved_cache_state_immediately_before_denoise(): from vllm_omni.diffusion.models.minimax_h3 import MiniMaxH3Pipeline from vllm_omni.diffusion.request import OmniDiffusionRequest @@ -244,9 +253,12 @@ def test_combined_task_inference_and_transformer_routing(): assert pipeline._resolve_task(None, {"image": object()}) == "fl2va" assert pipeline._resolve_task(None, {"audio": object()}) == "ref2va" assert pipeline._resolve_task(None, {"video": object()}) == "ref2va" - assert pipeline._resolve_task("fl2va", {"image": object()}, has_turbo_lora=True) == "fl2va" - with pytest.raises(OmniClientError, match="supports T2VA/FL2VA requests only"): - pipeline._resolve_task("ref2va", {}, has_turbo_lora=True) + fl2v_spec = _turbo_spec("minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors") + ref2v_spec = _turbo_spec("minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors") + assert pipeline._resolve_task("fl2va", {"image": object()}, turbo_spec=fl2v_spec) == "fl2va" + with pytest.raises(OmniClientError, match="serves"): + pipeline._resolve_task("ref2va", {}, turbo_spec=fl2v_spec) + assert pipeline._resolve_task("ref2va", {"image": object()}, turbo_spec=ref2v_spec) == "ref2va" pipeline.partition = "ref2va" pipeline.supported_tasks = frozenset({"ref2va"}) assert pipeline._resolve_task(None, {"image": object()}) == "ref2va" diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_fasth3.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_fasth3.py index bcd0deec0..5c874ff02 100644 --- a/tests/diffusion/models/minimax_h3/test_minimax_h3_fasth3.py +++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_fasth3.py @@ -303,7 +303,7 @@ def _pipeline_stub(fusion=None): torch.nn.Module.__init__(pipeline) pipeline.partition = "combined" pipeline.supported_tasks = frozenset({"t2va", "fl2va", "ref2va"}) - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._fasth3 = fusion pipeline.od_config = SimpleNamespace() pipeline.default_video_shift, pipeline.default_audio_shift = 12.0, 3.0 diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_lora.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_lora.py index c493ac58f..9039f94f8 100644 --- a/tests/diffusion/models/minimax_h3/test_minimax_h3_lora.py +++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_lora.py @@ -13,7 +13,9 @@ from vllm.lora.lora_weights import PackedLoRALayerWeights from vllm_omni.diffusion.lora.manager import DiffusionLoRAManager from vllm_omni.diffusion.models.minimax_h3 import lora as lora_module from vllm_omni.diffusion.models.minimax_h3.lora import ( + TurboSpec, load_minimax_h3_turbo_lora, + parse_turbo_filename, ) from vllm_omni.errors import OmniClientError from vllm_omni.lora.request import LoRARequest @@ -46,7 +48,7 @@ def _request(path) -> LoRARequest: def _write_tiny_turbo( path, *, - alpha: str = "128", + alpha: str | None = "128", key_format: str = "minimax-h3-diffusers", omit_target: str | None = None, shape_overrides: dict[str, tuple[int, int]] | None = None, @@ -88,11 +90,17 @@ def _write_tiny_turbo( ) else: tensors[b_name] = torch.ones(overrides.get(b_name, (output_dim, rank))) - save_file( - tensors, - str(path), - metadata={"alpha": alpha, "key_format": key_format}, - ) + metadata = {"key_format": key_format} + if alpha is not None: + metadata["alpha"] = alpha + save_file(tensors, str(path), metadata=metadata) + + +def _spec(filename: str = "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors") -> TurboSpec: + """Build the spec a loaded artifact of this name would carry.""" + spec = parse_turbo_filename(filename) + assert spec is not None + return spec def test_h3_turbo_loads_through_legacy_lora_model_and_swaps_ffn(tmp_path): @@ -107,9 +115,12 @@ def test_h3_turbo_loads_through_legacy_lora_model_and_swaps_ffn(tmp_path): ) assert loaded is not None - lora_model, peft_helper = loaded + lora_model, peft_helper, spec = loaded assert peft_helper.r == 128 assert peft_helper.lora_alpha == 128 + assert spec.denoise_steps == 4 + assert spec.sigma_points == 5 + assert spec.task_family == "fl2v" assert len(lora_model.loras) == 312 assert "blocks.0.attn.to_q" in lora_model.loras assert "blocks.0.mlp.fc1" in lora_model.loras @@ -152,7 +163,8 @@ def test_legacy_manager_uses_the_h3_model_loader_without_changing_its_interface( class _Pipeline: def _load_diffusion_lora_adapter(self, **kwargs): - return load_minimax_h3_turbo_lora(partition="fl2va", **kwargs) + lora_model, peft_helper, _spec_unused = load_minimax_h3_turbo_lora(partition="fl2va", **kwargs) + return lora_model, peft_helper manager = object.__new__(DiffusionLoRAManager) manager.pipeline = _Pipeline() @@ -168,22 +180,30 @@ def test_legacy_manager_uses_the_h3_model_loader_without_changing_its_interface( assert "blocks.0.mlp.fc1" in lora_model.loras -def test_h3_turbo_rejects_wrong_alpha_and_ref2va(tmp_path): - wrong_alpha = tmp_path / "wrong_alpha" / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors" - wrong_alpha.parent.mkdir() - _write_tiny_turbo(wrong_alpha, alpha="8") - with pytest.raises(ValueError, match="requires alpha=128"): - load_minimax_h3_turbo_lora( - partition="fl2va", - lora_request=_request(wrong_alpha), - lora_path=wrong_alpha, - dtype=torch.float32, - ) +def test_h3_turbo_honours_the_declared_alpha_and_task_family(tmp_path): + """Alpha comes from the artifact; the family decides which server serves it.""" + + eight_step_alpha = tmp_path / "alpha8" / "minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors" + eight_step_alpha.parent.mkdir() + _write_tiny_turbo(eight_step_alpha, alpha="8") + loaded = load_minimax_h3_turbo_lora( + partition="fl2va", + lora_request=_request(eight_step_alpha), + lora_path=eight_step_alpha, + dtype=torch.float32, + ) + assert loaded is not None + _, peft_helper, spec = loaded + # The eight-step artifact is a 1/16-strength adapter; pinning alpha to the + # four-step value would over-drive it by 16x. + assert peft_helper.lora_alpha == 8 + assert spec.alpha == 8 + assert spec.sigma_points == 9 valid = tmp_path / "valid" / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors" valid.parent.mkdir() _write_tiny_turbo(valid) - with pytest.raises(ValueError, match="supports FL2VA/T2VA only"): + with pytest.raises(ValueError, match="Ref2VA-only server cannot serve it"): load_minimax_h3_turbo_lora( partition="ref2va", lora_request=_request(valid), @@ -192,6 +212,43 @@ def test_h3_turbo_rejects_wrong_alpha_and_ref2va(tmp_path): ) +@pytest.mark.parametrize("partition", ["fl2va", "combined"]) +def test_h3_turbo_ref2v_artifact_needs_a_ref2va_only_server(tmp_path, partition): + """A combined server routes ref2va to ``transformers_ref``, which the + adapter cannot bind to: the LoRA target pattern only injects into + ``transformer``. Admitting it there would run an undistilled DiT on the + artifact's few-step schedule with no error anywhere.""" + + path = tmp_path / "minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors" + _write_tiny_turbo(path) + + with pytest.raises(ValueError, match="start the server with --task-type ref2va"): + load_minimax_h3_turbo_lora( + partition=partition, + lora_request=_request(path), + lora_path=path, + dtype=torch.float32, + ) + + +def test_h3_turbo_ref2v_artifact_loads_on_a_ref2va_server(tmp_path): + path = tmp_path / "minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors" + _write_tiny_turbo(path) + + loaded = load_minimax_h3_turbo_lora( + partition="ref2va", + lora_request=_request(path), + lora_path=path, + dtype=torch.float32, + ) + + assert loaded is not None + spec = loaded[2] + assert spec.task_family == "ref2v" + assert spec.supported_tasks == frozenset({"ref2va"}) + assert spec.sigma_points == 9 + + @pytest.mark.parametrize( "offload_mode", [ @@ -214,6 +271,8 @@ def test_h3_turbo_rejects_offload_modes(tmp_path, offload_mode): def test_h3_turbo_allows_distributed_layerwise_offload(monkeypatch): + """DLO keeps the LoRA buffers resident, so it is not an unsupported mode.""" + from vllm_omni.diffusion.models.minimax_h3 import MiniMaxH3Pipeline from vllm_omni.diffusion.models.minimax_h3 import ( pipeline_minimax_h3 as pipeline_module, @@ -222,7 +281,7 @@ def test_h3_turbo_allows_distributed_layerwise_offload(monkeypatch): pipeline = object.__new__(MiniMaxH3Pipeline) torch.nn.Module.__init__(pipeline) pipeline.partition = "fl2va" - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._native_lora_adapter_ids = set() pipeline._lora_sigma_schedules = {} pipeline.od_config = SimpleNamespace( @@ -231,10 +290,11 @@ def test_h3_turbo_allows_distributed_layerwise_offload(monkeypatch): enable_distributed_layerwise_offload=True, ) captured = {} + spec = _spec() def load_turbo(**kwargs): captured.update(kwargs) - return object(), object() + return object(), object(), spec monkeypatch.setattr(pipeline_module, "load_minimax_h3_turbo_lora", load_turbo) loaded = pipeline._load_diffusion_lora_adapter( @@ -245,6 +305,10 @@ def test_h3_turbo_allows_distributed_layerwise_offload(monkeypatch): assert loaded is not None assert captured["unsupported_offload_mode"] is None + # The pipeline passes its own partition through and keeps the spec that + # decides the request contract. + assert captured["partition"] == "fl2va" + assert pipeline._turbo_lora_specs == {1: spec} @pytest.mark.parametrize( @@ -268,7 +332,7 @@ def test_h3_turbo_offload_compatibility_follows_dit_selection( pipeline = object.__new__(MiniMaxH3Pipeline) torch.nn.Module.__init__(pipeline) pipeline.partition = "fl2va" - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._native_lora_adapter_ids = set() pipeline._lora_sigma_schedules = {} pipeline.od_config = SimpleNamespace( @@ -281,10 +345,11 @@ def test_h3_turbo_offload_compatibility_follows_dit_selection( pin_cpu_memory=True, ) captured = {} + spec = _spec() def load_turbo(**kwargs): captured.update(kwargs) - return object(), object() + return object(), object(), spec monkeypatch.setattr(pipeline_module, "load_minimax_h3_turbo_lora", load_turbo) @@ -296,31 +361,84 @@ def test_h3_turbo_offload_compatibility_follows_dit_selection( assert captured["unsupported_offload_mode"] == expected_mode -def test_h3_turbo_accepts_only_the_declared_v1_artifact(tmp_path): - unsupported = tmp_path / "minimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors" - _write_tiny_turbo(unsupported) +def test_h3_turbo_uses_the_reference_alpha_when_the_artifact_declares_none(tmp_path): + """``4step_v0.1`` ships no alpha. LightX2V's reference script applies + ``scale * alpha / rank`` with alpha defaulting to 8, so the fallback must be + 8; alpha == rank would drive the adapter 16x too strongly.""" - with pytest.raises(ValueError, match="supports only"): + path = tmp_path / "minimax_h3_fl2v_turbo_4step_v0.1.safetensors" + _write_tiny_turbo(path, alpha=None) + + loaded = load_minimax_h3_turbo_lora( + partition="fl2va", + lora_request=_request(path), + lora_path=path, + dtype=torch.float32, + ) + + assert loaded is not None + _, peft_helper, spec = loaded + assert spec.alpha == 8 + assert peft_helper.lora_alpha == 8 + assert spec.alpha / spec.rank == 0.0625 + + +def test_h3_turbo_declines_the_comfyui_export(tmp_path): + """The ComfyUI exports fuse Q/K/V, so they are not this loader's format.""" + + path = tmp_path / "minimax_h3_fl2v_turbo_4step_v1.0_768p_comfyui_bf16.safetensors" + _write_tiny_turbo(path) + + assert ( load_minimax_h3_turbo_lora( partition="fl2va", - lora_request=_request(unsupported), - lora_path=unsupported, + lora_request=_request(path), + lora_path=path, dtype=torch.float32, ) + is None + ) - supported = tmp_path / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors" - _write_tiny_turbo(supported) - # Directory resolution selects the declared artifact even when another - # v1.0 checkpoint is present beside it. - assert ( + +def test_h3_turbo_accepts_every_published_artifact_name(tmp_path): + """Each published filename loads and carries its own sampler contract.""" + + for index, (name, steps, shift) in enumerate( + ( + ("minimax_h3_fl2v_turbo_4step_v0.1.safetensors", 4, 12.0), + ("minimax_h3_fl2v_turbo_4step_v1.1_768p_bf16.safetensors", 4, 6.0), + ("minimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors", 8, 12.0), + ("minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors", 8, 6.0), + ) + ): + directory = tmp_path / f"artifact{index}" + directory.mkdir() + path = directory / name + _write_tiny_turbo(path) + loaded = load_minimax_h3_turbo_lora( + partition="fl2va", + lora_request=_request(path), + lora_path=path, + dtype=torch.float32, + ) + assert loaded is not None, name + spec = loaded[2] + assert spec.denoise_steps == steps + assert spec.sigma_points == steps + 1 + assert spec.video_shift == shift + + +def test_h3_turbo_directory_with_several_artifacts_is_ambiguous(tmp_path): + _write_tiny_turbo(tmp_path / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors") + _write_tiny_turbo(tmp_path / "minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors") + + with pytest.raises(ValueError, match="point --lora-path at one file"): load_minimax_h3_turbo_lora( partition="fl2va", lora_request=_request(tmp_path), lora_path=tmp_path, dtype=torch.float32, ) - is not None - ) def test_h3_turbo_rejects_a_truncated_declared_artifact(tmp_path): @@ -352,7 +470,7 @@ def test_non_h3_checkpoint_falls_back_to_the_generic_peft_loader(tmp_path): declared = tmp_path / "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors" _write_tiny_turbo(declared, key_format="other") - with pytest.raises(ValueError, match="requires safetensors metadata"): + with pytest.raises(ValueError, match="expected 'minimax-h3-diffusers'"): load_minimax_h3_turbo_lora( partition="fl2va", lora_request=_request(declared), @@ -371,7 +489,7 @@ def test_only_an_active_recognized_turbo_adapter_restricts_ref2va(monkeypatch): torch.nn.Module.__init__(pipeline) pipeline.partition = "combined" pipeline.supported_tasks = frozenset({"t2va", "fl2va", "ref2va"}) - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._native_lora_adapter_ids = set() pipeline._lora_sigma_schedules = {} request = _request("generic-peft") @@ -388,14 +506,14 @@ def test_only_an_active_recognized_turbo_adapter_restricts_ref2va(monkeypatch): return pipeline._resolve_task( "ref2va", {}, - has_turbo_lora=pipeline._has_active_turbo_lora(sampling), + turbo_spec=pipeline._active_turbo_spec(sampling), ) - recognized = (object(), object()) + recognized = (object(), object(), _spec()) monkeypatch.setattr(pipeline_module, "load_minimax_h3_turbo_lora", lambda **_: recognized) - assert load() is recognized + assert load() == recognized[:2] assert resolve(0.0) == "ref2va" - with pytest.raises(OmniClientError, match="supports T2VA/FL2VA requests only"): + with pytest.raises(OmniClientError, match="serves"): resolve(1.0) # Simulate manager eviction followed by a generic PEFT adapter reusing the @@ -410,7 +528,7 @@ def test_h3_turbo_requires_all_loaded_targets_to_bind(): pipeline = object.__new__(MiniMaxH3Pipeline) torch.nn.Module.__init__(pipeline) - pipeline._turbo_lora_adapter_ids = {1} + pipeline._turbo_lora_specs = {1: _spec()} lora_model = SimpleNamespace(id=1, loras={"to_q": object(), "to_k": object()}) pipeline._validate_diffusion_lora_binding( @@ -448,7 +566,7 @@ def test_h3_turbo_rejects_unsupported_sampling(num_inference_steps, extra_args, ) with pytest.raises(OmniClientError, match=error): - pipeline._validate_turbo_sampling(sampling) + pipeline._validate_turbo_sampling(sampling, _spec()) def test_h3_turbo_accepts_five_sigma_points_for_four_nfe(): @@ -461,5 +579,6 @@ def test_h3_turbo_accepts_five_sigma_points_for_four_nfe(): SimpleNamespace( num_inference_steps=5, extra_args={"flow_shift": 6.0, "audio_flow_shift": 3.0}, - ) + ), + _spec(), ) diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_native_lora.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_native_lora.py index c4e581381..28f074a19 100644 --- a/tests/diffusion/models/minimax_h3/test_minimax_h3_native_lora.py +++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_native_lora.py @@ -309,7 +309,7 @@ def test_pipeline_native_schedule_and_task_validation(monkeypatch): torch.nn.Module.__init__(pipeline) pipeline.partition = "fl2va" pipeline.supported_tasks = frozenset({"t2va", "fl2va", "ref2va"}) - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._native_lora_adapter_ids = {7} pipeline._lora_sigma_schedules = {7: DMD2SigmaSchedule.from_positions([1.0, 0.7, 0.4, 0.15, 0.0])} pipeline._base_schedule_by_partition = {"fl2va": None} @@ -330,7 +330,7 @@ def test_pipeline_native_schedule_and_task_validation(monkeypatch): pipeline._resolve_task( "fl2va", {}, - has_turbo_lora=False, + turbo_spec=None, has_native_lora=True, ) @@ -418,7 +418,7 @@ def test_pipeline_replaces_native_classification_after_reload(monkeypatch, tmp_p enable_layerwise_offload=False, enable_distributed_layerwise_offload=False, ) - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._native_lora_adapter_ids = set() pipeline._lora_sigma_schedules = {} @@ -472,7 +472,7 @@ def test_h3_native_allows_distributed_layerwise_offload(monkeypatch): enable_layerwise_offload=False, enable_distributed_layerwise_offload=True, ) - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._native_lora_adapter_ids = set() pipeline._lora_sigma_schedules = {} captured: dict[str, object] = {} @@ -612,7 +612,7 @@ def test_legacy_manager_uses_native_loader(tmp_path): enable_layerwise_offload=False, enable_distributed_layerwise_offload=False, ) - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._native_lora_adapter_ids = set() pipeline._lora_sigma_schedules = {} return pipeline._load_diffusion_lora_adapter(**kwargs) @@ -634,7 +634,7 @@ def test_pipeline_schedule_inactive_when_scale_zero(): pipeline = object.__new__(MiniMaxH3Pipeline) torch.nn.Module.__init__(pipeline) pipeline.partition = "fl2va" - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._native_lora_adapter_ids = {7} pipeline._lora_sigma_schedules = {7: DMD2SigmaSchedule.from_positions([1.0, 0.7, 0.4, 0.15, 0.0])} pipeline._base_schedule_by_partition = {"fl2va": None} @@ -664,7 +664,7 @@ def test_pipeline_schedule_falls_back_after_eviction(monkeypatch, tmp_path): enable_layerwise_offload=False, enable_distributed_layerwise_offload=False, ) - pipeline._turbo_lora_adapter_ids = set() + pipeline._turbo_lora_specs = {} pipeline._native_lora_adapter_ids = set() pipeline._lora_sigma_schedules = {} pipeline._base_schedule_by_partition = {"fl2va": None} diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_turbo_matrix.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_turbo_matrix.py new file mode 100644 index 000000000..35e46af6b --- /dev/null +++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_turbo_matrix.py @@ -0,0 +1,86 @@ +# SPDX-License-Identifier: Apache-2.0 +# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project + +"""Contract coverage for the LightX2V MiniMax-H3 Turbo artifact matrix.""" + +from __future__ import annotations + +import pytest + +from vllm_omni.diffusion.models.minimax_h3.lora import ( + _TURBO_AUDIO_SHIFT, + _TURBO_VIDEO_SHIFT_544P, + _TURBO_VIDEO_SHIFT_768P, + TurboSpec, + parse_turbo_filename, +) + +pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, pytest.mark.cpu] + +# The eight Diffusers-layout artifacts published at lightx2v/Minimax-h3-Turbo, +# with the contract each one implies. The ComfyUI export of an artifact is a +# fused-QKV re-packing of the same weights and is not served. +PUBLISHED = [ + ("minimax_h3_fl2v_turbo_4step_v0.1.safetensors", "fl2v", 4, _TURBO_VIDEO_SHIFT_544P), + ("minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors", "fl2v", 4, _TURBO_VIDEO_SHIFT_768P), + ("minimax_h3_fl2v_turbo_4step_v1.1_768p_bf16.safetensors", "fl2v", 4, _TURBO_VIDEO_SHIFT_768P), + ("minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", "fl2v", 4, _TURBO_VIDEO_SHIFT_768P), + ("minimax_h3_fl2v_turbo_8step_v1.0_bf16.safetensors", "fl2v", 8, _TURBO_VIDEO_SHIFT_544P), + ("minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors", "fl2v", 8, _TURBO_VIDEO_SHIFT_768P), + ("minimax_h3_ref2v_turbo_4step_v0.1_bf16.safetensors", "ref2v", 4, _TURBO_VIDEO_SHIFT_544P), + ("minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors", "ref2v", 8, _TURBO_VIDEO_SHIFT_768P), +] + + +@pytest.mark.parametrize(("filename", "task", "steps", "video_shift"), PUBLISHED) +def test_every_published_artifact_is_recognised(filename: str, task: str, steps: int, video_shift: float) -> None: + spec = parse_turbo_filename(filename) + assert spec is not None, filename + assert spec.filename == filename + assert spec.task_family == task + assert spec.denoise_steps == steps + assert spec.video_shift == video_shift + assert spec.audio_shift == _TURBO_AUDIO_SHIFT + # Until the loader reads metadata, alpha is LightX2V's reference default. + assert spec.rank == 128 + assert spec.alpha == 8.0 + + +def test_all_eight_artifacts_are_covered() -> None: + assert len(PUBLISHED) == 8 + assert len({name for name, *_ in PUBLISHED}) == 8 + + +@pytest.mark.parametrize( + "filename", + [ + "adapter_model.safetensors", + "minimax_h3_fl2v_turbo_v1.0_768p_bf16.safetensors", + "minimax_h3_t2v_turbo_4step_v1.0.safetensors", + "minimax_h3_fl2v_turbo_0step_v1.0.safetensors", + # The ComfyUI exports are a different tensor layout, not a variant of + # the supported one; the loader refuses them by name. + "minimax_h3_fl2v_turbo_4step_v1.0_768p_comfyui_bf16.safetensors", + "minimax_h3_ref2v_turbo_8step_v1.0_768p_comfyui_bf16.safetensors", + ], +) +def test_unrelated_names_are_rejected(filename: str) -> None: + assert parse_turbo_filename(filename) is None + + +def _spec(filename: str) -> TurboSpec: + spec = parse_turbo_filename(filename) + assert spec is not None + return spec + + +def test_sigma_points_follow_the_step_count() -> None: + assert _spec("minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors").sigma_points == 5 + assert _spec("minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors").sigma_points == 9 + + +def test_task_family_decides_supported_tasks() -> None: + fl2v = _spec("minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors") + ref2v = _spec("minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors") + assert fl2v.supported_tasks == frozenset({"t2va", "fl2va"}) + assert ref2v.supported_tasks == frozenset({"ref2va"}) diff --git a/vllm_omni/diffusion/models/minimax_h3/lora.py b/vllm_omni/diffusion/models/minimax_h3/lora.py index 026d9f6b3..fced23de3 100644 --- a/vllm_omni/diffusion/models/minimax_h3/lora.py +++ b/vllm_omni/diffusion/models/minimax_h3/lora.py @@ -1,20 +1,25 @@ # SPDX-License-Identifier: Apache-2.0 # SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -"""Loader for the LightX2V Turbo MiniMax-H3 LoRA. +"""Loader for the LightX2V Turbo MiniMax-H3 LoRA family. -Keyed by ``key_format=minimax-h3-diffusers``. The native FlashGen contract lives -in ``.npu.lora``; the two artifacts differ in rank, target naming and QKV -layout, so each owns its own parsing and packing. +The published artifacts differ in task family, step count, training resolution +and LoRA alpha, so :class:`TurboSpec` carries the contract of the file actually +loaded and the pipeline validates each request against it. Only the +Diffusers-PEFT exports are served. The native FlashGen contract lives in +``.npu.lora``. """ from __future__ import annotations import math +from dataclasses import dataclass, replace from pathlib import Path +import regex as re import torch from safetensors import safe_open +from vllm.logger import init_logger from vllm.lora.lora_model import LoRAModel from vllm.lora.lora_weights import PackedLoRALayerWeights from vllm.lora.peft_helper import PEFTHelper @@ -22,12 +27,79 @@ from vllm.model_executor.models.utils import WeightsMapper from vllm_omni.lora.request import LoRARequest +logger = init_logger(__name__) + _TURBO_RANK = 128 -_TURBO_ALPHA = 128 _TURBO_HIDDEN_SIZE = 5376 _TURBO_ATTENTION_INNER_SIZE = 7168 _TURBO_FFN_HIDDEN_SIZE = 14336 -_TURBO_FILENAME = "minimax_h3_fl2v_turbo_4step_v1.0_768p_bf16.safetensors" +# ``minimax_h3__turbo_step_v[_768p][_bf16]``. The name is +# the only place the sampler contract is recorded. +_TURBO_NAME_RE = re.compile( + r"^minimax_h3_(?Pfl2v|ref2v)_turbo_(?P\d+)step" + r"_v\d+\.\d+(?P_768p)?(?:_bf16)?\.safetensors$" +) +# The 768p retrains moved to a shorter video flow shift; the 544p artifacts keep +# the base model's. +_TURBO_VIDEO_SHIFT_768P = 6.0 +_TURBO_VIDEO_SHIFT_544P = 12.0 +_TURBO_AUDIO_SHIFT = 3.0 +# Only ``4step_v0.1`` declares no alpha. LightX2V's reference script never reads +# the metadata: it applies ``scale * alpha / rank`` with ``--lora-alpha`` +# defaulting to 8, and its documented v0.1 command does not override that. So an +# artifact that declares nothing is driven at 8, not at its rank. +_TURBO_DEFAULT_ALPHA = 8.0 + + +@dataclass(frozen=True) +class TurboSpec: + """The sampler contract one Turbo artifact was distilled for.""" + + filename: str + task_family: str + """``fl2v`` (serves t2va and fl2va) or ``ref2v`` (serves ref2va).""" + denoise_steps: int + video_shift: float + audio_shift: float + rank: int + alpha: float + + @property + def sigma_points(self) -> int: + """Sigma points the API contract expects: one more than the forwards.""" + return self.denoise_steps + 1 + + @property + def supported_tasks(self) -> frozenset[str]: + if self.task_family == "ref2v": + return frozenset({"ref2va"}) + return frozenset({"t2va", "fl2va"}) + + +def parse_turbo_filename(name: str) -> TurboSpec | None: + """Return the contract a Turbo filename encodes, or ``None``. + + ``alpha`` is the rank-matched default; the loader replaces it with the + artifact's declared value. + """ + + match = _TURBO_NAME_RE.match(name) + if match is None: + return None + steps = int(match.group("steps")) + if steps <= 0: + return None + return TurboSpec( + filename=name, + task_family=match.group("task"), + denoise_steps=steps, + video_shift=_TURBO_VIDEO_SHIFT_768P if match.group("res") else _TURBO_VIDEO_SHIFT_544P, + audio_shift=_TURBO_AUDIO_SHIFT, + rank=_TURBO_RANK, + alpha=_TURBO_DEFAULT_ALPHA, + ) + + _LORA_A_SUFFIX = ".lora_A.default.weight" _LORA_B_SUFFIX = ".lora_B.default.weight" _TURBO_TARGETS = frozenset({"to_q", "to_k", "to_v", "out_proj", "fc1", "fc2"}) @@ -81,11 +153,20 @@ def _select_turbo_file(artifact_path: str | Path) -> Path | None: if not path.is_dir(): return None - candidate = path / _TURBO_FILENAME - return candidate if candidate.is_file() else None + candidates = sorted(child for child in path.glob("*.safetensors") if parse_turbo_filename(child.name)) + if len(candidates) == 1: + return candidates[0] + if len(candidates) > 1: + raise ValueError( + f"{path} holds {len(candidates)} MiniMax-H3 Turbo artifacts " + f"({[c.name for c in candidates[:4]]}); point --lora-path at one file." + ) + return None def _validate_and_convert_tensors(checkpoint) -> dict[str, torch.Tensor]: + """Validate a Diffusers-layout Turbo tensor set and pack it for the manager.""" + tensors: dict[str, torch.Tensor] = {} pairs: dict[str, set[str]] = {} raw_targets: set[str] = set() @@ -134,7 +215,7 @@ def _validate_and_convert_tensors(checkpoint) -> dict[str, torch.Tensor]: unexpected = sorted(raw_targets - _TURBO_EXPECTED_RAW_TARGETS) if missing or unexpected: raise ValueError( - "MiniMax-H3 Turbo target set does not match the supported v1.0 artifact: " + "MiniMax-H3 Turbo target set does not match the published artifact layout: " f"missing={len(missing)} {missing[:5]}, unexpected={len(unexpected)} {unexpected[:5]}" ) return tensors @@ -164,39 +245,61 @@ def load_minimax_h3_turbo_lora( lora_path: str | Path, dtype: torch.dtype, unsupported_offload_mode: str | None = None, -) -> tuple[LoRAModel, PEFTHelper] | None: - """Load the published LightX2V Turbo v1.0 through the legacy manager.""" +) -> tuple[LoRAModel, PEFTHelper, TurboSpec] | None: + """Load any published LightX2V Turbo artifact through the legacy manager.""" lora_file = _select_turbo_file(lora_path) if lora_file is None: return None + spec = parse_turbo_filename(lora_file.name) + if spec is None: + # Not a Turbo artifact by name; leave it to the next loader. + return None with safe_open(lora_file, framework="pt", device="cpu") as checkpoint: + # Three published artifacts declare no key_format, so absence is not an + # error; a declaration naming a different format is. metadata = checkpoint.metadata() or {} - if metadata.get("key_format") != "minimax-h3-diffusers": - if lora_file.name == _TURBO_FILENAME: - raise ValueError( - "MiniMax-H3 Turbo v1.0 requires safetensors metadata key_format='minimax-h3-diffusers'" - ) - return None - if lora_file.name != _TURBO_FILENAME: - raise ValueError(f"MiniMax-H3 Turbo supports only {_TURBO_FILENAME!r}, got {lora_file.name!r}") + declared_format = metadata.get("key_format") + if declared_format not in (None, "minimax-h3-diffusers"): + raise ValueError( + f"{lora_file.name} declares key_format={declared_format!r}, expected 'minimax-h3-diffusers'" + ) raw_alpha = metadata.get("alpha") - try: - alpha = float(raw_alpha) if raw_alpha is not None else math.nan - except ValueError as exc: - raise ValueError(f"MiniMax-H3 Turbo alpha must be numeric, got {raw_alpha!r}") from exc - if alpha != _TURBO_ALPHA: - raise ValueError(f"MiniMax-H3 Turbo v1.0 requires alpha={_TURBO_ALPHA}, got {raw_alpha!r}") - if partition == "ref2va": - raise ValueError("MiniMax-H3 Turbo LoRA supports FL2VA/T2VA only") + if raw_alpha is None: + logger.warning( + "MiniMax-H3 Turbo artifact %s declares no alpha; using the LightX2V reference " + "default alpha=%g (scale %g). Override with the request-level LoRA scale.", + spec.filename, + spec.alpha, + spec.alpha / spec.rank, + ) + else: + try: + spec = replace(spec, alpha=float(raw_alpha)) + except (TypeError, ValueError) as exc: + raise ValueError(f"MiniMax-H3 Turbo alpha must be numeric, got {raw_alpha!r}") from exc + if not math.isfinite(spec.alpha) or spec.alpha <= 0: + raise ValueError(f"MiniMax-H3 Turbo alpha must be a positive number, got {raw_alpha!r}") + + if partition == "ref2va" and spec.task_family == "fl2v": + raise ValueError(f"{spec.filename} is an FL2VA/T2VA Turbo artifact; a Ref2VA-only server cannot serve it.") + # ``combined`` serves ref2va from ``transformers_ref``, but the LoRA + # target pattern only injects into ``transformer``: the adapter would + # bind to the stack that never runs. + if spec.task_family == "ref2v" and partition != "ref2va": + raise ValueError( + f"{spec.filename} is a Ref2VA Turbo artifact; start the server with --task-type ref2va " + f"(task_type={partition!r} serves ref2va from a DiT the adapter cannot bind to)" + ) if unsupported_offload_mode is not None: raise ValueError(f"MiniMax-H3 Turbo dynamic LoRA does not support {unsupported_offload_mode}") + tensors = _validate_and_convert_tensors(checkpoint) peft_helper = PEFTHelper.from_dict( { - "r": _TURBO_RANK, - "lora_alpha": _TURBO_ALPHA, + "r": spec.rank, + "lora_alpha": spec.alpha, "target_modules": _TURBO_TARGET_PATTERN, } ) @@ -209,4 +312,16 @@ def load_minimax_h3_turbo_lora( weights_mapper=_TURBO_WEIGHTS_MAPPER, ) _pack_h3_turbo_fc1(lora_model) - return lora_model, peft_helper + logger.info( + "MiniMax-H3 Turbo adapter %s: task=%s, %d denoiser forwards (%d sigma points), " + "rank=%d alpha=%g, flow_shift=%g/%g", + spec.filename, + spec.task_family, + spec.denoise_steps, + spec.sigma_points, + spec.rank, + spec.alpha, + spec.video_shift, + spec.audio_shift, + ) + return lora_model, peft_helper, spec diff --git a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py index 22f638e9a..794d2dc09 100644 --- a/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py +++ b/vllm_omni/diffusion/models/minimax_h3/pipeline_minimax_h3.py @@ -125,7 +125,7 @@ from .denoise_loop import ( ) from .encoder import MiniMaxH3Qwen3VLEncoder from .fasth3 import FastH3WeightFusion, resolve_fasth3_fusion -from .lora import load_minimax_h3_turbo_lora +from .lora import TurboSpec, load_minimax_h3_turbo_lora from .minimax_h3_transformer import ( MiniMaxH3Attention, MiniMaxH3DiTModel, @@ -188,9 +188,6 @@ MINIMAX_H3_MAX_REFERENCE_IMAGE_BYTES = 30 * 1024 * 1024 MINIMAX_H3_REFERENCE_IMAGE_FORMATS = frozenset({"jpeg", "png", "webp", "heic", "heif"}) MINIMAX_H3_MIN_OUTPUT_SECONDS = 4.0 MINIMAX_H3_MAX_OUTPUT_SECONDS = 15.0 -MINIMAX_H3_TURBO_SIGMA_POINTS = 5 -MINIMAX_H3_TURBO_VIDEO_SHIFT = 6.0 -MINIMAX_H3_TURBO_AUDIO_SHIFT = 3.0 MINIMAX_H3_DOWNLOAD_PATTERNS = [ "FL2VA/**", "Ref2VA/model_index.json", @@ -706,7 +703,7 @@ class MiniMaxH3Pipeline( ) -> tuple[LoRAModel, PEFTHelper] | None: # A cache eviction may be followed by a different adapter reusing the # same client-supplied ID. Every real load replaces the classification. - self._turbo_lora_adapter_ids.discard(lora_request.lora_int_id) + self._turbo_lora_specs.pop(lora_request.lora_int_id, None) self._native_lora_adapter_ids.discard(lora_request.lora_int_id) self._lora_sigma_schedules.pop(lora_request.lora_int_id, None) od_config = getattr(self, "od_config", None) @@ -726,8 +723,9 @@ class MiniMaxH3Pipeline( unsupported_offload_mode=" or ".join(offload_modes) or None, ) if loaded is not None: - self._turbo_lora_adapter_ids.add(lora_request.lora_int_id) - return loaded + lora_model, peft_helper, turbo_spec = loaded + self._turbo_lora_specs[lora_request.lora_int_id] = turbo_spec + return lora_model, peft_helper # Selection is by the artifact's safetensors ``key_format``, not by the # running platform: the native loader is checkpoint-format parsing with @@ -753,7 +751,7 @@ class MiniMaxH3Pipeline( lora_model: LoRAModel, bound_lora_names: frozenset[str], ) -> None: - if lora_model.id in self._turbo_lora_adapter_ids: + if lora_model.id in self._turbo_lora_specs: missing = sorted(set(lora_model.loras) - bound_lora_names) if missing: raise ValueError( @@ -770,14 +768,6 @@ class MiniMaxH3Pipeline( f"bound={len(bound_lora_names)}/{len(lora_model.loras)}, missing={missing[:5]}" ) - def _has_active_turbo_lora(self, sampling: Any) -> bool: - lora_request = sampling.lora_request - return ( - lora_request is not None - and not math.isclose(0.0, float(sampling.lora_scale)) - and lora_request.lora_int_id in self._turbo_lora_adapter_ids - ) - def _has_active_native_lora(self, sampling: Any) -> bool: lora_request = sampling.lora_request return ( @@ -837,27 +827,46 @@ class MiniMaxH3Pipeline( return adapter_schedule return self._base_schedule_for_task(task) - def _validate_turbo_sampling(self, sampling: Any) -> None: + def _active_turbo_spec(self, sampling: Any) -> TurboSpec | None: + """Return the spec of the Turbo adapter this request actually applies. + + A recognized adapter at scale 0 contributes nothing, so it neither + constrains the task nor imposes its sampler contract. + """ + + lora_request = sampling.lora_request + if lora_request is None or math.isclose(0.0, float(sampling.lora_scale)): + return None + return self._turbo_lora_specs.get(lora_request.lora_int_id) + + def _validate_turbo_sampling(self, sampling: Any, spec: TurboSpec) -> None: + """Hold a request to the contract of the artifact that is loaded. + + Sigma-point count and both flow shifts vary across the Turbo family, so + each is checked against the adapter's own spec rather than a single + published configuration. + """ + extra = sampling.extra_args or {} sigma_points = sampling.num_inference_steps - if sigma_points != MINIMAX_H3_TURBO_SIGMA_POINTS: + if sigma_points != spec.sigma_points: raise OmniClientError( - "MiniMax-H3 Turbo requires num_inference_steps=5 (five sigma points produce four denoiser evaluations)" + f"{spec.filename} is a {spec.denoise_steps}-step artifact and requires " + f"num_inference_steps={spec.sigma_points} " + f"({spec.sigma_points} sigma points produce {spec.denoise_steps} denoiser evaluations)" ) try: video_shift = float(extra.get("flow_shift", self.default_video_shift)) except (TypeError, ValueError) as exc: - raise OmniClientError(f"MiniMax-H3 Turbo requires flow_shift={MINIMAX_H3_TURBO_VIDEO_SHIFT:g}") from exc - if not math.isclose(video_shift, MINIMAX_H3_TURBO_VIDEO_SHIFT): - raise OmniClientError(f"MiniMax-H3 Turbo requires flow_shift={MINIMAX_H3_TURBO_VIDEO_SHIFT:g}") + raise OmniClientError(f"{spec.filename} requires flow_shift={spec.video_shift:g}") from exc + if not math.isclose(video_shift, spec.video_shift): + raise OmniClientError(f"{spec.filename} requires flow_shift={spec.video_shift:g}") try: audio_shift = float(extra.get("audio_flow_shift", self.default_audio_shift)) except (TypeError, ValueError) as exc: - raise OmniClientError( - f"MiniMax-H3 Turbo requires audio_flow_shift={MINIMAX_H3_TURBO_AUDIO_SHIFT:g}" - ) from exc - if not math.isclose(audio_shift, MINIMAX_H3_TURBO_AUDIO_SHIFT): - raise OmniClientError(f"MiniMax-H3 Turbo requires audio_flow_shift={MINIMAX_H3_TURBO_AUDIO_SHIFT:g}") + raise OmniClientError(f"{spec.filename} requires audio_flow_shift={spec.audio_shift:g}") from exc + if not math.isclose(audio_shift, spec.audio_shift): + raise OmniClientError(f"{spec.filename} requires audio_flow_shift={spec.audio_shift:g}") def adopt_cache_dit_backend(self, backend: CacheDiTBackend) -> None: """Adopt runner-installed generic Cache-DiT for request transitions.""" @@ -890,7 +899,7 @@ class MiniMaxH3Pipeline( getattr(od_config, "task_type", None), str(od_config.model), ) - self._turbo_lora_adapter_ids: set[int] = set() + self._turbo_lora_specs: dict[int, TurboSpec] = {} self._native_lora_adapter_ids: set[int] = set() self._lora_sigma_schedules: dict[int, DMD2SigmaSchedule] = {} model_root = _resolve_minimax_h3_model_root( @@ -1177,7 +1186,7 @@ class MiniMaxH3Pipeline( requested: str | None, multi_modal_data: dict[str, Any], *, - has_turbo_lora: bool = False, + turbo_spec: TurboSpec | None = None, has_native_lora: bool = False, ) -> str: if requested is None: @@ -1196,8 +1205,11 @@ class MiniMaxH3Pipeline( raise OmniClientError( f"checkpoint partition {self.partition!r} supports {sorted(self.supported_tasks)}, got task={task!r}" ) - if task == "ref2va" and has_turbo_lora: - raise OmniClientError("MiniMax-H3 Turbo LoRA supports T2VA/FL2VA requests only") + if turbo_spec is not None and task not in turbo_spec.supported_tasks: + raise OmniClientError( + f"{turbo_spec.filename} is a {turbo_spec.task_family} Turbo artifact and serves " + f"{sorted(turbo_spec.supported_tasks)}, got task={task!r}" + ) if has_native_lora and task != "t2va": raise OmniClientError("MiniMax-H3 native LoRA supports T2VA requests only") if self._fasth3 is not None: @@ -2216,16 +2228,16 @@ class MiniMaxH3Pipeline( quality = sampling.quality logger.debug("MiniMax H3 request quality=%s", quality) extra = sampling.extra_args or {} - has_turbo_lora = self._has_active_turbo_lora(sampling) + turbo_spec = self._active_turbo_spec(sampling) has_native_lora = self._has_active_native_lora(sampling) task = self._resolve_task( extra.get("task"), multi_modal_data, - has_turbo_lora=has_turbo_lora, + turbo_spec=turbo_spec, has_native_lora=has_native_lora, ) - if has_turbo_lora: - self._validate_turbo_sampling(sampling) + if turbo_spec is not None: + self._validate_turbo_sampling(sampling, turbo_spec) if has_native_lora: self._validate_native_sampling(sampling, task=task) if self._fasth3 is not None: