Compare commits

..
Author SHA1 Message Date
Hongsheng Liu 7b2f3682f7 [CI/Build] Clarify model-specific example filename
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-08-11 16:36:52 +08:00
Hongsheng Liu 8ef0d3f391 [CI/Build] Reuse examples policy in review-pr skill
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-08-11 16:31:36 +08:00
Hongsheng Liu 49985ec849 [CI/Build] Add examples policy to precheck-pr skill
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-08-11 16:23:31 +08:00
25 changed files with 362 additions and 314 deletions

No files matched your search

+4 -1
View File
@@ -1,6 +1,6 @@
--- ---
name: precheck-pr name: precheck-pr
description: Self-check your branch before creating a PR — catch dead code, verify accuracy/perf claims, validate PR title format, and confirm merge readiness. Use when the user says "precheck", "self review", "pre-submit check", or "check my PR before I open it." Never posts to GitHub. description: Self-check your branch before creating a PR — catch dead code, prevent new model-specific Python examples, verify accuracy/perf claims, validate PR title format, and confirm merge readiness. Use when the user says "precheck", "self review", "pre-submit check", or "check my PR before I open it." Never posts to GitHub.
--- ---
# PR Pre-Check # PR Pre-Check
@@ -65,6 +65,8 @@ Ask: "Quick mode or full mode?" Then walk the checklist for the detected PR type
**Also run the Code-Quality sweep on every PR**, regardless of type or mode: the five diff-scoped checks in [references/code-quality.md](references/code-quality.md) — kwargs fragility, broad-except swallow, `Any`/wrong type hints, hot-path `.clone()`/`deepcopy`, and event-loop blocking — plus the advisory conventions (log level, structured logging, synchronization, cleanup, dependencies, naming) in [checklists.md](references/checklists.md). These count only lines the PR *adds* — the pre-existing backlog across the repo is out of scope. **Also run the Code-Quality sweep on every PR**, regardless of type or mode: the five diff-scoped checks in [references/code-quality.md](references/code-quality.md) — kwargs fragility, broad-except swallow, `Any`/wrong type hints, hot-path `.clone()`/`deepcopy`, and event-loop blocking — plus the advisory conventions (log level, structured logging, synchronization, cleanup, dependencies, naming) in [checklists.md](references/checklists.md). These count only lines the PR *adds* — the pre-existing backlog across the repo is out of scope.
**Also run the Examples-Policy check on every PR** using [references/examples-policy.md](references/examples-policy.md). Inspect only Python paths introduced by the diff. Treat a new model-, checkpoint-, vendor-, or family-specific Python example as blocking; do not report pre-existing example debt.
### Step 5: Print Report ### Step 5: Print Report
``` ```
@@ -77,6 +79,7 @@ Pre-check report for <branch>
───────────────── ────── ───────────────── ──────
PR title format ✓ PR title format ✓
Code quality ⚠ 1 broad except, 2 Any hints Code quality ⚠ 1 broad except, 2 Any hints
Examples policy ✗ new model-specific Python example
PR desc integrity ✓ PR desc integrity ✓
Registry/config ✓ Registry/config ✓
Dead code ⚠ 2 warnings Dead code ⚠ 2 warnings
@@ -47,6 +47,19 @@ Diff-scoped sweep for five fragility patterns that are pervasive in the existing
Roll the ⚠/✗ counts into the report as a single **Code quality** dimension row. Roll the ⚠/✗ counts into the report as a single **Code quality** dimension row.
---
## All PRs: Examples Policy (run on every PR, quick & full)
Inspect Python files introduced under `examples/` using
[examples-policy.md](examples-policy.md). This is a diff-scoped ratchet: existing
model-specific examples may be edited or removed without triggering the check.
- [ ] **No new model-specific Python examples:** the diff adds, copies, or renames no Python file whose path or behavior is specific to one model, checkpoint, vendor, or model family. ✗ for any violation. Route inference through a shared task/protocol entrypoint, model request behavior through `vllm_omni/model_extras`, and commands through task docs or `recipes/`.
- [ ] **New generic example code is justified:** any new task-neutral Python file under `examples/` is an actual user-facing entrypoint rather than reusable production logic, a benchmark, an app, a setup tool, or a reproducer. ⚠ when the placement is questionable.
Roll the result into the report as a separate **Examples policy** dimension.
### Conventions (eyeball — no grep) ### Conventions (eyeball — no grep)
Judgment calls that don't grep cleanly — apply them while reading the diff, not as a mechanical check: Judgment calls that don't grep cleanly — apply them while reading the diff, not as a mechanical check:
@@ -0,0 +1,68 @@
# Examples Python Policy
Prevent new model-specific Python examples without forcing cleanup of the
existing backlog. Run this check on every PR in quick and full modes.
## Scope
Set the merge base as in the main workflow, then list Python paths introduced
by the PR, including additions, copies, and renames:
```bash
BASE="$(git merge-base HEAD origin/main)"
git diff --name-status --diff-filter=ACR --find-renames --find-copies \
"${BASE}"...HEAD -- 'examples/**/*.py'
```
Inspect destination paths for copies and renames. Existing model-specific files
that are only modified or deleted are grandfathered and must not be reported.
## Blocking classification
Mark ✗ when a newly introduced Python file does any of the following:
- Names or scopes a path to one model, checkpoint, vendor, or model family.
- Hard-codes a model identifier, model-specific prompt, request shape, output
conversion, or launch configuration behind an otherwise generic filename.
- Duplicates an existing task/protocol runner to demonstrate one model.
- Adds a per-model helper below a generic task hub such as `text_to_speech/`.
Do not infer safety from a generic filename alone. For example, a file named
`text_to_audio.py` is still model-specific if it only implements one model's
contract.
## Acceptable additions
A new Python example may pass when it is a genuinely model-neutral,
user-facing task or protocol entrypoint. It must accept model selection through
configuration and keep model-specific behavior out of the script.
Use ⚠ when the file is model-neutral but reusable logic appears misplaced under
`examples/`. Route code by purpose:
| Code purpose | Preferred location |
|---|---|
| Model prompt, defaults, request/output adaptation | `vllm_omni/model_extras/` or production model modules |
| Runnable model command and validation evidence | Task documentation or `recipes/` |
| UI or interactive application | `apps/` |
| Benchmark or evaluator | `benchmarks/` |
| Regression reproducer | `tests/` |
| Download, conversion, or setup utility | `tools/` |
If a model cannot use an existing shared runner, report the missing generic
capability. Do not solve the gap by adding a model-specific Python example.
## Report format
Add one row to the pre-check report:
```text
Examples policy ✓ no new Python example paths
Examples policy ⚠ generic helper belongs in tools/
Examples policy ✗ examples/offline_inference/new_model/end2end.py is model-specific
Examples policy ✗ examples/offline_inference/x_to_y_model_name.py is model-specific
```
For a blocker, name the path, the model-specific behavior, and the appropriate
shared runner or destination. Do not propose deleting unrelated existing
examples as part of the PR.
+8
View File
@@ -97,6 +97,7 @@ If docs and live code disagree, verify the code/tests and report the drift.
| [test-quality-evaluation.md](references/checks/test-quality-evaluation.md) | Tests change, are absent for risky code, or may not exercise production behavior. | | [test-quality-evaluation.md](references/checks/test-quality-evaluation.md) | Tests change, are absent for risky code, or may not exercise production behavior. |
| [tests-docs-checklist.md](references/checks/tests-docs-checklist.md) | Coverage, CI markers, examples, user docs, or PR evidence need review. | | [tests-docs-checklist.md](references/checks/tests-docs-checklist.md) | Coverage, CI markers, examples, user docs, or PR evidence need review. |
| [verification.md](references/checks/verification.md) | Hardware, a server, or a runnable affected path is available for active verification. | | [verification.md](references/checks/verification.md) | Hardware, a server, or a runnable affected path is available for active verification. |
| [examples-policy.md](../precheck-pr/references/examples-policy.md) | The PR adds, copies, or renames Python under `examples/`; apply the canonical policy shared with `precheck-pr`. |
### Delivery and reviewer coordination ### Delivery and reviewer coordination
@@ -152,6 +153,13 @@ contract they protect or use only the applicable evidence checks.
Apply every category in [general-checks.md](references/process/general-checks.md) before Apply every category in [general-checks.md](references/process/general-checks.md) before
lower-priority comments. lower-priority comments.
If the diff census contains an added, copied, or renamed Python path under
`examples/`, read and apply the canonical
[examples policy](../precheck-pr/references/examples-policy.md). Treat a new
model-specific Python example as blocking. Do not flag model-specific example
debt that the PR only modifies or removes, and do not run the rest of the
author-oriented `precheck-pr` workflow.
For each changed value or behavior, trace: For each changed value or behavior, trace:
```text ```text
@@ -61,9 +61,7 @@ from vllm_omni.diffusion.utils.param_utils import apply_declared_extra_args
from vllm_omni.entrypoints.omni import Omni from vllm_omni.entrypoints.omni import Omni
from vllm_omni.inputs.data import OmniDiffusionSamplingParams from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.model_extras import ( from vllm_omni.model_extras import (
build_image_to_video_prompt as build_model_image_to_video_prompt, build_image_to_video_prompt,
)
from vllm_omni.model_extras import (
get_extra_body_params, get_extra_body_params,
get_model_class_name, get_model_class_name,
) )
@@ -85,22 +83,6 @@ def parse_json_object(value: str, flag_name: str = "argument") -> dict[str, Any]
parse_profiler_config = functools.partial(parse_json_object, flag_name="--profiler-config") parse_profiler_config = functools.partial(parse_json_object, flag_name="--profiler-config")
def build_image_to_video_prompt(
prompt: str,
negative_prompt: str | None,
media_inputs: dict[str, Any],
) -> dict[str, Any]:
"""Build the canonical request envelope for the shared I2V example."""
result: dict[str, Any] = {
"prompt": prompt,
"modalities": ["video"],
"multi_modal_data": media_inputs,
}
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
def parse_args() -> argparse.Namespace: def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
description=( description=(
@@ -540,13 +522,10 @@ def main():
# Preserve the historical empty-prompt behavior for non-LTX examples. # Preserve the historical empty-prompt behavior for non-LTX examples.
negative_prompt = "" negative_prompt = ""
prompt_dict = build_image_to_video_prompt( prompt_dict = build_image_to_video_prompt(
model_class_name=model_class_name,
prompt=args.prompt, prompt=args.prompt,
negative_prompt=negative_prompt, negative_prompt=negative_prompt,
media_inputs=media_inputs, media_inputs=media_inputs,
)
prompt_dict = build_model_image_to_video_prompt(
model_class_name=model_class_name,
prompt=prompt_dict,
height=height, height=height,
width=width, width=width,
num_frames=num_frames, num_frames=num_frames,
@@ -57,9 +57,8 @@ python examples/offline_inference/text_to_image/text_to_image.py \
--output lingbot_t2i.png --output lingbot_t2i.png
``` ```
The shared T2I entry point selects the `image` output modality. The LingBot The LingBot prompt builder selects the `image` output modality and requests one
pipeline validates that T2I requests produce one frame and do not include an frame. Passing an input image is not supported in LingBot T2I mode.
input image.
## Quick Start ## Quick Start
@@ -21,9 +21,7 @@ from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.lora.request import LoRARequest from vllm_omni.lora.request import LoRARequest
from vllm_omni.lora.utils import stable_lora_int_id from vllm_omni.lora.utils import stable_lora_int_id
from vllm_omni.model_extras import ( from vllm_omni.model_extras import (
build_text_to_image_prompt as build_model_text_to_image_prompt, build_text_to_image_prompt,
)
from vllm_omni.model_extras import (
get_extra_body_params, get_extra_body_params,
get_model_class_name, get_model_class_name,
should_init_extra_args_for_non_diffusion_stages, should_init_extra_args_for_non_diffusion_stages,
@@ -58,17 +56,6 @@ def parse_json_object(value: str, flag_name: str = "argument") -> dict[str, Any]
parse_profiler_config = functools.partial(parse_json_object, flag_name="--profiler-config") parse_profiler_config = functools.partial(parse_json_object, flag_name="--profiler-config")
def build_text_to_image_prompt(prompt: str, negative_prompt: str | None) -> dict[str, Any]:
"""Build the canonical request envelope for the shared T2I example."""
result: dict[str, Any] = {
"prompt": prompt,
"modalities": ["image"],
}
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
def _normalize_images_for_save(images: list[Any]) -> list[Any]: def _normalize_images_for_save(images: list[Any]) -> list[Any]:
"""Convert NumPy diffusion outputs to PIL images before saving.""" """Convert NumPy diffusion outputs to PIL images before saving."""
normalized = [] normalized = []
@@ -518,12 +505,9 @@ def main():
generation_start = time.perf_counter() generation_start = time.perf_counter()
prompt_dict = build_text_to_image_prompt( prompt_dict = build_text_to_image_prompt(
model_class_name=model_class_name,
prompt=args.prompt, prompt=args.prompt,
negative_prompt=args.negative_prompt, negative_prompt=args.negative_prompt,
)
prompt_dict = build_model_text_to_image_prompt(
model_class_name=model_class_name,
prompt=prompt_dict,
height=args.height, height=args.height,
width=args.width, width=args.width,
) )
@@ -1,9 +1,6 @@
# Image-To-Video # Image-To-Video
This example demonstrates online image-to-video generation with vLLM-Omni. This example demonstrates how to deploy the Wan2.2 image-to-video model for online video generation using vLLM-Omni.
The startup script defaults to Wan2.2, but `MODEL` can select any supported
image-to-video model. The existing curl helper remains a concrete Wan2.2
request through the standard Videos API.
## Start Server ## Start Server
@@ -23,7 +20,9 @@ bash run_server.sh
The script allows overriding: The script allows overriding:
- `MODEL` (default: `Wan-AI/Wan2.2-I2V-A14B-Diffusers`) - `MODEL` (default: `Wan-AI/Wan2.2-I2V-A14B-Diffusers`)
- `PORT` (default: `8099`) - `PORT` (default: `8091`)
- `BOUNDARY_RATIO` (default: `0.875`)
- `FLOW_SHIFT` (default: `12.0`)
- `CACHE_BACKEND` (default: `none`) - `CACHE_BACKEND` (default: `none`)
- `ENABLE_CACHE_DIT_SUMMARY` (default: `0`) - `ENABLE_CACHE_DIT_SUMMARY` (default: `0`)
@@ -1,12 +1,12 @@
#!/bin/bash #!/bin/bash
# Shared image-to-video server start script # Wan2.2 image-to-video server start script
MODEL="${MODEL:-Wan-AI/Wan2.2-I2V-A14B-Diffusers}" MODEL="${MODEL:-Wan-AI/Wan2.2-I2V-A14B-Diffusers}"
PORT="${PORT:-8099}" PORT="${PORT:-8099}"
CACHE_BACKEND="${CACHE_BACKEND:-none}" CACHE_BACKEND="${CACHE_BACKEND:-none}"
ENABLE_CACHE_DIT_SUMMARY="${ENABLE_CACHE_DIT_SUMMARY:-0}" ENABLE_CACHE_DIT_SUMMARY="${ENABLE_CACHE_DIT_SUMMARY:-0}"
echo "Starting image-to-video server..." echo "Starting Wan2.2 I2V server..."
echo "Model: $MODEL" echo "Model: $MODEL"
echo "Port: $PORT" echo "Port: $PORT"
echo "Cache backend: $CACHE_BACKEND" echo "Cache backend: $CACHE_BACKEND"
@@ -1,9 +1,6 @@
# Text-To-Image # Text-To-Image
This example demonstrates online text-to-image generation with vLLM-Omni. The This example demonstrates how to deploy Qwen-Image model for online image generation service using vLLM-Omni.
startup script and Python clients default to Qwen-Image, but `MODEL` can select
any supported text-to-image model. The existing curl helper remains a concrete
Qwen-Image request through the standard Images API.
## Start Server ## Start Server
@@ -229,8 +226,8 @@ count, use `size` and `n` rather than `height`, `width`, or
| `height` | int | None | Image height in pixels | | `height` | int | None | Image height in pixels |
| `width` | int | None | Image width in pixels | | `width` | int | None | Image width in pixels |
| `size` | str | None | Image size (e.g., "1024x1024") | | `size` | str | None | Image size (e.g., "1024x1024") |
| `num_inference_steps` | int | model | Number of denoising steps | | `num_inference_steps` | int | 50 | Number of denoising steps |
| `true_cfg_scale` | float | model | True CFG scale when supported | | `true_cfg_scale` | float | 4.0 | Qwen-Image CFG scale |
| `seed` | int | None | Random seed (reproducible) | | `seed` | int | None | Random seed (reproducible) |
| `negative_prompt` | str | None | Negative prompt | | `negative_prompt` | str | None | Negative prompt |
| `num_outputs_per_prompt` | int | 1 | Number of images to generate | | `num_outputs_per_prompt` | int | 1 | Number of images to generate |
@@ -1,6 +1,6 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Shared text-to-image Gradio demo for online serving. Qwen-Image Gradio Demo for online serving.
Usage: Usage:
python gradio_demo.py [--server http://localhost:8091] [--port 7860] python gradio_demo.py [--server http://localhost:8091] [--port 7860]
@@ -77,9 +77,9 @@ def generate_image(
def create_demo(server_url: str): def create_demo(server_url: str):
"""Create Gradio demo interface.""" """Create Gradio demo interface."""
with gr.Blocks(title="Text-to-Image Demo") as demo: with gr.Blocks(title="Qwen-Image Demo") as demo:
gr.Markdown("# Text-to-Image Online Generation") gr.Markdown("# Qwen-Image Online Generation")
gr.Markdown("Generate images using the model configured on the server") gr.Markdown("Generate images using Qwen-Image model")
with gr.Row(): with gr.Row():
with gr.Column(scale=1): with gr.Column(scale=1):
@@ -115,6 +115,7 @@ def create_demo(server_url: str):
label="Inference Steps", label="Inference Steps",
minimum=10, minimum=10,
maximum=100, maximum=100,
# Default steps aligned with run_curl_text_to_image.sh to 100
value=100, value=100,
step=5, step=5,
) )
@@ -172,7 +173,7 @@ def create_demo(server_url: str):
def main(): def main():
parser = argparse.ArgumentParser(description="Shared text-to-image Gradio demo") parser = argparse.ArgumentParser(description="Qwen-Image Gradio Demo")
parser.add_argument("--server", default="http://localhost:8091", help="Server URL") parser.add_argument("--server", default="http://localhost:8091", help="Server URL")
parser.add_argument("--port", type=int, default=7860, help="Gradio port") parser.add_argument("--port", type=int, default=7860, help="Gradio port")
parser.add_argument("--share", action="store_true", help="Create public link") parser.add_argument("--share", action="store_true", help="Create public link")
@@ -1,6 +1,6 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
""" """
Shared OpenAI-compatible text-to-image generation client. Qwen-Image OpenAI-compatible image generation client.
Usage: Usage:
python openai_chat_client.py --prompt "A beautiful landscape" --output output.png python openai_chat_client.py --prompt "A beautiful landscape" --output output.png
@@ -39,7 +39,7 @@ def generate_image(
height: Image height in pixels height: Image height in pixels
width: Image width in pixels width: Image width in pixels
steps: Number of diffusion steps steps: Number of diffusion steps
true_cfg_scale: True CFG scale for models that support it true_cfg_scale: Qwen-Image CFG scale
seed: Random seed seed: Random seed
negative_prompt: Negative prompt negative_prompt: Negative prompt
num_outputs_per_prompt: Number of images to generate num_outputs_per_prompt: Number of images to generate
@@ -70,7 +70,7 @@ def generate_image(
payload["num_inference_steps"] = steps payload["num_inference_steps"] = steps
if true_cfg_scale is not None: if true_cfg_scale is not None:
payload["true_cfg_scale"] = true_cfg_scale payload["true_cfg_scale"] = true_cfg_scale
if negative_prompt is not None: if negative_prompt:
payload["negative_prompt"] = negative_prompt payload["negative_prompt"] = negative_prompt
if seed is not None: if seed is not None:
payload["seed"] = seed payload["seed"] = seed
@@ -115,15 +115,15 @@ def generate_image(
def main(): def main():
parser = argparse.ArgumentParser(description="Shared text-to-image client") parser = argparse.ArgumentParser(description="Qwen-Image chat client")
parser.add_argument("--prompt", "-p", default="a cup of coffee on the table", help="Text prompt") parser.add_argument("--prompt", "-p", default="a cup of coffee on the table", help="Text prompt")
parser.add_argument("--output", "-o", default="text_to_image_output.png", help="Output file") parser.add_argument("--output", "-o", default="qwen_image_output.png", help="Output file")
parser.add_argument("--server", "-s", default="http://localhost:8091", help="Server URL") parser.add_argument("--server", "-s", default="http://localhost:8091", help="Server URL")
parser.add_argument("--height", type=int, help="Image height; defaults to the model setting") parser.add_argument("--height", type=int, default=1024, help="Image height")
parser.add_argument("--width", type=int, help="Image width; defaults to the model setting") parser.add_argument("--width", type=int, default=1024, help="Image width")
parser.add_argument("--steps", type=int, help="Inference steps; defaults to the model setting") parser.add_argument("--steps", type=int, default=50, help="Inference steps")
parser.add_argument("--cfg-scale", type=float, help="True CFG scale when supported") parser.add_argument("--cfg-scale", type=float, default=4.0, help="True CFG scale")
parser.add_argument("--seed", type=int, help="Random seed") parser.add_argument("--seed", type=int, default=0, help="Random seed")
parser.add_argument("--negative", help="Negative prompt") parser.add_argument("--negative", help="Negative prompt")
parser.add_argument("--lora-path", default=None, help="Server-local LoRA adapter folder (PEFT format)") parser.add_argument("--lora-path", default=None, help="Server-local LoRA adapter folder (PEFT format)")
@@ -1,10 +1,10 @@
#!/bin/bash #!/bin/bash
# Shared text-to-image online serving startup script # Qwen-Image online serving startup script
MODEL="${MODEL:-Qwen/Qwen-Image}" MODEL="${MODEL:-Qwen/Qwen-Image}"
PORT="${PORT:-8091}" PORT="${PORT:-8091}"
echo "Starting text-to-image server..." echo "Starting Qwen-Image server..."
echo "Model: $MODEL" echo "Model: $MODEL"
echo "Port: $PORT" echo "Port: $PORT"
-18
View File
@@ -2206,24 +2206,6 @@ class TestBaseConfigInheritance:
class TestPlatformOverrides: class TestPlatformOverrides:
"""Test platform-specific deploy config overrides.""" """Test platform-specific deploy config overrides."""
def test_qwen3_omni_musa_clears_parent_quantization_for_audio_stages(self):
pipeline = resolve_pipeline_config(
"qwen3_omni_moe",
Q3_OMNI_ALL_STAGES_HF_CONFIG,
)
assert isinstance(pipeline, PipelineConfig)
deploy_path = Path(get_deploy_config_path("qwen3_omni_moe.yaml"))
base_stages = merge_pipeline_deploy(pipeline, load_deploy_config(deploy_path))
assert "hf_overrides" not in base_stages[1].yaml_engine_args
assert "hf_overrides" not in base_stages[2].yaml_engine_args
musa = _apply_platform_overrides(load_deploy_config(deploy_path), platform="musa")
musa_stages = merge_pipeline_deploy(pipeline, musa)
expected = {"quantization_config": None}
assert musa_stages[1].yaml_engine_args["hf_overrides"] == expected
assert musa_stages[2].yaml_engine_args["hf_overrides"] == expected
def test_qwen3_tts_rocm_disables_code2wav_outer_cudagraph(self): def test_qwen3_tts_rocm_disables_code2wav_outer_cudagraph(self):
deploy_path = Path(get_deploy_config_path("qwen3_tts.yaml")) deploy_path = Path(get_deploy_config_path("qwen3_tts.yaml"))
@@ -1,47 +0,0 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
from __future__ import annotations
import pytest
from PIL import Image
from examples.offline_inference.image_to_video.image_to_video import build_image_to_video_prompt
from examples.offline_inference.text_to_image.text_to_image import build_text_to_image_prompt
pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
@pytest.mark.parametrize(
("negative_prompt", "expected_negative_prompt"),
[(None, None), ("", ""), ("blurry", "blurry")],
)
def test_text_to_image_builds_canonical_prompt(
negative_prompt: str | None,
expected_negative_prompt: str | None,
) -> None:
result = build_text_to_image_prompt("a red fox", negative_prompt)
assert result["prompt"] == "a red fox"
assert result["modalities"] == ["image"]
if expected_negative_prompt is None:
assert "negative_prompt" not in result
else:
assert result["negative_prompt"] == expected_negative_prompt
def test_image_to_video_builds_canonical_prompt() -> None:
image = Image.new("RGB", (32, 16), "red")
result = build_image_to_video_prompt(
prompt="the fox turns toward the camera",
negative_prompt="flicker",
media_inputs={"image": image},
)
assert result == {
"prompt": "the fox turns toward the camera",
"modalities": ["video"],
"multi_modal_data": {"image": image},
"negative_prompt": "flicker",
}
+107 -38
View File
@@ -3,8 +3,6 @@
from __future__ import annotations from __future__ import annotations
from typing import Any
import pytest import pytest
from PIL import Image from PIL import Image
@@ -137,6 +135,67 @@ def test_lingbot_extra_registry_declares_request_params() -> None:
assert should_init_extra_args_for_non_diffusion_stages("LingBotVideoPipeline") is False assert should_init_extra_args_for_non_diffusion_stages("LingBotVideoPipeline") is False
@pytest.mark.core_model
@pytest.mark.cpu
def test_lingbot_text_to_image_prompt_builder_preserves_empty_negative_prompt() -> None:
assert build_text_to_image_prompt(
"LingBotVideoPipeline",
prompt="a red fox in fresh snow",
negative_prompt="",
height=192,
width=320,
) == {
"prompt": "a red fox in fresh snow",
"modalities": ["image"],
"negative_prompt": "",
}
@pytest.mark.core_model
@pytest.mark.cpu
def test_lingbot_image_to_video_prompt_builder() -> None:
image = Image.new("RGB", (320, 192), "red")
result = build_image_to_video_prompt(
"LingBotVideoPipeline",
prompt="the fox looks toward the camera",
negative_prompt=None,
media_inputs={"image": image},
height=192,
width=320,
num_frames=9,
)
assert result == {
"prompt": "the fox looks toward the camera",
"modalities": ["video"],
"multi_modal_data": {"image": image},
"num_frames": 9,
}
@pytest.mark.core_model
@pytest.mark.cpu
@pytest.mark.parametrize(
"media_inputs",
[
{},
{"image": "input.png"},
{"image": [Image.new("RGB", (16, 16))]},
{"image": Image.new("RGB", (16, 16)), "video": b"video"},
],
ids=["missing", "path", "multiple", "extra-modality"],
)
def test_lingbot_image_to_video_prompt_builder_rejects_invalid_media(
media_inputs: dict[str, object],
) -> None:
with pytest.raises(ValueError, match="exactly one PIL image"):
build_image_to_video_prompt(
"LingBotVideoPipeline",
prompt="move",
negative_prompt=None,
media_inputs=media_inputs,
)
@pytest.mark.core_model @pytest.mark.core_model
@pytest.mark.cpu @pytest.mark.cpu
@pytest.mark.parametrize("pipeline_name", ["Cosmos3OmniDiffusersPipeline", "Cosmos3OmniPipeline"]) @pytest.mark.parametrize("pipeline_name", ["Cosmos3OmniDiffusersPipeline", "Cosmos3OmniPipeline"])
@@ -283,11 +342,8 @@ def test_ming_flash_omni_declared_extra_args_route_into_sampling_params() -> Non
def test_ming_flash_omni_text_to_image_prompt_builder() -> None: def test_ming_flash_omni_text_to_image_prompt_builder() -> None:
assert build_text_to_image_prompt( assert build_text_to_image_prompt(
"MingImagePipeline", "MingImagePipeline",
{ prompt="Please draw a cute cat.",
"prompt": "Please draw a cute cat.", negative_prompt="ugly, blurry",
"modalities": ["image"],
"negative_prompt": "ugly, blurry",
},
height=512, height=512,
width=768, width=768,
) == { ) == {
@@ -304,7 +360,8 @@ def test_ming_flash_omni_text_to_image_prompt_builder() -> None:
# pipeline's 1024x1024 default still applies. # pipeline's 1024x1024 default still applies.
assert build_text_to_image_prompt( assert build_text_to_image_prompt(
"MingImagePipeline", "MingImagePipeline",
{"prompt": "Draw a poster.", "modalities": ["image"]}, prompt="Draw a poster.",
negative_prompt=None,
) == { ) == {
"prompt": "Draw a poster.", "prompt": "Draw a poster.",
"modalities": ["image"], "modalities": ["image"],
@@ -334,6 +391,28 @@ def test_ming_flash_omni_image_to_image_prompt_builder() -> None:
assert "negative_prompt" not in result assert "negative_prompt" not in result
@pytest.mark.core_model
@pytest.mark.cpu
@pytest.mark.parametrize("pipeline_name", ["Cosmos3OmniDiffusersPipeline", "Cosmos3OmniPipeline"])
def test_cosmos3_text_to_image_prompt_builder_selects_image_modality(pipeline_name: str) -> None:
assert build_text_to_image_prompt(
pipeline_name,
prompt="a red sports car at golden hour",
negative_prompt="blurry, distorted",
height=1024,
width=1024,
) == {
"prompt": "a red sports car at golden hour",
"modalities": ["image"],
"negative_prompt": "blurry, distorted",
}
assert build_text_to_image_prompt(
pipeline_name,
prompt="a red sports car",
negative_prompt=None,
) == {"prompt": "a red sports car", "modalities": ["image"]}
@pytest.mark.core_model @pytest.mark.core_model
@pytest.mark.cpu @pytest.mark.cpu
def test_audiox_extra_registry_declares_request_and_response_params() -> None: def test_audiox_extra_registry_declares_request_and_response_params() -> None:
@@ -426,11 +505,8 @@ def test_unknown_pipeline_has_empty_extra_registry() -> None:
def test_bagel_text_to_image_prompt_builder() -> None: def test_bagel_text_to_image_prompt_builder() -> None:
assert build_text_to_image_prompt( assert build_text_to_image_prompt(
"BagelPipeline", "BagelPipeline",
{ prompt="a cat",
"prompt": "a cat", negative_prompt="blurry",
"modalities": ["image"],
"negative_prompt": "blurry",
},
height=512, height=512,
width=768, width=768,
) == { ) == {
@@ -467,17 +543,14 @@ def test_bagel_image_to_image_prompt_builder() -> None:
@pytest.mark.core_model @pytest.mark.core_model
@pytest.mark.cpu @pytest.mark.cpu
def test_unknown_pipeline_preserves_canonical_text_to_image_prompt() -> None: def test_unknown_pipeline_uses_default_text_to_image_prompt() -> None:
canonical_prompt = {"prompt": "a cat", "modalities": ["image"]} assert build_text_to_image_prompt(
assert ( "UnknownPipeline",
build_text_to_image_prompt( prompt="a cat",
"UnknownPipeline", negative_prompt=None,
canonical_prompt, height=512,
height=512, width=512,
width=512, ) == {"prompt": "a cat"}
)
is canonical_prompt
)
@pytest.mark.core_model @pytest.mark.core_model
@@ -496,14 +569,12 @@ def test_unknown_pipeline_uses_default_image_to_image_prompt() -> None:
} }
def _build_vace_prompt(media_inputs: dict[str, Any], *, num_frames: int = 5) -> dict: def _build_vace_prompt(media_inputs: dict[str, object], *, num_frames: int = 5) -> dict:
return build_image_to_video_prompt( return build_image_to_video_prompt(
"WanVACEPipeline", "WanVACEPipeline",
{ prompt="a bird flying",
"prompt": "a bird flying", negative_prompt=None,
"modalities": ["video"], media_inputs=media_inputs,
"multi_modal_data": media_inputs,
},
height=16, height=16,
width=320, width=320,
num_frames=num_frames, num_frames=num_frames,
@@ -526,7 +597,7 @@ def _build_vace_prompt(media_inputs: dict[str, Any], *, num_frames: int = 5) ->
], ],
ids=["i2v", "v2lf", "flf2v", "inpaint", "r2v"], ids=["i2v", "v2lf", "flf2v", "inpaint", "r2v"],
) )
def test_vace_image_to_video_prompt_builder(media_inputs: dict[str, Any]) -> None: def test_vace_image_to_video_prompt_builder(media_inputs: dict[str, object]) -> None:
result = _build_vace_prompt(media_inputs) result = _build_vace_prompt(media_inputs)
mmd = result["multi_modal_data"] mmd = result["multi_modal_data"]
if "reference_images" in mmd: if "reference_images" in mmd:
@@ -545,7 +616,7 @@ def test_vace_image_to_video_prompt_builder(media_inputs: dict[str, Any]) -> Non
({"control_image": Image.new("RGB", (320, 16))}, "Unsupported VACE media input"), ({"control_image": Image.new("RGB", (320, 16))}, "Unsupported VACE media input"),
], ],
) )
def test_vace_rejects_invalid_media_combinations(media_inputs: dict[str, Any], message: str) -> None: def test_vace_rejects_invalid_media_combinations(media_inputs: dict[str, object], message: str) -> None:
with pytest.raises(ValueError, match=message): with pytest.raises(ValueError, match=message):
_build_vace_prompt(media_inputs) _build_vace_prompt(media_inputs)
@@ -593,7 +664,8 @@ def test_mammothmoda2_extra_registry_declares_request_and_response_params() -> N
wrapper_prompt = build_text_to_image_prompt( wrapper_prompt = build_text_to_image_prompt(
"MammothModa2ForConditionalGeneration", "MammothModa2ForConditionalGeneration",
{"prompt": "a cat", "modalities": ["image"]}, prompt="a cat",
negative_prompt=None,
height=256, height=256,
width=256, width=256,
) )
@@ -607,11 +679,8 @@ def test_mammothmoda2_text_to_image_prompt_builder() -> None:
# prompt is ignored (MammothModa2 t2i uses CFG, not an explicit negative path). # prompt is ignored (MammothModa2 t2i uses CFG, not an explicit negative path).
assert build_text_to_image_prompt( assert build_text_to_image_prompt(
"MammothModa2DiTPipeline", "MammothModa2DiTPipeline",
{ prompt="a cat",
"prompt": "a cat", negative_prompt="blurry",
"modalities": ["image"],
"negative_prompt": "blurry",
},
height=512, height=512,
width=768, width=768,
) == { ) == {
+1 -76
View File
@@ -3,88 +3,13 @@ from types import SimpleNamespace
import pytest import pytest
import torch import torch
from vllm.v1.cudagraph_dispatcher import CUDAGraphMode
from vllm_omni.worker.gpu_model_runner import ( from vllm_omni.worker.gpu_model_runner import OmniGPUModelRunner, _filter_mrope_kwargs_for_model
OmniGPUModelRunner,
_filter_mrope_kwargs_for_model,
)
from vllm_omni.worker.omni_connector_model_runner_mixin import OmniConnectorModelRunnerMixin from vllm_omni.worker.omni_connector_model_runner_mixin import OmniConnectorModelRunnerMixin
pytestmark = [pytest.mark.core_model, pytest.mark.cpu] pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
def _runner_for_talker_graph_init(
*,
talker_mtp_graph_safe: bool | None,
has_separate_talker: bool = True,
model_stage: str = "talker",
) -> OmniGPUModelRunner:
runner = object.__new__(OmniGPUModelRunner)
runner.model = SimpleNamespace(
talker=object() if has_separate_talker else None,
talker_mtp=object(),
model_stage=model_stage,
)
if talker_mtp_graph_safe is not None:
runner.model.talker_mtp_graph_safe = talker_mtp_graph_safe
runner.model_config = SimpleNamespace(hf_text_config=SimpleNamespace(hidden_size=4))
runner.compilation_config = SimpleNamespace(
cudagraph_mode=CUDAGraphMode.FULL,
max_cudagraph_capture_size=1,
)
runner.vllm_config = object()
runner.max_num_reqs = 1
runner.dtype = torch.float32
runner._make_buffer = lambda *args, **kwargs: SimpleNamespace(args=args, kwargs=kwargs)
return runner
def test_talker_mtp_skips_graph_when_model_declares_unsafe(monkeypatch):
runner = _runner_for_talker_graph_init(talker_mtp_graph_safe=False)
talker_mtp = runner.model.talker_mtp
monkeypatch.setattr(
"vllm_omni.worker.gpu_model_runner.current_omni_platform.get_graph_wrapper_cls",
lambda: pytest.fail("graph wrapper must not be selected"),
)
OmniGPUModelRunner._init_talker_mtp(runner)
assert runner.talker_mtp is talker_mtp
@pytest.mark.parametrize("model_stage", ["thinker", "code2wav"])
def test_non_talker_stage_does_not_use_talker_mtp_graph(monkeypatch, model_stage: str):
runner = _runner_for_talker_graph_init(
talker_mtp_graph_safe=None,
has_separate_talker=False,
model_stage=model_stage,
)
talker_mtp = runner.model.talker_mtp
monkeypatch.setattr(
"vllm_omni.worker.gpu_model_runner.current_omni_platform.get_graph_wrapper_cls",
lambda: pytest.fail("graph wrapper must not be selected"),
)
OmniGPUModelRunner._init_talker_mtp(runner)
assert runner.talker_mtp is talker_mtp
@pytest.mark.parametrize("talker_mtp_graph_safe", [None, True])
def test_talker_mtp_uses_graph_for_legacy_or_explicit_safe_model(monkeypatch, talker_mtp_graph_safe):
runner = _runner_for_talker_graph_init(talker_mtp_graph_safe=talker_mtp_graph_safe)
wrapped = object()
monkeypatch.setattr(
"vllm_omni.worker.gpu_model_runner.current_omni_platform.get_graph_wrapper_cls",
lambda: lambda *args, **kwargs: wrapped,
)
OmniGPUModelRunner._init_talker_mtp(runner)
assert runner.talker_mtp is wrapped
class DummyBuffer: class DummyBuffer:
"""A minimal buffer wrapper that exposes the `.gpu` attribute.""" """A minimal buffer wrapper that exposes the `.gpu` attribute."""
-12
View File
@@ -69,18 +69,6 @@ stages:
repetition_penalty: 1.1 repetition_penalty: 1.1
platforms: platforms:
# The supported Qwen3-Omni ModelOpt checkpoint quantizes the thinker only. Clear the
# checkpoint-level metadata for the BF16 audio stages on the verified MUSA
# profile before vLLM performs its normal quantization auto-detection.
musa:
stages:
- stage_id: 1
hf_overrides:
quantization_config: null
- stage_id: 2
hf_overrides:
quantization_config: null
npu: npu:
stages: stages:
- stage_id: 0 - stage_id: 0
@@ -152,9 +152,6 @@ class Qwen3OmniMoeForConditionalGeneration(
dtype=torch.long, dtype=torch.long,
) )
elif self.model_stage == "talker": elif self.model_stage == "talker":
# The outer wrapper exposes talker_mtp for every stage, but only
# the talker stage owns the module that the method invokes.
self.talker_mtp_graph_safe = current_omni_platform.supports_talker_mtp_graph_capture()
multimodal_config.skip_mm_profiling = True multimodal_config.skip_mm_profiling = True
self.has_preprocess = True self.has_preprocess = True
self.has_postprocess = True self.has_postprocess = True
+26
View File
@@ -3,6 +3,32 @@
from __future__ import annotations from __future__ import annotations
from typing import Any
def build_text_to_image_prompt(
prompt: str,
negative_prompt: str | None,
height: int | None = None,
width: int | None = None,
) -> dict[str, Any]:
"""Build a Cosmos3 T2I prompt by selecting image output modality.
Cosmos3 uses the same pipeline class for T2I and video modes. The OpenAI
image endpoint selects T2I by adding ``modalities=["image"]``. ``height``
and ``width`` are accepted for registry compatibility; sizing travels
through sampling params instead of the prompt payload.
"""
del height, width
text_prompt: dict[str, Any] = {
"prompt": prompt,
"modalities": ["image"],
}
if negative_prompt is not None:
text_prompt["negative_prompt"] = negative_prompt
return text_prompt
COSMOS3_EXTRA_BODY_PARAMS = frozenset( COSMOS3_EXTRA_BODY_PARAMS = frozenset(
{ {
"flow_shift", "flow_shift",
+45
View File
@@ -3,6 +3,11 @@
from __future__ import annotations from __future__ import annotations
from collections.abc import Mapping
from typing import Any
from PIL import Image
LINGBOT_VIDEO_EXTRA_BODY_PARAMS = frozenset( LINGBOT_VIDEO_EXTRA_BODY_PARAMS = frozenset(
{ {
"batch_cfg", "batch_cfg",
@@ -19,3 +24,43 @@ LINGBOT_VIDEO_EXTRA_BODY_PARAMS = frozenset(
"t_thresh", "t_thresh",
} }
) )
def build_text_to_image_prompt(
prompt: str,
negative_prompt: str | None,
height: int | None = None,
width: int | None = None,
) -> dict[str, Any]:
del height, width
result: dict[str, Any] = {
"prompt": prompt,
"modalities": ["image"],
}
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
def build_image_to_video_prompt(
prompt: str,
negative_prompt: str | None,
media_inputs: Mapping[str, Any],
height: int | None = None,
width: int | None = None,
num_frames: int | None = None,
) -> dict[str, Any]:
del height, width
if set(media_inputs) != {"image"} or not isinstance(media_inputs["image"], Image.Image):
raise ValueError("LingBot text-image-to-video requires exactly one PIL image input.")
result: dict[str, Any] = {
"prompt": prompt,
"modalities": ["video"],
"multi_modal_data": {"image": media_inputs["image"]},
}
if num_frames is not None:
result["num_frames"] = num_frames
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
+55 -29
View File
@@ -28,12 +28,21 @@ from vllm_omni.model_extras.cosmos3 import (
COSMOS3_EXTRA_BODY_PARAMS, COSMOS3_EXTRA_BODY_PARAMS,
COSMOS3_EXTRA_OUTPUT_PARAMS, COSMOS3_EXTRA_OUTPUT_PARAMS,
) )
from vllm_omni.model_extras.cosmos3 import (
build_text_to_image_prompt as build_cosmos3_text_to_image_prompt,
)
from vllm_omni.model_extras.helios import ( from vllm_omni.model_extras.helios import (
HELIOS_EXTRA_BODY_PARAMS, HELIOS_EXTRA_BODY_PARAMS,
HELIOS_EXTRA_OUTPUT_PARAMS, HELIOS_EXTRA_OUTPUT_PARAMS,
) )
from vllm_omni.model_extras.hunyuan_image3 import build_x_to_text_prompt as build_hunyuan_x_to_text_prompt from vllm_omni.model_extras.hunyuan_image3 import build_x_to_text_prompt as build_hunyuan_x_to_text_prompt
from vllm_omni.model_extras.lingbot_video import LINGBOT_VIDEO_EXTRA_BODY_PARAMS from vllm_omni.model_extras.lingbot_video import LINGBOT_VIDEO_EXTRA_BODY_PARAMS
from vllm_omni.model_extras.lingbot_video import (
build_image_to_video_prompt as build_lingbot_image_to_video_prompt,
)
from vllm_omni.model_extras.lingbot_video import (
build_text_to_image_prompt as build_lingbot_text_to_image_prompt,
)
from vllm_omni.model_extras.ltx2 import LTX_EXTRA_BODY_PARAMS, LTX_EXTRA_OUTPUT_PARAMS from vllm_omni.model_extras.ltx2 import LTX_EXTRA_BODY_PARAMS, LTX_EXTRA_OUTPUT_PARAMS
from vllm_omni.model_extras.magi_human import ( from vllm_omni.model_extras.magi_human import (
MAGI_HUMAN_EXTRA_BODY_PARAMS, MAGI_HUMAN_EXTRA_BODY_PARAMS,
@@ -138,6 +147,18 @@ def build_x_to_text_prompt(
return builder(model, prompt, has_image) return builder(model, prompt, has_image)
def default_text_to_image_prompt(
prompt: str,
negative_prompt: str | None,
height: int | None = None,
width: int | None = None,
) -> dict[str, Any]:
result: dict[str, Any] = {"prompt": prompt}
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
def default_image_to_image_prompt( def default_image_to_image_prompt(
prompt: str, prompt: str,
negative_prompt: str | None, negative_prompt: str | None,
@@ -154,6 +175,20 @@ def default_image_to_image_prompt(
return result return result
def default_image_to_video_prompt(
prompt: str,
negative_prompt: str | None,
media_inputs: Mapping[str, Any],
height: int | None = None,
width: int | None = None,
num_frames: int | None = None,
) -> dict[str, Any]:
del height, width, num_frames
if set(media_inputs) != {"image"} or not isinstance(media_inputs["image"], Image.Image):
raise ValueError("This model only supports a single --image input in the shared image-to-video example.")
return default_image_to_image_prompt(prompt, negative_prompt, media_inputs["image"])
_EXTRA_SPECS: dict[str, dict[str, Any]] = { _EXTRA_SPECS: dict[str, dict[str, Any]] = {
"AudioXPipeline": { "AudioXPipeline": {
"extra_body_params": AUDIOX_EXTRA_BODY_PARAMS, "extra_body_params": AUDIOX_EXTRA_BODY_PARAMS,
@@ -173,11 +208,12 @@ _EXTRA_SPECS: dict[str, dict[str, Any]] = {
"Cosmos3OmniDiffusersPipeline": { "Cosmos3OmniDiffusersPipeline": {
"extra_body_params": COSMOS3_EXTRA_BODY_PARAMS, "extra_body_params": COSMOS3_EXTRA_BODY_PARAMS,
"extra_output_params": COSMOS3_EXTRA_OUTPUT_PARAMS, "extra_output_params": COSMOS3_EXTRA_OUTPUT_PARAMS,
# The shared T2I example already supplies modalities=["image"]. "text_to_image_prompt_builder": build_cosmos3_text_to_image_prompt,
}, },
"Cosmos3OmniPipeline": { "Cosmos3OmniPipeline": {
"extra_body_params": COSMOS3_EXTRA_BODY_PARAMS, "extra_body_params": COSMOS3_EXTRA_BODY_PARAMS,
"extra_output_params": COSMOS3_EXTRA_OUTPUT_PARAMS, "extra_output_params": COSMOS3_EXTRA_OUTPUT_PARAMS,
"text_to_image_prompt_builder": build_cosmos3_text_to_image_prompt,
}, },
"MagiHumanPipeline": { "MagiHumanPipeline": {
"extra_body_params": MAGI_HUMAN_EXTRA_BODY_PARAMS, "extra_body_params": MAGI_HUMAN_EXTRA_BODY_PARAMS,
@@ -193,8 +229,8 @@ _EXTRA_SPECS: dict[str, dict[str, Any]] = {
}, },
"LingBotVideoPipeline": { "LingBotVideoPipeline": {
"extra_body_params": LINGBOT_VIDEO_EXTRA_BODY_PARAMS, "extra_body_params": LINGBOT_VIDEO_EXTRA_BODY_PARAMS,
# Shared T2I/I2V envelopes select the output modality. LingBot's "text_to_image_prompt_builder": build_lingbot_text_to_image_prompt,
# pipeline owns model-specific validation and normalization. "image_to_video_prompt_builder": build_lingbot_image_to_video_prompt,
}, },
**{ **{
model_class_name: { model_class_name: {
@@ -227,7 +263,7 @@ _EXTRA_SPECS: dict[str, dict[str, Any]] = {
} }
# Multi-stage discovery reports the top-level wrapper rather than its DiT # Multi-stage discovery reports the top-level wrapper rather than its DiT
# submodule, so both names must resolve to the same request builders. # submodule, so both names must resolve to the same request adapters.
_EXTRA_SPECS["MammothModa2ForConditionalGeneration"] = _EXTRA_SPECS["MammothModa2DiTPipeline"] _EXTRA_SPECS["MammothModa2ForConditionalGeneration"] = _EXTRA_SPECS["MammothModa2DiTPipeline"]
_EXTRA_SPECS["Mammothmoda2Model"] = _EXTRA_SPECS["MammothModa2DiTPipeline"] _EXTRA_SPECS["Mammothmoda2Model"] = _EXTRA_SPECS["MammothModa2DiTPipeline"]
@@ -269,21 +305,18 @@ def should_init_extra_args_for_non_diffusion_stages(model_class_name: str | None
def build_text_to_image_prompt( def build_text_to_image_prompt(
model_class_name: str | None, model_class_name: str | None,
prompt: dict[str, Any], prompt: str,
negative_prompt: str | None,
height: int | None = None, height: int | None = None,
width: int | None = None, width: int | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Build a model-specific T2I prompt from an example-owned envelope."""
spec = _get_spec(model_class_name) spec = _get_spec(model_class_name)
builder: TextToImagePromptBuilder | None = spec.get("text_to_image_prompt_builder") if spec else None builder: TextToImagePromptBuilder = (
if builder is None: spec.get("text_to_image_prompt_builder", default_text_to_image_prompt)
return prompt if spec is not None
return builder( else default_text_to_image_prompt
prompt=str(prompt["prompt"]),
negative_prompt=prompt.get("negative_prompt"),
height=height,
width=width,
) )
return builder(prompt, negative_prompt, height, width)
def build_image_to_image_prompt( def build_image_to_image_prompt(
@@ -305,24 +338,17 @@ def build_image_to_image_prompt(
def build_image_to_video_prompt( def build_image_to_video_prompt(
model_class_name: str | None, model_class_name: str | None,
prompt: dict[str, Any], prompt: str,
negative_prompt: str | None,
media_inputs: Mapping[str, Any],
height: int | None = None, height: int | None = None,
width: int | None = None, width: int | None = None,
num_frames: int | None = None, num_frames: int | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""Build a model-specific I2V prompt from an example-owned envelope."""
spec = _get_spec(model_class_name) spec = _get_spec(model_class_name)
builder: ImageToVideoPromptBuilder | None = spec.get("image_to_video_prompt_builder") if spec else None builder: ImageToVideoPromptBuilder = (
if builder is None: spec.get("image_to_video_prompt_builder", default_image_to_video_prompt)
return prompt if spec is not None
media_inputs = prompt.get("multi_modal_data") or {} else default_image_to_video_prompt
if not isinstance(media_inputs, Mapping):
raise TypeError("Canonical I2V prompt multi_modal_data must be a mapping.")
return builder(
prompt=str(prompt["prompt"]),
negative_prompt=prompt.get("negative_prompt"),
media_inputs=media_inputs,
height=height,
width=width,
num_frames=num_frames,
) )
return builder(prompt, negative_prompt, media_inputs, height, width, num_frames)
-5
View File
@@ -155,11 +155,6 @@ class OmniPlatform(Platform):
"""Check if the platform supports torch.compile with inductor backend.""" """Check if the platform supports torch.compile with inductor backend."""
raise NotImplementedError raise NotImplementedError
@classmethod
def supports_talker_mtp_graph_capture(cls) -> bool:
"""Whether a model may capture its dedicated talker MTP graph."""
return True
@classmethod @classmethod
def has_flash_attn_package(cls) -> bool: def has_flash_attn_package(cls) -> bool:
"""Check if a Flash Attention package is available and usable on this platform.""" """Check if a Flash Attention package is available and usable on this platform."""
-5
View File
@@ -115,11 +115,6 @@ class MUSAOmniPlatform(OmniPlatform, MUSAPlatformBase):
"""MUSA supports torch.compile with inductor backend.""" """MUSA supports torch.compile with inductor backend."""
return True return True
@classmethod
def supports_talker_mtp_graph_capture(cls) -> bool:
"""MUSA keeps Qwen3 talker MTP outside its dedicated FULL graph."""
return False
@classmethod @classmethod
def supports_float64(cls) -> bool: def supports_float64(cls) -> bool:
"""MUSA does not support float64 yet.""" """MUSA does not support float64 yet."""
+2 -6
View File
@@ -206,12 +206,8 @@ class OmniGPUModelRunner(GPUModelRunner):
cudagraph_mode = self.compilation_config.cudagraph_mode cudagraph_mode = self.compilation_config.cudagraph_mode
assert cudagraph_mode is not None assert cudagraph_mode is not None
has_separate_talker = getattr(self.model, "talker", None) is not None has_separate_talker = getattr(self.model, "talker", None) is not None
talker_mtp_graph_safe = getattr(self.model, "talker_mtp_graph_safe", None) talker_mtp_graph_safe = getattr(self.model, "talker_mtp_graph_safe", False)
# Preserve the legacy separate-talker default only when the model has if cudagraph_mode.has_full_cudagraphs() and (has_separate_talker or talker_mtp_graph_safe):
# not declared graph safety. An explicit False must disable wrapping.
if talker_mtp_graph_safe is None:
talker_mtp_graph_safe = has_separate_talker
if cudagraph_mode.has_full_cudagraphs() and talker_mtp_graph_safe:
graph_wrapper_cls = current_omni_platform.get_graph_wrapper_cls() graph_wrapper_cls = current_omni_platform.get_graph_wrapper_cls()
self.talker_mtp = graph_wrapper_cls(talker_mtp, self.vllm_config, runtime_mode=CUDAGraphMode.FULL) self.talker_mtp = graph_wrapper_cls(talker_mtp, self.vllm_config, runtime_mode=CUDAGraphMode.FULL)
# TTS exposes mtp_hidden_size; Omni uses hf_text_config.hidden_size. # TTS exposes mtp_hidden_size; Omni uses hf_text_config.hidden_size.