Compare commits

..
Author SHA1 Message Date
Hongsheng Liu 7b2f3682f7 [CI/Build] Clarify model-specific example filename
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-08-11 16:36:52 +08:00
Hongsheng Liu 8ef0d3f391 [CI/Build] Reuse examples policy in review-pr skill
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-08-11 16:31:36 +08:00
Hongsheng Liu 49985ec849 [CI/Build] Add examples policy to precheck-pr skill
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-08-11 16:23:31 +08:00
25 changed files with 362 additions and 314 deletions

No files matched your search

+4 -1
View File
@@ -1,6 +1,6 @@
---
name: precheck-pr
description: Self-check your branch before creating a PR — catch dead code, verify accuracy/perf claims, validate PR title format, and confirm merge readiness. Use when the user says "precheck", "self review", "pre-submit check", or "check my PR before I open it." Never posts to GitHub.
description: Self-check your branch before creating a PR — catch dead code, prevent new model-specific Python examples, verify accuracy/perf claims, validate PR title format, and confirm merge readiness. Use when the user says "precheck", "self review", "pre-submit check", or "check my PR before I open it." Never posts to GitHub.
---
# PR Pre-Check
@@ -65,6 +65,8 @@ Ask: "Quick mode or full mode?" Then walk the checklist for the detected PR type
**Also run the Code-Quality sweep on every PR**, regardless of type or mode: the five diff-scoped checks in [references/code-quality.md](references/code-quality.md) — kwargs fragility, broad-except swallow, `Any`/wrong type hints, hot-path `.clone()`/`deepcopy`, and event-loop blocking — plus the advisory conventions (log level, structured logging, synchronization, cleanup, dependencies, naming) in [checklists.md](references/checklists.md). These count only lines the PR *adds* — the pre-existing backlog across the repo is out of scope.
**Also run the Examples-Policy check on every PR** using [references/examples-policy.md](references/examples-policy.md). Inspect only Python paths introduced by the diff. Treat a new model-, checkpoint-, vendor-, or family-specific Python example as blocking; do not report pre-existing example debt.
### Step 5: Print Report
```
@@ -77,6 +79,7 @@ Pre-check report for <branch>
───────────────── ──────
PR title format ✓
Code quality ⚠ 1 broad except, 2 Any hints
Examples policy ✗ new model-specific Python example
PR desc integrity ✓
Registry/config ✓
Dead code ⚠ 2 warnings
@@ -47,6 +47,19 @@ Diff-scoped sweep for five fragility patterns that are pervasive in the existing
Roll the ⚠/✗ counts into the report as a single **Code quality** dimension row.
---
## All PRs: Examples Policy (run on every PR, quick & full)
Inspect Python files introduced under `examples/` using
[examples-policy.md](examples-policy.md). This is a diff-scoped ratchet: existing
model-specific examples may be edited or removed without triggering the check.
- [ ] **No new model-specific Python examples:** the diff adds, copies, or renames no Python file whose path or behavior is specific to one model, checkpoint, vendor, or model family. ✗ for any violation. Route inference through a shared task/protocol entrypoint, model request behavior through `vllm_omni/model_extras`, and commands through task docs or `recipes/`.
- [ ] **New generic example code is justified:** any new task-neutral Python file under `examples/` is an actual user-facing entrypoint rather than reusable production logic, a benchmark, an app, a setup tool, or a reproducer. ⚠ when the placement is questionable.
Roll the result into the report as a separate **Examples policy** dimension.
### Conventions (eyeball — no grep)
Judgment calls that don't grep cleanly — apply them while reading the diff, not as a mechanical check:
@@ -0,0 +1,68 @@
# Examples Python Policy
Prevent new model-specific Python examples without forcing cleanup of the
existing backlog. Run this check on every PR in quick and full modes.
## Scope
Set the merge base as in the main workflow, then list Python paths introduced
by the PR, including additions, copies, and renames:
```bash
BASE="$(git merge-base HEAD origin/main)"
git diff --name-status --diff-filter=ACR --find-renames --find-copies \
"${BASE}"...HEAD -- 'examples/**/*.py'
```
Inspect destination paths for copies and renames. Existing model-specific files
that are only modified or deleted are grandfathered and must not be reported.
## Blocking classification
Mark ✗ when a newly introduced Python file does any of the following:
- Names or scopes a path to one model, checkpoint, vendor, or model family.
- Hard-codes a model identifier, model-specific prompt, request shape, output
conversion, or launch configuration behind an otherwise generic filename.
- Duplicates an existing task/protocol runner to demonstrate one model.
- Adds a per-model helper below a generic task hub such as `text_to_speech/`.
Do not infer safety from a generic filename alone. For example, a file named
`text_to_audio.py` is still model-specific if it only implements one model's
contract.
## Acceptable additions
A new Python example may pass when it is a genuinely model-neutral,
user-facing task or protocol entrypoint. It must accept model selection through
configuration and keep model-specific behavior out of the script.
Use ⚠ when the file is model-neutral but reusable logic appears misplaced under
`examples/`. Route code by purpose:
| Code purpose | Preferred location |
|---|---|
| Model prompt, defaults, request/output adaptation | `vllm_omni/model_extras/` or production model modules |
| Runnable model command and validation evidence | Task documentation or `recipes/` |
| UI or interactive application | `apps/` |
| Benchmark or evaluator | `benchmarks/` |
| Regression reproducer | `tests/` |
| Download, conversion, or setup utility | `tools/` |
If a model cannot use an existing shared runner, report the missing generic
capability. Do not solve the gap by adding a model-specific Python example.
## Report format
Add one row to the pre-check report:
```text
Examples policy ✓ no new Python example paths
Examples policy ⚠ generic helper belongs in tools/
Examples policy ✗ examples/offline_inference/new_model/end2end.py is model-specific
Examples policy ✗ examples/offline_inference/x_to_y_model_name.py is model-specific
```
For a blocker, name the path, the model-specific behavior, and the appropriate
shared runner or destination. Do not propose deleting unrelated existing
examples as part of the PR.
+8
View File
@@ -97,6 +97,7 @@ If docs and live code disagree, verify the code/tests and report the drift.
| [test-quality-evaluation.md](references/checks/test-quality-evaluation.md) | Tests change, are absent for risky code, or may not exercise production behavior. |
| [tests-docs-checklist.md](references/checks/tests-docs-checklist.md) | Coverage, CI markers, examples, user docs, or PR evidence need review. |
| [verification.md](references/checks/verification.md) | Hardware, a server, or a runnable affected path is available for active verification. |
| [examples-policy.md](../precheck-pr/references/examples-policy.md) | The PR adds, copies, or renames Python under `examples/`; apply the canonical policy shared with `precheck-pr`. |
### Delivery and reviewer coordination
@@ -152,6 +153,13 @@ contract they protect or use only the applicable evidence checks.
Apply every category in [general-checks.md](references/process/general-checks.md) before
lower-priority comments.
If the diff census contains an added, copied, or renamed Python path under
`examples/`, read and apply the canonical
[examples policy](../precheck-pr/references/examples-policy.md). Treat a new
model-specific Python example as blocking. Do not flag model-specific example
debt that the PR only modifies or removes, and do not run the rest of the
author-oriented `precheck-pr` workflow.
For each changed value or behavior, trace:
```text
@@ -61,9 +61,7 @@ from vllm_omni.diffusion.utils.param_utils import apply_declared_extra_args
from vllm_omni.entrypoints.omni import Omni
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.model_extras import (
build_image_to_video_prompt as build_model_image_to_video_prompt,
)
from vllm_omni.model_extras import (
build_image_to_video_prompt,
get_extra_body_params,
get_model_class_name,
)
@@ -85,22 +83,6 @@ def parse_json_object(value: str, flag_name: str = "argument") -> dict[str, Any]
parse_profiler_config = functools.partial(parse_json_object, flag_name="--profiler-config")
def build_image_to_video_prompt(
prompt: str,
negative_prompt: str | None,
media_inputs: dict[str, Any],
) -> dict[str, Any]:
"""Build the canonical request envelope for the shared I2V example."""
result: dict[str, Any] = {
"prompt": prompt,
"modalities": ["video"],
"multi_modal_data": media_inputs,
}
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description=(
@@ -540,13 +522,10 @@ def main():
# Preserve the historical empty-prompt behavior for non-LTX examples.
negative_prompt = ""
prompt_dict = build_image_to_video_prompt(
model_class_name=model_class_name,
prompt=args.prompt,
negative_prompt=negative_prompt,
media_inputs=media_inputs,
)
prompt_dict = build_model_image_to_video_prompt(
model_class_name=model_class_name,
prompt=prompt_dict,
height=height,
width=width,
num_frames=num_frames,
@@ -57,9 +57,8 @@ python examples/offline_inference/text_to_image/text_to_image.py \
--output lingbot_t2i.png
```
The shared T2I entry point selects the `image` output modality. The LingBot
pipeline validates that T2I requests produce one frame and do not include an
input image.
The LingBot prompt builder selects the `image` output modality and requests one
frame. Passing an input image is not supported in LingBot T2I mode.
## Quick Start
@@ -21,9 +21,7 @@ from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.lora.request import LoRARequest
from vllm_omni.lora.utils import stable_lora_int_id
from vllm_omni.model_extras import (
build_text_to_image_prompt as build_model_text_to_image_prompt,
)
from vllm_omni.model_extras import (
build_text_to_image_prompt,
get_extra_body_params,
get_model_class_name,
should_init_extra_args_for_non_diffusion_stages,
@@ -58,17 +56,6 @@ def parse_json_object(value: str, flag_name: str = "argument") -> dict[str, Any]
parse_profiler_config = functools.partial(parse_json_object, flag_name="--profiler-config")
def build_text_to_image_prompt(prompt: str, negative_prompt: str | None) -> dict[str, Any]:
"""Build the canonical request envelope for the shared T2I example."""
result: dict[str, Any] = {
"prompt": prompt,
"modalities": ["image"],
}
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
def _normalize_images_for_save(images: list[Any]) -> list[Any]:
"""Convert NumPy diffusion outputs to PIL images before saving."""
normalized = []
@@ -518,12 +505,9 @@ def main():
generation_start = time.perf_counter()
prompt_dict = build_text_to_image_prompt(
model_class_name=model_class_name,
prompt=args.prompt,
negative_prompt=args.negative_prompt,
)
prompt_dict = build_model_text_to_image_prompt(
model_class_name=model_class_name,
prompt=prompt_dict,
height=args.height,
width=args.width,
)
@@ -1,9 +1,6 @@
# Image-To-Video
This example demonstrates online image-to-video generation with vLLM-Omni.
The startup script defaults to Wan2.2, but `MODEL` can select any supported
image-to-video model. The existing curl helper remains a concrete Wan2.2
request through the standard Videos API.
This example demonstrates how to deploy the Wan2.2 image-to-video model for online video generation using vLLM-Omni.
## Start Server
@@ -23,7 +20,9 @@ bash run_server.sh
The script allows overriding:
- `MODEL` (default: `Wan-AI/Wan2.2-I2V-A14B-Diffusers`)
- `PORT` (default: `8099`)
- `PORT` (default: `8091`)
- `BOUNDARY_RATIO` (default: `0.875`)
- `FLOW_SHIFT` (default: `12.0`)
- `CACHE_BACKEND` (default: `none`)
- `ENABLE_CACHE_DIT_SUMMARY` (default: `0`)
@@ -1,12 +1,12 @@
#!/bin/bash
# Shared image-to-video server start script
# Wan2.2 image-to-video server start script
MODEL="${MODEL:-Wan-AI/Wan2.2-I2V-A14B-Diffusers}"
PORT="${PORT:-8099}"
CACHE_BACKEND="${CACHE_BACKEND:-none}"
ENABLE_CACHE_DIT_SUMMARY="${ENABLE_CACHE_DIT_SUMMARY:-0}"
echo "Starting image-to-video server..."
echo "Starting Wan2.2 I2V server..."
echo "Model: $MODEL"
echo "Port: $PORT"
echo "Cache backend: $CACHE_BACKEND"
@@ -1,9 +1,6 @@
# Text-To-Image
This example demonstrates online text-to-image generation with vLLM-Omni. The
startup script and Python clients default to Qwen-Image, but `MODEL` can select
any supported text-to-image model. The existing curl helper remains a concrete
Qwen-Image request through the standard Images API.
This example demonstrates how to deploy Qwen-Image model for online image generation service using vLLM-Omni.
## Start Server
@@ -229,8 +226,8 @@ count, use `size` and `n` rather than `height`, `width`, or
| `height` | int | None | Image height in pixels |
| `width` | int | None | Image width in pixels |
| `size` | str | None | Image size (e.g., "1024x1024") |
| `num_inference_steps` | int | model | Number of denoising steps |
| `true_cfg_scale` | float | model | True CFG scale when supported |
| `num_inference_steps` | int | 50 | Number of denoising steps |
| `true_cfg_scale` | float | 4.0 | Qwen-Image CFG scale |
| `seed` | int | None | Random seed (reproducible) |
| `negative_prompt` | str | None | Negative prompt |
| `num_outputs_per_prompt` | int | 1 | Number of images to generate |
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""
Shared text-to-image Gradio demo for online serving.
Qwen-Image Gradio Demo for online serving.
Usage:
python gradio_demo.py [--server http://localhost:8091] [--port 7860]
@@ -77,9 +77,9 @@ def generate_image(
def create_demo(server_url: str):
"""Create Gradio demo interface."""
with gr.Blocks(title="Text-to-Image Demo") as demo:
gr.Markdown("# Text-to-Image Online Generation")
gr.Markdown("Generate images using the model configured on the server")
with gr.Blocks(title="Qwen-Image Demo") as demo:
gr.Markdown("# Qwen-Image Online Generation")
gr.Markdown("Generate images using Qwen-Image model")
with gr.Row():
with gr.Column(scale=1):
@@ -115,6 +115,7 @@ def create_demo(server_url: str):
label="Inference Steps",
minimum=10,
maximum=100,
# Default steps aligned with run_curl_text_to_image.sh to 100
value=100,
step=5,
)
@@ -172,7 +173,7 @@ def create_demo(server_url: str):
def main():
parser = argparse.ArgumentParser(description="Shared text-to-image Gradio demo")
parser = argparse.ArgumentParser(description="Qwen-Image Gradio Demo")
parser.add_argument("--server", default="http://localhost:8091", help="Server URL")
parser.add_argument("--port", type=int, default=7860, help="Gradio port")
parser.add_argument("--share", action="store_true", help="Create public link")
@@ -1,6 +1,6 @@
#!/usr/bin/env python3
"""
Shared OpenAI-compatible text-to-image generation client.
Qwen-Image OpenAI-compatible image generation client.
Usage:
python openai_chat_client.py --prompt "A beautiful landscape" --output output.png
@@ -39,7 +39,7 @@ def generate_image(
height: Image height in pixels
width: Image width in pixels
steps: Number of diffusion steps
true_cfg_scale: True CFG scale for models that support it
true_cfg_scale: Qwen-Image CFG scale
seed: Random seed
negative_prompt: Negative prompt
num_outputs_per_prompt: Number of images to generate
@@ -70,7 +70,7 @@ def generate_image(
payload["num_inference_steps"] = steps
if true_cfg_scale is not None:
payload["true_cfg_scale"] = true_cfg_scale
if negative_prompt is not None:
if negative_prompt:
payload["negative_prompt"] = negative_prompt
if seed is not None:
payload["seed"] = seed
@@ -115,15 +115,15 @@ def generate_image(
def main():
parser = argparse.ArgumentParser(description="Shared text-to-image client")
parser = argparse.ArgumentParser(description="Qwen-Image chat client")
parser.add_argument("--prompt", "-p", default="a cup of coffee on the table", help="Text prompt")
parser.add_argument("--output", "-o", default="text_to_image_output.png", help="Output file")
parser.add_argument("--output", "-o", default="qwen_image_output.png", help="Output file")
parser.add_argument("--server", "-s", default="http://localhost:8091", help="Server URL")
parser.add_argument("--height", type=int, help="Image height; defaults to the model setting")
parser.add_argument("--width", type=int, help="Image width; defaults to the model setting")
parser.add_argument("--steps", type=int, help="Inference steps; defaults to the model setting")
parser.add_argument("--cfg-scale", type=float, help="True CFG scale when supported")
parser.add_argument("--seed", type=int, help="Random seed")
parser.add_argument("--height", type=int, default=1024, help="Image height")
parser.add_argument("--width", type=int, default=1024, help="Image width")
parser.add_argument("--steps", type=int, default=50, help="Inference steps")
parser.add_argument("--cfg-scale", type=float, default=4.0, help="True CFG scale")
parser.add_argument("--seed", type=int, default=0, help="Random seed")
parser.add_argument("--negative", help="Negative prompt")
parser.add_argument("--lora-path", default=None, help="Server-local LoRA adapter folder (PEFT format)")
@@ -1,10 +1,10 @@
#!/bin/bash
# Shared text-to-image online serving startup script
# Qwen-Image online serving startup script
MODEL="${MODEL:-Qwen/Qwen-Image}"
PORT="${PORT:-8091}"
echo "Starting text-to-image server..."
echo "Starting Qwen-Image server..."
echo "Model: $MODEL"
echo "Port: $PORT"
-18
View File
@@ -2206,24 +2206,6 @@ class TestBaseConfigInheritance:
class TestPlatformOverrides:
"""Test platform-specific deploy config overrides."""
def test_qwen3_omni_musa_clears_parent_quantization_for_audio_stages(self):
pipeline = resolve_pipeline_config(
"qwen3_omni_moe",
Q3_OMNI_ALL_STAGES_HF_CONFIG,
)
assert isinstance(pipeline, PipelineConfig)
deploy_path = Path(get_deploy_config_path("qwen3_omni_moe.yaml"))
base_stages = merge_pipeline_deploy(pipeline, load_deploy_config(deploy_path))
assert "hf_overrides" not in base_stages[1].yaml_engine_args
assert "hf_overrides" not in base_stages[2].yaml_engine_args
musa = _apply_platform_overrides(load_deploy_config(deploy_path), platform="musa")
musa_stages = merge_pipeline_deploy(pipeline, musa)
expected = {"quantization_config": None}
assert musa_stages[1].yaml_engine_args["hf_overrides"] == expected
assert musa_stages[2].yaml_engine_args["hf_overrides"] == expected
def test_qwen3_tts_rocm_disables_code2wav_outer_cudagraph(self):
deploy_path = Path(get_deploy_config_path("qwen3_tts.yaml"))
@@ -1,47 +0,0 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
from __future__ import annotations
import pytest
from PIL import Image
from examples.offline_inference.image_to_video.image_to_video import build_image_to_video_prompt
from examples.offline_inference.text_to_image.text_to_image import build_text_to_image_prompt
pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
@pytest.mark.parametrize(
("negative_prompt", "expected_negative_prompt"),
[(None, None), ("", ""), ("blurry", "blurry")],
)
def test_text_to_image_builds_canonical_prompt(
negative_prompt: str | None,
expected_negative_prompt: str | None,
) -> None:
result = build_text_to_image_prompt("a red fox", negative_prompt)
assert result["prompt"] == "a red fox"
assert result["modalities"] == ["image"]
if expected_negative_prompt is None:
assert "negative_prompt" not in result
else:
assert result["negative_prompt"] == expected_negative_prompt
def test_image_to_video_builds_canonical_prompt() -> None:
image = Image.new("RGB", (32, 16), "red")
result = build_image_to_video_prompt(
prompt="the fox turns toward the camera",
negative_prompt="flicker",
media_inputs={"image": image},
)
assert result == {
"prompt": "the fox turns toward the camera",
"modalities": ["video"],
"multi_modal_data": {"image": image},
"negative_prompt": "flicker",
}
+107 -38
View File
@@ -3,8 +3,6 @@
from __future__ import annotations
from typing import Any
import pytest
from PIL import Image
@@ -137,6 +135,67 @@ def test_lingbot_extra_registry_declares_request_params() -> None:
assert should_init_extra_args_for_non_diffusion_stages("LingBotVideoPipeline") is False
@pytest.mark.core_model
@pytest.mark.cpu
def test_lingbot_text_to_image_prompt_builder_preserves_empty_negative_prompt() -> None:
assert build_text_to_image_prompt(
"LingBotVideoPipeline",
prompt="a red fox in fresh snow",
negative_prompt="",
height=192,
width=320,
) == {
"prompt": "a red fox in fresh snow",
"modalities": ["image"],
"negative_prompt": "",
}
@pytest.mark.core_model
@pytest.mark.cpu
def test_lingbot_image_to_video_prompt_builder() -> None:
image = Image.new("RGB", (320, 192), "red")
result = build_image_to_video_prompt(
"LingBotVideoPipeline",
prompt="the fox looks toward the camera",
negative_prompt=None,
media_inputs={"image": image},
height=192,
width=320,
num_frames=9,
)
assert result == {
"prompt": "the fox looks toward the camera",
"modalities": ["video"],
"multi_modal_data": {"image": image},
"num_frames": 9,
}
@pytest.mark.core_model
@pytest.mark.cpu
@pytest.mark.parametrize(
"media_inputs",
[
{},
{"image": "input.png"},
{"image": [Image.new("RGB", (16, 16))]},
{"image": Image.new("RGB", (16, 16)), "video": b"video"},
],
ids=["missing", "path", "multiple", "extra-modality"],
)
def test_lingbot_image_to_video_prompt_builder_rejects_invalid_media(
media_inputs: dict[str, object],
) -> None:
with pytest.raises(ValueError, match="exactly one PIL image"):
build_image_to_video_prompt(
"LingBotVideoPipeline",
prompt="move",
negative_prompt=None,
media_inputs=media_inputs,
)
@pytest.mark.core_model
@pytest.mark.cpu
@pytest.mark.parametrize("pipeline_name", ["Cosmos3OmniDiffusersPipeline", "Cosmos3OmniPipeline"])
@@ -283,11 +342,8 @@ def test_ming_flash_omni_declared_extra_args_route_into_sampling_params() -> Non
def test_ming_flash_omni_text_to_image_prompt_builder() -> None:
assert build_text_to_image_prompt(
"MingImagePipeline",
{
"prompt": "Please draw a cute cat.",
"modalities": ["image"],
"negative_prompt": "ugly, blurry",
},
prompt="Please draw a cute cat.",
negative_prompt="ugly, blurry",
height=512,
width=768,
) == {
@@ -304,7 +360,8 @@ def test_ming_flash_omni_text_to_image_prompt_builder() -> None:
# pipeline's 1024x1024 default still applies.
assert build_text_to_image_prompt(
"MingImagePipeline",
{"prompt": "Draw a poster.", "modalities": ["image"]},
prompt="Draw a poster.",
negative_prompt=None,
) == {
"prompt": "Draw a poster.",
"modalities": ["image"],
@@ -334,6 +391,28 @@ def test_ming_flash_omni_image_to_image_prompt_builder() -> None:
assert "negative_prompt" not in result
@pytest.mark.core_model
@pytest.mark.cpu
@pytest.mark.parametrize("pipeline_name", ["Cosmos3OmniDiffusersPipeline", "Cosmos3OmniPipeline"])
def test_cosmos3_text_to_image_prompt_builder_selects_image_modality(pipeline_name: str) -> None:
assert build_text_to_image_prompt(
pipeline_name,
prompt="a red sports car at golden hour",
negative_prompt="blurry, distorted",
height=1024,
width=1024,
) == {
"prompt": "a red sports car at golden hour",
"modalities": ["image"],
"negative_prompt": "blurry, distorted",
}
assert build_text_to_image_prompt(
pipeline_name,
prompt="a red sports car",
negative_prompt=None,
) == {"prompt": "a red sports car", "modalities": ["image"]}
@pytest.mark.core_model
@pytest.mark.cpu
def test_audiox_extra_registry_declares_request_and_response_params() -> None:
@@ -426,11 +505,8 @@ def test_unknown_pipeline_has_empty_extra_registry() -> None:
def test_bagel_text_to_image_prompt_builder() -> None:
assert build_text_to_image_prompt(
"BagelPipeline",
{
"prompt": "a cat",
"modalities": ["image"],
"negative_prompt": "blurry",
},
prompt="a cat",
negative_prompt="blurry",
height=512,
width=768,
) == {
@@ -467,17 +543,14 @@ def test_bagel_image_to_image_prompt_builder() -> None:
@pytest.mark.core_model
@pytest.mark.cpu
def test_unknown_pipeline_preserves_canonical_text_to_image_prompt() -> None:
canonical_prompt = {"prompt": "a cat", "modalities": ["image"]}
assert (
build_text_to_image_prompt(
"UnknownPipeline",
canonical_prompt,
height=512,
width=512,
)
is canonical_prompt
)
def test_unknown_pipeline_uses_default_text_to_image_prompt() -> None:
assert build_text_to_image_prompt(
"UnknownPipeline",
prompt="a cat",
negative_prompt=None,
height=512,
width=512,
) == {"prompt": "a cat"}
@pytest.mark.core_model
@@ -496,14 +569,12 @@ def test_unknown_pipeline_uses_default_image_to_image_prompt() -> None:
}
def _build_vace_prompt(media_inputs: dict[str, Any], *, num_frames: int = 5) -> dict:
def _build_vace_prompt(media_inputs: dict[str, object], *, num_frames: int = 5) -> dict:
return build_image_to_video_prompt(
"WanVACEPipeline",
{
"prompt": "a bird flying",
"modalities": ["video"],
"multi_modal_data": media_inputs,
},
prompt="a bird flying",
negative_prompt=None,
media_inputs=media_inputs,
height=16,
width=320,
num_frames=num_frames,
@@ -526,7 +597,7 @@ def _build_vace_prompt(media_inputs: dict[str, Any], *, num_frames: int = 5) ->
],
ids=["i2v", "v2lf", "flf2v", "inpaint", "r2v"],
)
def test_vace_image_to_video_prompt_builder(media_inputs: dict[str, Any]) -> None:
def test_vace_image_to_video_prompt_builder(media_inputs: dict[str, object]) -> None:
result = _build_vace_prompt(media_inputs)
mmd = result["multi_modal_data"]
if "reference_images" in mmd:
@@ -545,7 +616,7 @@ def test_vace_image_to_video_prompt_builder(media_inputs: dict[str, Any]) -> Non
({"control_image": Image.new("RGB", (320, 16))}, "Unsupported VACE media input"),
],
)
def test_vace_rejects_invalid_media_combinations(media_inputs: dict[str, Any], message: str) -> None:
def test_vace_rejects_invalid_media_combinations(media_inputs: dict[str, object], message: str) -> None:
with pytest.raises(ValueError, match=message):
_build_vace_prompt(media_inputs)
@@ -593,7 +664,8 @@ def test_mammothmoda2_extra_registry_declares_request_and_response_params() -> N
wrapper_prompt = build_text_to_image_prompt(
"MammothModa2ForConditionalGeneration",
{"prompt": "a cat", "modalities": ["image"]},
prompt="a cat",
negative_prompt=None,
height=256,
width=256,
)
@@ -607,11 +679,8 @@ def test_mammothmoda2_text_to_image_prompt_builder() -> None:
# prompt is ignored (MammothModa2 t2i uses CFG, not an explicit negative path).
assert build_text_to_image_prompt(
"MammothModa2DiTPipeline",
{
"prompt": "a cat",
"modalities": ["image"],
"negative_prompt": "blurry",
},
prompt="a cat",
negative_prompt="blurry",
height=512,
width=768,
) == {
+1 -76
View File
@@ -3,88 +3,13 @@ from types import SimpleNamespace
import pytest
import torch
from vllm.v1.cudagraph_dispatcher import CUDAGraphMode
from vllm_omni.worker.gpu_model_runner import (
OmniGPUModelRunner,
_filter_mrope_kwargs_for_model,
)
from vllm_omni.worker.gpu_model_runner import OmniGPUModelRunner, _filter_mrope_kwargs_for_model
from vllm_omni.worker.omni_connector_model_runner_mixin import OmniConnectorModelRunnerMixin
pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
def _runner_for_talker_graph_init(
*,
talker_mtp_graph_safe: bool | None,
has_separate_talker: bool = True,
model_stage: str = "talker",
) -> OmniGPUModelRunner:
runner = object.__new__(OmniGPUModelRunner)
runner.model = SimpleNamespace(
talker=object() if has_separate_talker else None,
talker_mtp=object(),
model_stage=model_stage,
)
if talker_mtp_graph_safe is not None:
runner.model.talker_mtp_graph_safe = talker_mtp_graph_safe
runner.model_config = SimpleNamespace(hf_text_config=SimpleNamespace(hidden_size=4))
runner.compilation_config = SimpleNamespace(
cudagraph_mode=CUDAGraphMode.FULL,
max_cudagraph_capture_size=1,
)
runner.vllm_config = object()
runner.max_num_reqs = 1
runner.dtype = torch.float32
runner._make_buffer = lambda *args, **kwargs: SimpleNamespace(args=args, kwargs=kwargs)
return runner
def test_talker_mtp_skips_graph_when_model_declares_unsafe(monkeypatch):
runner = _runner_for_talker_graph_init(talker_mtp_graph_safe=False)
talker_mtp = runner.model.talker_mtp
monkeypatch.setattr(
"vllm_omni.worker.gpu_model_runner.current_omni_platform.get_graph_wrapper_cls",
lambda: pytest.fail("graph wrapper must not be selected"),
)
OmniGPUModelRunner._init_talker_mtp(runner)
assert runner.talker_mtp is talker_mtp
@pytest.mark.parametrize("model_stage", ["thinker", "code2wav"])
def test_non_talker_stage_does_not_use_talker_mtp_graph(monkeypatch, model_stage: str):
runner = _runner_for_talker_graph_init(
talker_mtp_graph_safe=None,
has_separate_talker=False,
model_stage=model_stage,
)
talker_mtp = runner.model.talker_mtp
monkeypatch.setattr(
"vllm_omni.worker.gpu_model_runner.current_omni_platform.get_graph_wrapper_cls",
lambda: pytest.fail("graph wrapper must not be selected"),
)
OmniGPUModelRunner._init_talker_mtp(runner)
assert runner.talker_mtp is talker_mtp
@pytest.mark.parametrize("talker_mtp_graph_safe", [None, True])
def test_talker_mtp_uses_graph_for_legacy_or_explicit_safe_model(monkeypatch, talker_mtp_graph_safe):
runner = _runner_for_talker_graph_init(talker_mtp_graph_safe=talker_mtp_graph_safe)
wrapped = object()
monkeypatch.setattr(
"vllm_omni.worker.gpu_model_runner.current_omni_platform.get_graph_wrapper_cls",
lambda: lambda *args, **kwargs: wrapped,
)
OmniGPUModelRunner._init_talker_mtp(runner)
assert runner.talker_mtp is wrapped
class DummyBuffer:
"""A minimal buffer wrapper that exposes the `.gpu` attribute."""
-12
View File
@@ -69,18 +69,6 @@ stages:
repetition_penalty: 1.1
platforms:
# The supported Qwen3-Omni ModelOpt checkpoint quantizes the thinker only. Clear the
# checkpoint-level metadata for the BF16 audio stages on the verified MUSA
# profile before vLLM performs its normal quantization auto-detection.
musa:
stages:
- stage_id: 1
hf_overrides:
quantization_config: null
- stage_id: 2
hf_overrides:
quantization_config: null
npu:
stages:
- stage_id: 0
@@ -152,9 +152,6 @@ class Qwen3OmniMoeForConditionalGeneration(
dtype=torch.long,
)
elif self.model_stage == "talker":
# The outer wrapper exposes talker_mtp for every stage, but only
# the talker stage owns the module that the method invokes.
self.talker_mtp_graph_safe = current_omni_platform.supports_talker_mtp_graph_capture()
multimodal_config.skip_mm_profiling = True
self.has_preprocess = True
self.has_postprocess = True
+26
View File
@@ -3,6 +3,32 @@
from __future__ import annotations
from typing import Any
def build_text_to_image_prompt(
prompt: str,
negative_prompt: str | None,
height: int | None = None,
width: int | None = None,
) -> dict[str, Any]:
"""Build a Cosmos3 T2I prompt by selecting image output modality.
Cosmos3 uses the same pipeline class for T2I and video modes. The OpenAI
image endpoint selects T2I by adding ``modalities=["image"]``. ``height``
and ``width`` are accepted for registry compatibility; sizing travels
through sampling params instead of the prompt payload.
"""
del height, width
text_prompt: dict[str, Any] = {
"prompt": prompt,
"modalities": ["image"],
}
if negative_prompt is not None:
text_prompt["negative_prompt"] = negative_prompt
return text_prompt
COSMOS3_EXTRA_BODY_PARAMS = frozenset(
{
"flow_shift",
+45
View File
@@ -3,6 +3,11 @@
from __future__ import annotations
from collections.abc import Mapping
from typing import Any
from PIL import Image
LINGBOT_VIDEO_EXTRA_BODY_PARAMS = frozenset(
{
"batch_cfg",
@@ -19,3 +24,43 @@ LINGBOT_VIDEO_EXTRA_BODY_PARAMS = frozenset(
"t_thresh",
}
)
def build_text_to_image_prompt(
prompt: str,
negative_prompt: str | None,
height: int | None = None,
width: int | None = None,
) -> dict[str, Any]:
del height, width
result: dict[str, Any] = {
"prompt": prompt,
"modalities": ["image"],
}
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
def build_image_to_video_prompt(
prompt: str,
negative_prompt: str | None,
media_inputs: Mapping[str, Any],
height: int | None = None,
width: int | None = None,
num_frames: int | None = None,
) -> dict[str, Any]:
del height, width
if set(media_inputs) != {"image"} or not isinstance(media_inputs["image"], Image.Image):
raise ValueError("LingBot text-image-to-video requires exactly one PIL image input.")
result: dict[str, Any] = {
"prompt": prompt,
"modalities": ["video"],
"multi_modal_data": {"image": media_inputs["image"]},
}
if num_frames is not None:
result["num_frames"] = num_frames
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
+55 -29
View File
@@ -28,12 +28,21 @@ from vllm_omni.model_extras.cosmos3 import (
COSMOS3_EXTRA_BODY_PARAMS,
COSMOS3_EXTRA_OUTPUT_PARAMS,
)
from vllm_omni.model_extras.cosmos3 import (
build_text_to_image_prompt as build_cosmos3_text_to_image_prompt,
)
from vllm_omni.model_extras.helios import (
HELIOS_EXTRA_BODY_PARAMS,
HELIOS_EXTRA_OUTPUT_PARAMS,
)
from vllm_omni.model_extras.hunyuan_image3 import build_x_to_text_prompt as build_hunyuan_x_to_text_prompt
from vllm_omni.model_extras.lingbot_video import LINGBOT_VIDEO_EXTRA_BODY_PARAMS
from vllm_omni.model_extras.lingbot_video import (
build_image_to_video_prompt as build_lingbot_image_to_video_prompt,
)
from vllm_omni.model_extras.lingbot_video import (
build_text_to_image_prompt as build_lingbot_text_to_image_prompt,
)
from vllm_omni.model_extras.ltx2 import LTX_EXTRA_BODY_PARAMS, LTX_EXTRA_OUTPUT_PARAMS
from vllm_omni.model_extras.magi_human import (
MAGI_HUMAN_EXTRA_BODY_PARAMS,
@@ -138,6 +147,18 @@ def build_x_to_text_prompt(
return builder(model, prompt, has_image)
def default_text_to_image_prompt(
prompt: str,
negative_prompt: str | None,
height: int | None = None,
width: int | None = None,
) -> dict[str, Any]:
result: dict[str, Any] = {"prompt": prompt}
if negative_prompt is not None:
result["negative_prompt"] = negative_prompt
return result
def default_image_to_image_prompt(
prompt: str,
negative_prompt: str | None,
@@ -154,6 +175,20 @@ def default_image_to_image_prompt(
return result
def default_image_to_video_prompt(
prompt: str,
negative_prompt: str | None,
media_inputs: Mapping[str, Any],
height: int | None = None,
width: int | None = None,
num_frames: int | None = None,
) -> dict[str, Any]:
del height, width, num_frames
if set(media_inputs) != {"image"} or not isinstance(media_inputs["image"], Image.Image):
raise ValueError("This model only supports a single --image input in the shared image-to-video example.")
return default_image_to_image_prompt(prompt, negative_prompt, media_inputs["image"])
_EXTRA_SPECS: dict[str, dict[str, Any]] = {
"AudioXPipeline": {
"extra_body_params": AUDIOX_EXTRA_BODY_PARAMS,
@@ -173,11 +208,12 @@ _EXTRA_SPECS: dict[str, dict[str, Any]] = {
"Cosmos3OmniDiffusersPipeline": {
"extra_body_params": COSMOS3_EXTRA_BODY_PARAMS,
"extra_output_params": COSMOS3_EXTRA_OUTPUT_PARAMS,
# The shared T2I example already supplies modalities=["image"].
"text_to_image_prompt_builder": build_cosmos3_text_to_image_prompt,
},
"Cosmos3OmniPipeline": {
"extra_body_params": COSMOS3_EXTRA_BODY_PARAMS,
"extra_output_params": COSMOS3_EXTRA_OUTPUT_PARAMS,
"text_to_image_prompt_builder": build_cosmos3_text_to_image_prompt,
},
"MagiHumanPipeline": {
"extra_body_params": MAGI_HUMAN_EXTRA_BODY_PARAMS,
@@ -193,8 +229,8 @@ _EXTRA_SPECS: dict[str, dict[str, Any]] = {
},
"LingBotVideoPipeline": {
"extra_body_params": LINGBOT_VIDEO_EXTRA_BODY_PARAMS,
# Shared T2I/I2V envelopes select the output modality. LingBot's
# pipeline owns model-specific validation and normalization.
"text_to_image_prompt_builder": build_lingbot_text_to_image_prompt,
"image_to_video_prompt_builder": build_lingbot_image_to_video_prompt,
},
**{
model_class_name: {
@@ -227,7 +263,7 @@ _EXTRA_SPECS: dict[str, dict[str, Any]] = {
}
# Multi-stage discovery reports the top-level wrapper rather than its DiT
# submodule, so both names must resolve to the same request builders.
# submodule, so both names must resolve to the same request adapters.
_EXTRA_SPECS["MammothModa2ForConditionalGeneration"] = _EXTRA_SPECS["MammothModa2DiTPipeline"]
_EXTRA_SPECS["Mammothmoda2Model"] = _EXTRA_SPECS["MammothModa2DiTPipeline"]
@@ -269,21 +305,18 @@ def should_init_extra_args_for_non_diffusion_stages(model_class_name: str | None
def build_text_to_image_prompt(
model_class_name: str | None,
prompt: dict[str, Any],
prompt: str,
negative_prompt: str | None,
height: int | None = None,
width: int | None = None,
) -> dict[str, Any]:
"""Build a model-specific T2I prompt from an example-owned envelope."""
spec = _get_spec(model_class_name)
builder: TextToImagePromptBuilder | None = spec.get("text_to_image_prompt_builder") if spec else None
if builder is None:
return prompt
return builder(
prompt=str(prompt["prompt"]),
negative_prompt=prompt.get("negative_prompt"),
height=height,
width=width,
builder: TextToImagePromptBuilder = (
spec.get("text_to_image_prompt_builder", default_text_to_image_prompt)
if spec is not None
else default_text_to_image_prompt
)
return builder(prompt, negative_prompt, height, width)
def build_image_to_image_prompt(
@@ -305,24 +338,17 @@ def build_image_to_image_prompt(
def build_image_to_video_prompt(
model_class_name: str | None,
prompt: dict[str, Any],
prompt: str,
negative_prompt: str | None,
media_inputs: Mapping[str, Any],
height: int | None = None,
width: int | None = None,
num_frames: int | None = None,
) -> dict[str, Any]:
"""Build a model-specific I2V prompt from an example-owned envelope."""
spec = _get_spec(model_class_name)
builder: ImageToVideoPromptBuilder | None = spec.get("image_to_video_prompt_builder") if spec else None
if builder is None:
return prompt
media_inputs = prompt.get("multi_modal_data") or {}
if not isinstance(media_inputs, Mapping):
raise TypeError("Canonical I2V prompt multi_modal_data must be a mapping.")
return builder(
prompt=str(prompt["prompt"]),
negative_prompt=prompt.get("negative_prompt"),
media_inputs=media_inputs,
height=height,
width=width,
num_frames=num_frames,
builder: ImageToVideoPromptBuilder = (
spec.get("image_to_video_prompt_builder", default_image_to_video_prompt)
if spec is not None
else default_image_to_video_prompt
)
return builder(prompt, negative_prompt, media_inputs, height, width, num_frames)
-5
View File
@@ -155,11 +155,6 @@ class OmniPlatform(Platform):
"""Check if the platform supports torch.compile with inductor backend."""
raise NotImplementedError
@classmethod
def supports_talker_mtp_graph_capture(cls) -> bool:
"""Whether a model may capture its dedicated talker MTP graph."""
return True
@classmethod
def has_flash_attn_package(cls) -> bool:
"""Check if a Flash Attention package is available and usable on this platform."""
-5
View File
@@ -115,11 +115,6 @@ class MUSAOmniPlatform(OmniPlatform, MUSAPlatformBase):
"""MUSA supports torch.compile with inductor backend."""
return True
@classmethod
def supports_talker_mtp_graph_capture(cls) -> bool:
"""MUSA keeps Qwen3 talker MTP outside its dedicated FULL graph."""
return False
@classmethod
def supports_float64(cls) -> bool:
"""MUSA does not support float64 yet."""
+2 -6
View File
@@ -206,12 +206,8 @@ class OmniGPUModelRunner(GPUModelRunner):
cudagraph_mode = self.compilation_config.cudagraph_mode
assert cudagraph_mode is not None
has_separate_talker = getattr(self.model, "talker", None) is not None
talker_mtp_graph_safe = getattr(self.model, "talker_mtp_graph_safe", None)
# Preserve the legacy separate-talker default only when the model has
# not declared graph safety. An explicit False must disable wrapping.
if talker_mtp_graph_safe is None:
talker_mtp_graph_safe = has_separate_talker
if cudagraph_mode.has_full_cudagraphs() and talker_mtp_graph_safe:
talker_mtp_graph_safe = getattr(self.model, "talker_mtp_graph_safe", False)
if cudagraph_mode.has_full_cudagraphs() and (has_separate_talker or talker_mtp_graph_safe):
graph_wrapper_cls = current_omni_platform.get_graph_wrapper_cls()
self.talker_mtp = graph_wrapper_cls(talker_mtp, self.vllm_config, runtime_mode=CUDAGraphMode.FULL)
# TTS exposes mtp_hidden_size; Omni uses hf_text_config.hidden_size.