anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Test] Cleanup for helpers (#6523)

Browse Source 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
This commit is contained in:
wangyuandamy-why-3459 authored and GitHub committed 2026-08-25 09:10:17 +08:00
1 parent 8360346a66
commit 4af0ddc1c2
140 files changed
+4121 -4194
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

.buildkite/cuda/test-ready.yml
+2 -26
View File
@@ -125,32 +125,8 @@ steps:
- vllm_omni/outputs/
commands:
- timeout 20m pytest -s -v tests/e2e/features/custom_pipeline/ -m "core_model"
agents:
queue: "gpu_1_queue"
plugins:
- docker#v5.2.0:
image: public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:$BUILDKITE_COMMIT
always-pull: true
propagate-environment: true
# vLLM 0.27 added a pre-flight capacity check in
# distributed/device_communicators/shm_broadcast.py. Without this
# the container gets Docker's 64 MiB default and every test in
# this step dies at engine start with
# RuntimeError: Insufficient space in /dev/shm:
# 240 MiB required, 64 MiB available
# (9 failed in 8.7s on build #2949). vLLM 0.26 had no such check
# and allocated regardless, so this step passed on main while
# being under-provisioned the whole time — 0.27 turned a silent
# risk into an explicit error.
#
# This is the only inline docker plugin block in .buildkite/cuda/;
# every other step inherits common/ci_mirror_hardwares.yml, which
# already sets 8gb. Matching that rather than inventing a value.
shm-size: "8gb"
environment:
- "HF_HOME=/fsx/hf_cache"
volumes:
- "/fsx/hf_cache:/fsx/hf_cache"
mirror_hardwares: l4_1
- label: "RLHF · VeRL-Omni E2E Test"
source_file_dependencies:
.claude/skills/add-tts-model/SKILL.md
+5 -5
View File
@@ -345,8 +345,8 @@ Heavier scenarios in the same file use **`advanced_model` only** (streaming, ext
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_voice_clone_en_non_streaming_001(omni_server, openai_client) -> None:
openai_client.send_audio_speech_request({...})
def test_voice_clone_en_non_streaming_001(omni_server, online_client) -> None:
online_client.send_audio_speech_request({...})
```
**L4 consolidation:** Prefer parametrized `OmniServerParams` rows (`default`, `async_chunk`, feature flags) in one expansion module rather than many merge-only files ([#1832](https://github.com/vllm-project/vllm-omni/issues/1832)).
@@ -363,8 +363,8 @@ def test_voice_clone_en_non_streaming_001(omni_server, openai_client) -> None:
| Path | Fixture | Call |
|------|---------|------|
| Online `/v1/*` | `openai_client` | `openai_client.send_*_request(request_config)` |
| Offline inference | `omni_runner_handler` | `omni_runner_handler.send_*_request(request_config)` |
| Online `/v1/*` | `online_client` | `online_client.send_*_request(request_config)` |
| Offline inference | `offline_client` | `offline_client.send_*_request(request_config)` |
1. **Grep `runtime.py` first** — reuse `send_omni_request`, `send_diffusion_request`, `send_audio_speech_request` (online + offline Qwen-style TTS), `send_single_stage_tts_request` (Coqui XTTS / MOSS-TTS-Nano offline), etc.
2. **No matching helper** → add `send_<feature>_request` (or `send_<route>_http_request` for negative/dfx) in **`runtime.py`** with general `assert_*` bundled inside, **then** call it from the test.
@@ -396,7 +396,7 @@ See [vllm-omni-test skill](../vllm-omni-test/SKILL.md) § **Runtime send helpers
`/health` return non-200 until you're actually ready.
- **Mark tests per the CI Levels table** — baseline smoke: `core_model` + `advanced_model`; heavier cases: `advanced_model` only; L4 expansion: `full_model`
- **No per-model helper modules** — do not create `tests/helpers/{slug}.py`; keep constants and `request_config` payloads in the test file
- **Online and offline e2e go through `runtime.py`** — `openai_client.send_audio_speech_request` (online); `omni_runner_handler.send_audio_speech_request` (Qwen-style offline) or `send_single_stage_tts_request` (single-stage offline). Add a new `send_*_request` in `runtime.py` when none fits; do not embed `omni.generate` or HTTP in tests
- **Online and offline e2e go through `runtime.py`** — `online_client.send_audio_speech_request` (online); `offline_client.send_audio_speech_request` (Qwen-style offline) or `send_single_stage_tts_request` (single-stage offline). Add a new `send_*_request` in `runtime.py` when none fits; do not embed `omni.generate` or HTTP in tests
## Phase 4: Async Chunk (Streaming)
.claude/skills/vllm-omni-test/SKILL.md
+47 -47
View File
@@ -94,8 +94,8 @@ Existing references: `tests/e2e/offline_inference/test_qwen2_5_omni.py` (L2-styl
| Scenario | Typical location | Fixtures / runner pattern | Baseline markers & level |
|----------|------------------|---------------------------|---------------------------|
| **Offline inference e2e** | `tests/e2e/offline_inference/` | **Module (default):** `omni_runner` + `omni_runner_handler`. **Function (isolation only):** `omni_runner_function` + `omni_runner_handler_function`. Diffusion/TTS may use `Omni(...).generate` directly | L2: `core_model` + **one of** `omni` / `tts` / `diffusion`; `@hardware_test(...)` when GPU/NPU is required |
| **Online serving e2e** | `tests/e2e/online_serving/` | **Module (default):** `omni_server` + `openai_client`. **Function (isolation only):** `omni_server_function` + `openai_client_function`. Clients: `send_omni_request` (omni), `send_audio_speech_request` (tts), `send_diffusion_request` / `send_video_diffusion_request` / `send_images_generations_request` (diffusion) | Baseline smoke: **`core_model` + `advanced_model`**; heavier paths: `advanced_model` only; L4 expansion: `full_model` |
| **Offline inference e2e** | `tests/e2e/offline_inference/` | **Module (default):** `omni_runner` + `offline_client`. **Function (isolation only):** `omni_runner_function` + `offline_client_function`. Diffusion/TTS may use `Omni(...).generate` directly | L2: `core_model` + **one of** `omni` / `tts` / `diffusion`; `@hardware_test(...)` when GPU/NPU is required |
| **Online serving e2e** | `tests/e2e/online_serving/` | **Module (default):** `omni_server` + `online_client`. **Function (isolation only):** `omni_server_function` + `online_client_function`. Clients: `send_omni_request` (omni), `send_audio_speech_request` (tts), `send_diffusion_request` / `send_video_diffusion_request` / `send_images_generations_request` (diffusion) | Baseline smoke: **`core_model` + `advanced_model`**; heavier paths: `advanced_model` only; L4 expansion: `full_model` |
| **Documentation / runnable examples** | `tests/examples/offline_inference/`, `tests/examples/online_serving/` | **Offline docs (preferred):** extract Python/Bash blocks from the doc README (e.g. `ReadmeSnippet.extract_readme_snippets`), `pytest.mark.parametrize` each snippet, run via `example_runner.run` with a stable `output_subfolder`. **Online docs:** copy client/request scripts into dedicated tests and keep them in sync with the doc page. | Usually **L4**: `advanced_model`, often `example` plus hardware marks matching the nightly docs-example job (see `.buildkite/cuda/test-nightly.yml`). Full conventions: [docs/contributing/ci/test_examples/l4_doc_example_tests.inc.md](../../../docs/contributing/ci/test_examples/l4_doc_example_tests.inc.md) (introduced in [PR #1910](https://github.com/vllm-project/vllm-omni/pull/1910): naming, output directory layout, skip rules, avoid trimming `num_inference_steps` without a strong CI reason). |
| **Performance / benchmark** | `tests/dfx/perf/tests/*.json` + `run_*_benchmark.py` | JSON or script-driven server + load config; assert explicit metrics / baselines | L4 Perf: JSON `mark` with `full_model` + `omni`/`tts`/`diffusion`; wire `test-nightly.yml` Perf steps |
| **Invalid parameter / negative HTTP validation** | `tests/dfx/reliability/invalid_param_test/` | Live `omni_server` + low-level `send_*_http_request` with `err_code` / `err_message` | `pytest.mark.slow` + `omni` / `tts` / `diffusion` + `@hardware_marks` (`H100` or `L4`); CI in **`test-weekly.yml`** (not ready/merge/nightly) |
@@ -110,9 +110,9 @@ After choosing offline/online/docs/perf, classify the **product under test** and
|-----------|------------------------------|----------------------------|----------------------------------------|
| **What it is** | Multimodal LLM pipeline (thinker/talker/stages; text + vision + audio I/O) | Speech synthesis / voice models | Generative diffusion (noise → image, audio, text, or video) |
| **Examples** | Qwen2.5-Omni, Qwen3-Omni | Qwen3-TTS, VoxCPM2, Higgs-Audio, Step-Audio2 | Qwen-Image, BAGEL, Wan2.2, HunyuanVideo |
| **Offline runner** | `omni_runner` + `omni_runner_handler`, `generate_multimodal` | `Omni(...).generate` with deploy YAML + TTS params | `Omni(...).generate` + `OmniDiffusionSamplingParams` |
| **Online client** | `openai_client.send_omni_request` (chat completions, modalities) | `openai_client.send_audio_speech_request` (`/v1/audio/speech`) | `send_diffusion_request` (chat/T2I), `send_video_diffusion_request` (`/v1/videos`, X2V), or `send_images_generations_http_request` / `send_images_edits_http_request` (DALL-E routes) |
| **Typical assertions** | Stage outputs, text/audio keywords via `OmniRunnerHandler` / response handler | WAV bytes, stream chunks, speech endpoint contract | Image/video dimensions, `final_output_type`, `assert_diffusion_response` |
| **Offline runner** | `omni_runner` + `offline_client`, `generate_multimodal` | `Omni(...).generate` with deploy YAML + TTS params | `Omni(...).generate` + `OmniDiffusionSamplingParams` |
| **Online client** | `online_client.send_omni_request` (chat completions, modalities) | `online_client.send_audio_speech_request` (`/v1/audio/speech`) | `send_diffusion_request` (chat/T2I), `send_video_diffusion_request` (`/v1/videos`, X2V), or `send_images_generations_http_request` / `send_images_edits_http_request` (DALL-E routes) |
| **Typical assertions** | Stage outputs, text/audio keywords via `OfflineOmniClient` / response handler | WAV bytes, stream chunks, speech endpoint contract | Image/video dimensions, `final_output_type`, `assert_diffusion_response` |
| **Deploy YAML** | Per-model deploy configs (`ci/qwen3_omni_moe.yaml`, …) | `qwen3_tts.yaml`, `voxcpm2.yaml`, … | Often default serve; parallel/offload YAML for heavy DiT |
| **Nightly group** (`test-nightly.yml`) | **Omni Model Test** — `-m "full_model and omni"` | **TTS Model Test** — `-m "full_model and tts"` | **Diffusion X2I(&A&T)** *or* **Diffusion X2V** (see Step 3 table; same `diffusion` marker, different YAML group / file shard) |
| **L4 pressure** | Expansion per modality/model as needed | Expansion + accuracy/perf in TTS group | X2I: merge feature combos per [#1832](https://github.com/vllm-project/vllm-omni/issues/1832); X2V: separate nightly group |
@@ -175,9 +175,9 @@ See **L1 unit test constraints (mocking)** below for the full do/don't list.
@pytest.mark.omni
@hardware_test(...)
@pytest.mark.parametrize("omni_runner", test_params, indirect=True)
def test_<scenario>(omni_runner, omni_runner_handler) -> None:
def test_<scenario>(omni_runner, offline_client) -> None:
request_config = {"prompts": ..., "modalities": [...]} # optional: images, videos, audios
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
```
*Offline generative e2e — **Diffusion** (representative):*
@@ -187,8 +187,8 @@ def test_<scenario>(omni_runner, omni_runner_handler) -> None:
@pytest.mark.diffusion
@hardware_test(...)
@pytest.mark.parametrize("omni_runner", test_params, indirect=True)
def test_text_to_image_001(omni_runner_handler) -> None:
omni_runner_handler.send_diffusion_request({"prompt": "...", "extra_body": {"num_inference_steps": 4, ...}})
def test_text_to_image_001(offline_client) -> None:
offline_client.send_diffusion_request({"prompt": "...", "extra_body": {"num_inference_steps": 4, ...}})
```
*Offline TTS e2e — **Qwen3-TTS** (two-stage; representative):*
@@ -198,8 +198,8 @@ def test_text_to_image_001(omni_runner_handler) -> None:
@pytest.mark.tts
@hardware_test(...)
@pytest.mark.parametrize("omni_runner", tts_server_params, indirect=True)
def test_text_to_audio_001(omni_runner, omni_runner_handler) -> None:
omni_runner_handler.send_audio_speech_request({
def test_text_to_audio_001(omni_runner, offline_client) -> None:
offline_client.send_audio_speech_request({
"input": "...",
"task_type": "Base",
"ref_audio": REF_AUDIO_URL,
@@ -214,8 +214,8 @@ def test_text_to_audio_001(omni_runner, omni_runner_handler) -> None:
@pytest.mark.tts
@hardware_test(...)
@pytest.mark.parametrize("omni_runner", tts_server_params, indirect=True)
def test_voice_clone_001(omni_runner_handler) -> None:
omni_runner_handler.send_single_stage_tts_request({
def test_voice_clone_001(offline_client) -> None:
offline_client.send_single_stage_tts_request({
"input": "...",
"language": "en",
"prompt_audio_path": REF_AUDIO_PATH,
@@ -231,9 +231,9 @@ def test_voice_clone_001(omni_runner_handler) -> None:
@pytest.mark.omni
@hardware_test(...)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_to_text_001(omni_server, openai_client) -> None:
def test_text_to_text_001(omni_server, online_client) -> None:
request_config = {"model": omni_server.model, "messages": ..., "modalities": ["text"]}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
```
*Online serving e2e — **TTS** (representative):*
@@ -244,9 +244,9 @@ def test_text_to_text_001(omni_server, openai_client) -> None:
@pytest.mark.tts
@hardware_test(...)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_001(omni_server, openai_client) -> None:
def test_text_to_audio_001(omni_server, online_client) -> None:
request_config = {"model": omni_server.model, "input": "...", "response_format": "wav", ...}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
```
*Online serving e2e — **Diffusion** X2I (representative):*
@@ -255,8 +255,8 @@ def test_text_to_audio_001(omni_server, openai_client) -> None:
@pytest.mark.core_model
@pytest.mark.diffusion
@pytest.mark.parametrize("omni_server", _get_default_case(MODEL), indirect=True)
def test_text_to_image_001(omni_server, openai_client) -> None:
openai_client.send_diffusion_request({...}) # chat completions + extra_body
def test_text_to_image_001(omni_server, online_client) -> None:
online_client.send_diffusion_request({...}) # chat completions + extra_body
```
*Online serving e2e — **Diffusion** X2V (representative):*
@@ -264,8 +264,8 @@ def test_text_to_image_001(omni_server, openai_client) -> None:
```python
@pytest.mark.core_model
@pytest.mark.diffusion
def test_text_to_video_001(omni_server, openai_client) -> None:
openai_client.send_video_diffusion_request({"model": ..., "form_data": {...}}) # /v1/videos
def test_text_to_video_001(omni_server, online_client) -> None:
online_client.send_video_diffusion_request({"model": ..., "form_data": {...}}) # /v1/videos
```
*Documentation example tests:* follow the **Preferred Test Strategy** in [l4_doc_example_tests.inc.md](../../../docs/contributing/ci/test_examples/l4_doc_example_tests.inc.md): dynamic extraction for offline READMEs; explicit copied client code for online pages until extraction is justified; use the documented **naming**, **output directory** (page folder + case id), and **skipping** rules (e.g. Gradio-only scripts).
@@ -324,21 +324,21 @@ vLLM-Omni e2e tests start a real **OmniServer** (online) or **OmniRunner** (offl
| Scope | Online fixtures | Offline fixtures | When to use |
|-------|-----------------|------------------|-------------|
| **Module** (default) | `omni_server` → `openai_client` | `omni_runner` → `omni_runner_handler` | **Default for L2/L3/L4 expansion** — amortize model/server init across `test_*` in the same module. Same `OmniServerParams` / runner config can be reused by multiple tests. |
| **Function** | `omni_server_function` → `openai_client_function` | `omni_runner_function` → `omni_runner_handler_function` | **Only when required** — each `test_*` must get a **clean** server/runner (no shared engine/GPU state). Typical: `tests/dfx/reliability/`, sleep/wakeup, crash/restart, tests that mutate global server state. |
| **Module** (default) | `omni_server` → `online_client` | `omni_runner` → `offline_client` | **Default for L2/L3/L4 expansion** — amortize model/server init across `test_*` in the same module. Same `OmniServerParams` / runner config can be reused by multiple tests. |
| **Function** | `omni_server_function` → `online_client_function` | `omni_runner_function` → `offline_client_function` | **Only when required** — each `test_*` must get a **clean** server/runner (no shared engine/GPU state). Typical: `tests/dfx/reliability/`, sleep/wakeup, crash/restart, tests that mutate global server state. |
**Rules:**
1. **Default to module scope** (`omni_server` / `omni_runner`) unless the scenario **explicitly** needs a fresh instance per test function.
2. **Indirect parametrize name must match the fixture name**: `@pytest.mark.parametrize("omni_server", ...)` with `omni_server` + `openai_client`; `@pytest.mark.parametrize("omni_server_function", ...)` with `omni_server_function` + `openai_client_function`. Do not mix module fixture with function client (or vice versa).
2. **Indirect parametrize name must match the fixture name**: `@pytest.mark.parametrize("omni_server", ...)` with `omni_server` + `online_client`; `@pytest.mark.parametrize("omni_server_function", ...)` with `omni_server_function` + `online_client_function`. Do not mix module fixture with function client (or vice versa).
3. Different `OmniServerParams` per `test_*` (e.g. default vs `--enable-cpu-offload`) is still OK with **module** `omni_server` — pytest parametrizes per test node; only switch to `_function` when isolation between tests matters, not merely because `server_args` differ.
4. One `test_*` with **many** server configs (expansion matrix) → single function + `@pytest.mark.parametrize("omni_server", [...], indirect=True)` + module `omni_server` (see in-tree `test_qwen_image_expansion.py`).
```python
# Default — module-scoped server (L4 expansion)
@pytest.mark.parametrize("omni_server", [pytest.param(OmniServerParams(model=MODEL), marks=H100)], indirect=True)
def test_foo_images_generations_default_1024(omni_server, openai_client) -> None:
openai_client.send_images_generations_request({...})
def test_foo_images_generations_default_1024(omni_server, online_client) -> None:
online_client.send_images_generations_request({...})
# Function-scoped — reliability / per-test clean state only
@pytest.mark.parametrize(
@@ -346,9 +346,9 @@ def test_foo_images_generations_default_1024(omni_server, openai_client) -> None
[pytest.param(OmniServerParams(model=MODEL), marks=H100)],
indirect=True,
)
def test_foo_sleep_wakeup_cycle(omni_server_function, openai_client_function) -> None:
openai_client_function.send_omni_sleep_http_request({...})
openai_client_function.send_omni_wakeup_http_request({...})
def test_foo_sleep_wakeup_cycle(omni_server_function, online_client_function) -> None:
online_client_function.send_omni_sleep_http_request({...})
online_client_function.send_omni_wakeup_http_request({...})
```
#### Runtime send helpers — online **and** offline (`tests/helpers/runtime.py`)
@@ -357,19 +357,19 @@ def test_foo_sleep_wakeup_cycle(omni_server_function, openai_client_function) ->
| Principle | Action |
|-----------|--------|
| **Reuse first** | Grep `runtime.py` for an existing **`send_*_request`** on `OpenAIClientHandler` (online) or `OmniRunnerHandler` (offline) that matches the endpoint / pipeline shape. |
| **Reuse first** | Grep `runtime.py` for an existing **`send_*_request`** on `OnlineOmniClient` (online) or `OfflineOmniClient` (offline) that matches the endpoint / pipeline shape. |
| **Extend when close** | If an existing `send_*_request` almost fits (e.g. missing one optional field), extend it in `runtime.py` — do not fork logic in the test file. |
| **Add when missing** | No suitable helper → add **`send_<feature>_request`** (high-level: call + general `assert_*`) or **`send_<route>_<verb>_http_request`** (low-level HTTP for negative/dfx) in **`runtime.py` first**, then call it from tests. |
| **Test module owns payload only** | In `test_*.py`: `MODEL`, deploy path, vendored media, `get_prompt()`, and inline **`request_config` dicts** only. **No** `omni.generate(...)`, raw `requests.post`, OpenAI SDK calls, or `_collect_audio()` / `_process_output()` in e2e tests. |
**Online** (`openai_client` from `omni_server`): `OpenAIClientHandler.send_*_request`.
**Online** (`online_client` from `omni_server`): `OnlineOmniClient.send_*_request`.
**Offline** (`omni_runner_handler` from `omni_runner`): `OmniRunnerHandler.send_*_request`.
**Offline** (`offline_client` from `omni_runner`): `OfflineOmniClient.send_*_request`.
| Do | Don't |
|----|-------|
| Online: `openai_client.send_omni_request`, `send_diffusion_request`, `send_audio_speech_request`, `send_video_diffusion_request`, `send_images_generations_request`, … | `requests.post(f"{base_url}/v1/...", json=…)` or `client.chat.completions.create(...)` inside a test |
| Offline: `omni_runner_handler.send_omni_request`, `send_diffusion_request`, `send_audio_speech_request`, `send_single_stage_tts_request`, `send_single_stage_tts_batch_request`, … | `omni_runner.omni.generate(...)` + hand-rolled tensor/WAV extraction in `test_*.py` |
| Online: `online_client.send_omni_request`, `send_diffusion_request`, `send_audio_speech_request`, `send_video_diffusion_request`, `send_images_generations_request`, … | `requests.post(f"{base_url}/v1/...", json=…)` or `client.chat.completions.create(...)` inside a test |
| Offline: `offline_client.send_omni_request`, `send_diffusion_request`, `send_audio_speech_request`, `send_single_stage_tts_request`, `send_single_stage_tts_batch_request`, … | `omni_runner.omni.generate(...)` + hand-rolled tensor/WAV extraction in `test_*.py` |
| Add missing `send_*` to **`runtime.py`** first; bundle general `assert_*` inside the send helper | A one-off `def _post_*` / `def _collect_audio` at the bottom of a test module |
| Mirror naming/style of neighboring `send_*` (docstring, `request_config` dict, optional `run_level`, `err_code` / `err_message` for negative cases) | Different parameter shapes per test file for the same endpoint |
@@ -392,7 +392,7 @@ def test_foo_sleep_wakeup_cycle(omni_server_function, openai_client_function) ->
Changing `request_config` fields (`size`, `n`, `seed`, server flags) is **not** special validation — the general `assert_*` should read those from `request_config` inside `send_*`.
**`send_*` ↔ `assert_*` pairing (online `OpenAIClientHandler`):**
**`send_*` ↔ `assert_*` pairing (online `OnlineOmniClient`):**
| High-level `send_*` (prefer in L2+ e2e) | Assert already invoked inside `runtime.py` | Low-level HTTP-only sibling |
|----------------------------------------|--------------------------------------------|-----------------------------|
@@ -405,7 +405,7 @@ Changing `request_config` fields (`size`, `n`, `seed`, server flags) is **not**
**Rule:** Tests call high-level `send_*_request` for the general contract. **Never** call the same bundled `assert_*` again. Call an extra `assert_*` in the test **only** for special, case-specific validation.
**Common `OpenAIClientHandler` entry points** (non-exhaustive — grep `runtime.py` before adding):
**Common `OnlineOmniClient` entry points** (non-exhaustive — grep `runtime.py` before adding):
| Area | High-level (SDK + assert) | Low-level HTTP (`*_http_request`) |
|------|---------------------------|-----------------------------------|
@@ -416,7 +416,7 @@ Changing `request_config` fields (`size`, `n`, `seed`, server flags) is **not**
| TTS (online) | `send_audio_speech_request` | `send_audio_speech_http_request`, `send_audio_generate_http_request`, … |
| Ops / meta | — | `send_health_http_request`, `send_models_http_request`, `send_omni_sleep_http_request`, … |
**Common `OmniRunnerHandler` entry points** (offline — grep `runtime.py` before adding):
**Common `OfflineOmniClient` entry points** (offline — grep `runtime.py` before adding):
| Area | High-level (`send_*_request` + assert) | Notes |
|------|----------------------------------------|-------|
@@ -426,7 +426,7 @@ Changing `request_config` fields (`size`, `n`, `seed`, server flags) is **not**
| Single-stage TTS (Coqui XTTS, MOSS-TTS-Nano, …) | `send_single_stage_tts_request`, `send_single_stage_tts_batch_request` | `prompt` + `additional_information` + `omni.generate` — **do not** duplicate in tests |
| New model family | **Add** `send_<family>_request` here first | Then call from `tests/e2e/offline_inference/test_*.py` |
L1 tests under `tests/entrypoints/` may use **FastAPI `TestClient`** or direct handler calls with mocks; they do **not** need `OpenAIClientHandler` / `OmniRunnerHandler`. **L2+ online and offline e2e** must use `runtime.py` `send_*` helpers.
L1 tests under `tests/entrypoints/` may use **FastAPI `TestClient`** or direct handler calls with mocks; they do **not** need `OnlineOmniClient` / `OfflineOmniClient`. **L2+ online and offline e2e** must use `runtime.py` `send_*` helpers.
#### Invalid parameter validation (`tests/dfx/reliability/invalid_param_test/`)
@@ -452,7 +452,7 @@ When the user asks for **invalid parameter validation**, **invalid request bodie
| **Module markers** | `pytestmark = [pytest.mark.slow, pytest.mark.<omni or tts or diffusion>]` |
| **Server fixture** | `_PARAMS` / `_QWEN3_TTS_SPEECH`-style list of `pytest.param(OmniServerParams(...), id="...", marks=hardware_marks(...))`; `@pytest.mark.parametrize("omni_server", _PARAMS, indirect=True)` |
| **Hardware** | `hardware_marks(res={"cuda": "H100"})` for heavy diffusion/omni/video; `hardware_marks(res={"cuda": "L4"})` for smaller TTS models (must match weekly `-m "slow and L4"` step) |
| **HTTP client** | **Low-level** `openai_client.send_*_http_request({..., "err_code": 400, "err_message": (...)} )` — **not** `send_*_request` (success path) |
| **HTTP client** | **Low-level** `online_client.send_*_http_request({..., "err_code": 400, "err_message": (...)} )` — **not** `send_*_request` (success path) |
| **Case shape** | Prefer **one `test_*` per route family** + `@pytest.mark.parametrize("body_spec, err_message", [...])` with stable `id=` per case; or dedicated `test_<route>_malformed_json` when not parametrized |
| **Body helpers** | `_minimal_<endpoint>_json(omni_server)` / `_minimal_*_form_data()`; merge overrides with `body.update(body_spec)` |
| **Known gaps** | `pytest.mark.skip(reason="…#3649")` as `_SKIP_ISSUE_3649` when server validation is not yet strict (mirror neighboring cases) |
@@ -485,13 +485,13 @@ When the user asks for **invalid parameter validation**, **invalid request bodie
@pytest.mark.parametrize("omni_server", _PARAMS, indirect=True)
def test_images_generations_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
body_spec: dict[str, object],
err_message: str | tuple[str, ...],
) -> None:
body = _minimal_images_gen_json(omni_server)
body.update(body_spec)
openai_client.send_images_generations_http_request(
online_client.send_images_generations_http_request(
{"json": body, "timeout": 300, "err_code": 400, "err_message": err_message}
)
```
@@ -542,18 +542,18 @@ def send_images_generations_request(self, request_config: dict[str, Any], ...) -
# tests/e2e/.../test_foo_expansion.py — one case per test_* (no case_id branching)
@pytest.mark.parametrize("omni_server", [pytest.param(OmniServerParams(model=MODEL), marks=SINGLE_L4)], indirect=True)
def test_foo_images_generations_default_1024(omni_server, openai_client) -> None:
openai_client.send_images_generations_request({"json": body, "timeout": 300})
def test_foo_images_generations_default_1024(omni_server, online_client) -> None:
online_client.send_images_generations_request({"json": body, "timeout": 300})
@pytest.mark.parametrize("omni_server", [pytest.param(OmniServerParams(model=MODEL), marks=SINGLE_L4)], indirect=True)
def test_foo_images_generations_sizes_256_512_1024(omni_server, openai_client) -> None:
def test_foo_images_generations_sizes_256_512_1024(omni_server, online_client) -> None:
for size in ("256x256", "512x512", "1024x1024"):
openai_client.send_images_generations_request({"json": {**body, "size": size}, "timeout": 300})
online_client.send_images_generations_request({"json": {**body, "size": size}, "timeout": 300})
# chat diffusion — separate test; send_diffusion_request bundles assert_diffusion_response
@pytest.mark.parametrize("omni_server", [pytest.param(OmniServerParams(model=MODEL), marks=SINGLE_L4)], indirect=True)
def test_foo_chat_completions_t2i_fallback(omni_server, openai_client) -> None:
openai_client.send_diffusion_request({...})
def test_foo_chat_completions_t2i_fallback(omni_server, online_client) -> None:
online_client.send_diffusion_request({...})
```
L1 tests under `tests/entrypoints/` may keep **minimal** asserts next to the handler under test when validating a single branch; still prefer `assertions.py` when the same JSON/media check appears in more than one file or level (e.g. L1 protocol test + L4 e2e).
.claude/skills/vllm-omni-test/references/test-routing.md
+1 -1
View File
@@ -279,7 +279,7 @@ When adding or modifying tests, do not stop at “where the file lives” — al
4. **API client + assert placement**: **General** validation → implement in `assertions.py`, call **inside** `send_*_request` in `runtime.py`; tests call **`send_*_request` only**. **Special** case validation → `assert_*` in `assertions.py`, called **in the test** after `send_*_request`. Low-level `send_*_http_request` is for negative/dfx tests (`err_code`), not ordinary L2+ success e2e.
5. **Shared assertions**: logic in **`tests/helpers/assertions.py`**; general checks inside `send_*_request`; special checks only in the test. No `_assert_*` in `test_*.py`.
6. **One case → one `test_*`**: function name reflects what is validated; no `if case_id == ...` mega-test merging multiple scenarios.
7. **Fixture scope**: default **`omni_server` + `openai_client`** / **`omni_runner` + `omni_runner_handler`** (module). Use **`omni_server_function` + `openai_client_function`** / **`omni_runner_function` + `omni_runner_handler_function`** only when each `test_*` must spawn a fresh instance. Parametrize name must match fixture (`omni_server` vs `omni_server_function`).
7. **Fixture scope**: default **`omni_server` + `online_client`** / **`omni_runner` + `offline_client`** (module). Use **`omni_server_function` + `online_client_function`** / **`omni_runner_function` + `offline_client_function`** only when each `test_*` must spawn a fresh instance. Parametrize name must match fixture (`omni_server` vs `omni_server_function`).
8. **Type marker**: `omni`, `tts`, or `diffusion` on every model e2e module.
9. **Diffusion L4 Function**: wire `*_expansion.py` into **X2I(&A&T)** or **X2V** **Function Test** in **`test-nightly.yml` only** — do not add `test-merge.yml` unless the user also requested L3.
10. **Diffusion L4 Perf** (only when requested): add `tests/dfx/perf/tests/test_<slug>_vllm_omni.json` + **Perf Test · &lt;Model&gt;** step (artifact upload); not part of “L4 functional cases” by default.
docs/contributing/ci/test_examples/l4_functionality_tests.inc.md
+5 -5
View File
@@ -1,8 +1,8 @@
**Scope**
#### Scope
For diffusion models, the L4 functionality test covers all or common *diffusion features* that are supported by this model, including several [parallelism acceleration methods](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/parallelism_acceleration/), [CPU offloading](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/cpu_offload/), [TeaCache](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/teacache/) and [Cache-DiT](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/diffusion/cache_dit_acceleration/) cache backends, [quantization methods](https://docs.vllm.ai/projects/vllm-omni/en/latest/user_guide/quantization/overview/).
**Test Case Design**
#### Test Case Design
For a *high priority* model (currently listed in [issue #1832](https://github.com/vllm-project/vllm-omni/issues/1832)), we use several test cases, each with multiple features turned on, so that each supported feature is tested in at least one test case. This is to relieve the GPU workload on the CI machine. The suggested test case combination is as follows:
@@ -34,12 +34,12 @@ For a *normal priority* model, further reduce the number of test cases.
Currently all the features are available in online serving mode. Hence, only need to add `tests/e2e/online_serving/test_{model}_expansion.py`.
**Code Style**
#### Code Style
- Validation: test that the multimodal output files of your model have the correct shapes. `OpenAIClientHandler.send_diffusion_request` should have taken care of this.
- Validation: test that the multimodal output files of your model have the correct shapes. `OnlineOmniClient.send_diffusion_request` should have taken care of this.
- Test marks: always add `full_model` and `diffusion` for L4 nightly `test_*_expansion.py` cases. Add GPU-related marks if needed. Ref: [Markers for Tests](https://docs.vllm.ai/projects/vllm-omni/en/latest/contributing/ci/test_writing_guide/#markers-for-tests).
- To maximize code reuse, you may refer to
- `tests/conftest.py` for `omni_server` (running server in subprocess) and `openai_client` fixtures (sending requests and validating output), `generate_synthetic_image` and `assert_XXX_valid` helper.
- `tests/conftest.py` for `omni_server` (running server in subprocess) and `online_client` fixtures (sending requests and validating output), `generate_synthetic_image` and `assert_XXX_valid` helper.
- `tests/helpers/mark.py` for `@hardware_test(...)` and `hardware_marks`.
- [Parametrizing tests (pytest doc)](https://docs.pytest.org/en/stable/example/parametrize.html) to reuse test function implementation for different cases.
- Doc: add a concise docstring for each test function.
tests/conftest.py
+5 -46
View File
@@ -1,7 +1,10 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Root pytest entrypoint for the vLLM-Omni test suite.
- `tests/conftest.py` stays thin: plugin registration + compatibility re-exports.
- `tests/conftest.py` stays thin: plugin registration only.
- Importable utilities live under `tests/helpers/`.
- Helper unit tests live under `tests/helpers/tests/`.
- Fixtures live under `tests/helpers/fixtures/` and are loaded via `pytest_plugins`.
@@ -19,12 +22,11 @@ bootstrap_vllm_layer_custom_op_modules()
pytest_plugins = (
"tests.helpers.fixtures.config",
"tests.helpers.fixtures.env",
"tests.helpers.fixtures.clean",
"tests.helpers.fixtures.log",
"tests.helpers.fixtures.media",
"tests.helpers.fixtures.run_args",
"tests.helpers.fixtures.runtime",
"tests.helpers.fixtures.speaker_cache",
"tests.helpers.fixtures.xdist",
)
@@ -32,46 +34,3 @@ pytest_plugins = (
def pytest_terminal_summary(terminalreporter, exitstatus, config):
# Marker for Buildkite log folding before pytest summary lines.
terminalreporter.write_line("--- Running Summary")
# Backward-compatible lazy re-exports.
# (Many tests still import from `tests.conftest`; migrate these imports to `tests.helpers.*` over time.)
# Keep these lazy so conftest import does not trigger heavy helper dependencies.
_ASSERTION_EXPORT_NAMES = (
"assert_audio_speech_response",
"assert_diffusion_response",
"assert_http_error",
"assert_image_diffusion_response",
"assert_image_valid",
"assert_omni_response",
"assert_video_diffusion_response",
"assert_video_valid",
)
_MEDIA_EXPORT_NAMES = (
"convert_audio_bytes_to_text",
"convert_audio_file_to_text",
"cosine_similarity_text",
"decode_b64_image",
"generate_synthetic_audio",
"generate_synthetic_image",
"generate_synthetic_video",
)
_STAGE_CONFIG_EXPORT_NAMES = ("modify_stage_config",)
_RUNTIME_EXPORT_NAMES = (
"DiffusionResponse",
"HttpResponse",
"OmniResponse",
"OmniRunner",
"OmniRunnerHandler",
"OmniServer",
"OmniServerParams",
"OmniServerStageCli",
"OpenAIClientHandler",
"dummy_messages_from_mix_data",
)
_LAZY_EXPORT_MODULES = {
**{name: "tests.helpers.assertions" for name in _ASSERTION_EXPORT_NAMES},
**{name: "tests.helpers.media" for name in _MEDIA_EXPORT_NAMES},
**{name: "tests.helpers.stage_config" for name in _STAGE_CONFIG_EXPORT_NAMES},
**{name: "tests.helpers.runtime" for name in _RUNTIME_EXPORT_NAMES},
}
tests/dfx/reliability/invalid_param_test/test_invalid_audio_diffusion.py
+9 -7
View File
@@ -1,13 +1,15 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""``POST /v1/audio/generate`` validation (live Stable Audio Open server)."""
from __future__ import annotations
from typing import Any
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient
pytestmark = [pytest.mark.slow, pytest.mark.diffusion]
@@ -95,20 +97,20 @@ _PARAMS = [
@pytest.mark.parametrize("omni_server", _PARAMS, indirect=True)
def test_audio_generate_invalid_field_values(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
overrides: dict[str, object],
err_message: str | tuple[str, ...],
) -> None:
body = {"model": omni_server.model, "input": "ambient electronic pad"}
body: dict[str, Any] = {"model": omni_server.model, "input": "ambient electronic pad"}
body.update(overrides)
openai_client.send_audio_generate_http_request(
online_client.send_audio_generate_http_request(
{"json": body, "timeout": 120, "err_code": 400, "err_message": err_message}
)
@pytest.mark.parametrize("omni_server", _PARAMS, indirect=True)
def test_audio_generate_missing_input(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
openai_client.send_audio_generate_http_request(
def test_audio_generate_missing_input(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
online_client.send_audio_generate_http_request(
{
"json": {"model": omni_server.model},
"timeout": 120,
tests/dfx/reliability/invalid_param_test/test_invalid_audio_speech.py
+32 -31
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""HTTP + WebSocket validation for Qwen3-TTS Base, Higgs Audio V3 and Audex TTA ``/v1/audio/speech`` (and related routes)."""
from __future__ import annotations
@@ -13,8 +13,8 @@ from typing import Any
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient
from tests.helpers.stage_config import get_deploy_config_path
# Speech / voice-upload numeric caps for these tests only (not exported from vllm_omni).
@@ -26,7 +26,7 @@ _SPEECH_API_MAX_NEW_TOKENS = 4096
_VOICE_UPLOAD_MAX_CONSENT_LEN = 1024
_VOICE_UPLOAD_MAX_REF_TEXT_CHARS = 8192
_VOICE_UPLOAD_MAX_SPEAKER_DESCRIPTION_CHARS = 2048
REF_AUDIO_URL = load_test_audio_data_url("qwen3_tts/clone_2.wav")
REF_AUDIO_URL = get_asset_path("qwen3_tts/clone_2.wav", as_data_url=True)
REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
# Qwen3-TTS 0.6B expects 1024-dim speaker embeddings (see serving_speech / talker hidden_size).
_QWEN3_TTS_06B_SPEAKER_EMBEDDING_DIM = 1024
@@ -57,6 +57,7 @@ _QWEN3_TTS_SPEECH = [
),
]
_HIGGS_AUDIO_V3_SPEECH = [
pytest.param(
OmniServerParams(
@@ -84,7 +85,7 @@ _AUDEX_TTA_SPEECH = [
]
def _apply_batch_overrides(body: dict[str, object], *, loc: str, overrides: dict[str, object]) -> None:
def _apply_batch_overrides(body: dict[str, Any], *, loc: str, overrides: dict[str, object]) -> None:
if loc == "batch":
body.update(overrides)
return
@@ -112,8 +113,8 @@ def _pcm_wav_mono_bytes(*, duration_s: float = 1.1, sample_rate: int = 44100) ->
@pytest.mark.parametrize("omni_server", _QWEN3_TTS_SPEECH, indirect=True)
def test_speech_malformed_json(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
openai_client.send_audio_speech_http_request(
def test_speech_malformed_json(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
online_client.send_audio_speech_http_request(
{
"raw_body": "{",
"timeout": 120,
@@ -124,8 +125,8 @@ def test_speech_malformed_json(omni_server: OmniServer, openai_client: OpenAICli
@pytest.mark.parametrize("omni_server", _QWEN3_TTS_SPEECH, indirect=True)
def test_speech_missing_required_fields(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
openai_client.send_audio_speech_http_request(
def test_speech_missing_required_fields(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
online_client.send_audio_speech_http_request(
{
"json": {"model": omni_server.model},
"timeout": 120,
@@ -242,13 +243,13 @@ def test_speech_missing_required_fields(omni_server: OmniServer, openai_client:
@pytest.mark.parametrize("omni_server", _QWEN3_TTS_SPEECH, indirect=True)
def test_speech_invalid_field_values(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
overrides: dict[str, object],
err_message: str | tuple[str, ...],
) -> None:
ov = dict(overrides)
embedding_only = bool(ov.pop(_SPEECH_INVALID_EMBEDDING_ONLY_BODY, False))
body = {
body: dict[str, Any] = {
"model": omni_server.model,
"input": "Hello.",
"voice": "clone",
@@ -261,7 +262,7 @@ def test_speech_invalid_field_values(
body.pop("ref_audio", None)
body.pop("ref_text", None)
body.update(ov)
openai_client.send_audio_speech_http_request(
online_client.send_audio_speech_http_request(
{"json": body, "timeout": 120, "err_code": 400, "err_message": err_message}
)
@@ -328,12 +329,12 @@ def test_speech_invalid_field_values(
@pytest.mark.parametrize("omni_server", _HIGGS_AUDIO_V3_SPEECH, indirect=True)
def test_speech_higgs_invalid_field_values(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
extra_json: dict[str, object],
err_message: str | tuple[str, ...],
) -> None:
body = {"model": omni_server.model, **extra_json}
openai_client.send_audio_speech_http_request(
online_client.send_audio_speech_http_request(
{"json": body, "timeout": 120, "err_code": 400, "err_message": err_message}
)
@@ -359,12 +360,12 @@ def test_speech_higgs_invalid_field_values(
@pytest.mark.parametrize("omni_server", _AUDEX_TTA_SPEECH, indirect=True)
def test_speech_audex_tta_invalid_field_values(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
overrides: dict[str, object],
err_message: str | tuple[str, ...],
) -> None:
body = {"model": omni_server.model, "input": _AUDEX_TTA_CAPTION, **overrides}
openai_client.send_audio_speech_http_request(
online_client.send_audio_speech_http_request(
{"json": body, "timeout": 120, "err_code": 400, "err_message": err_message}
)
@@ -375,15 +376,15 @@ def test_speech_audex_tta_invalid_field_values(
@pytest.mark.parametrize("omni_server", _QWEN3_TTS_SPEECH, indirect=True)
def test_speech_batch_empty_items(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
openai_client.send_audio_speech_batch_http_request(
def test_speech_batch_empty_items(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
online_client.send_audio_speech_batch_http_request(
{"json": {"items": []}, "timeout": 120, "err_code": (400, 422), "err_message": ("items", "least 1 item")}
)
@pytest.mark.parametrize("omni_server", _QWEN3_TTS_SPEECH, indirect=True)
def test_speech_batch_missing_items(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
openai_client.send_audio_speech_batch_http_request(
def test_speech_batch_missing_items(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
online_client.send_audio_speech_batch_http_request(
{
"json": {"model": omni_server.model},
"timeout": 120,
@@ -618,14 +619,14 @@ def test_speech_batch_missing_items(omni_server: OmniServer, openai_client: Open
@pytest.mark.parametrize("omni_server", _QWEN3_TTS_SPEECH, indirect=True)
def test_speech_batch_invalid_field_values(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
loc: str,
overrides: dict[str, object],
err_message: str | tuple[str, ...],
) -> None:
ov = dict(overrides)
strip_clone = bool(ov.pop(_SPEECH_BATCH_NO_CLONE_FIELDS, False))
body = {
body: dict[str, Any] = {
"model": omni_server.model,
"voice": "clone",
"ref_audio": REF_AUDIO_URL,
@@ -638,7 +639,7 @@ def test_speech_batch_invalid_field_values(
body.pop("ref_audio", None)
body.pop("ref_text", None)
_apply_batch_overrides(body, loc=loc, overrides=ov)
openai_client.send_audio_speech_batch_http_request(
online_client.send_audio_speech_batch_http_request(
{"json": body, "timeout": 120, "err_code": 400, "err_message": err_message}
)
@@ -670,7 +671,7 @@ _SPEECH_STREAM_WS_SESSION_STREAM_AUDIO_WAV_FRAMES = object()
@pytest.mark.parametrize("omni_server", _QWEN3_TTS_SPEECH, indirect=True)
def test_speech_stream_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
send_frames_spec: Any,
err_message: str | tuple[str, ...],
) -> None:
@@ -687,7 +688,7 @@ def test_speech_stream_invalid_requests(
else:
send_frames = send_frames_spec
assert isinstance(send_frames, str)
openai_client.send_audio_speech_stream_ws_request(
online_client.send_audio_speech_stream_ws_request(
{
"send_frames": send_frames,
"timeout": 120,
@@ -893,7 +894,7 @@ def _voices_upload_multipart_files(kind: str | None) -> dict[str, Any] | None:
@pytest.mark.parametrize("omni_server", _QWEN3_TTS_SPEECH, indirect=True)
def test_voices_create_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
multipart_kind: str | None,
form_template: dict[str, Any],
err_message: str | tuple[str, ...],
@@ -909,7 +910,7 @@ def test_voices_create_invalid_requests(
files = _voices_upload_multipart_files(multipart_kind)
if files is not None:
cfg["files"] = files
openai_client.send_audio_voices_create_http_request(cfg)
online_client.send_audio_voices_create_http_request(cfg)
@pytest.mark.parametrize(
@@ -962,13 +963,13 @@ def test_voices_create_invalid_requests(
@pytest.mark.parametrize("omni_server", _QWEN3_TTS_SPEECH, indirect=True)
def test_voices_delete_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
voice_name: str,
err_code: int | tuple[int, ...],
err_message: str | tuple[str, ...],
) -> None:
"""DELETE ``/v1/audio/voices/{name}``: missing voice (404), odd segments (often 404), ``.``paths (405)."""
openai_client.send_audio_voices_delete_http_request(
online_client.send_audio_voices_delete_http_request(
{
"name": voice_name,
"timeout": 120,
@@ -1021,7 +1022,7 @@ _MINIMAX_MUSIC3_MAX_FRAMES = 9000
@pytest.mark.parametrize("omni_server", _MINIMAX_MUSIC3_SPEECH, indirect=True)
def test_minimax_music3_speech_invalid_field_values(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
overrides: dict[str, object],
err_message: str | tuple[str, ...],
) -> None:
@@ -1038,6 +1039,6 @@ def test_minimax_music3_speech_invalid_field_values(
body.pop(key, None)
else:
body[key] = value
openai_client.send_audio_speech_http_request(
online_client.send_audio_speech_http_request(
{"json": body, "timeout": 120, "err_code": 400, "err_message": err_message}
)
tests/dfx/reliability/invalid_param_test/test_invalid_image_editing.py
+4 -4
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""``POST /v1/images/edits`` multipart validation (live ``Qwen/Qwen-Image-Edit``)."""
from __future__ import annotations
@@ -9,7 +9,7 @@ from typing import Any
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient
pytestmark = [pytest.mark.slow, pytest.mark.diffusion]
@@ -72,7 +72,7 @@ def _finalize_edits_form_data(template: dict[str, Any], server_model: str) -> di
@pytest.mark.parametrize("omni_server", _PARAMS, indirect=True)
def test_images_edits_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
tiny_png_bytes: bytes,
include_image: bool,
data_template: dict[str, Any],
@@ -87,4 +87,4 @@ def test_images_edits_invalid_requests(
}
if include_image:
cfg["files"] = {"image": ("x.png", tiny_png_bytes, "image/png")}
openai_client.send_images_edits_http_request(cfg)
online_client.send_images_edits_http_request(cfg)
tests/dfx/reliability/invalid_param_test/test_invalid_image_generation.py
+4 -4
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""``POST /v1/images/generations`` validation (live ``Qwen/Qwen-Image``)."""
from __future__ import annotations
@@ -9,7 +9,7 @@ from typing import Any
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient
pytestmark = [pytest.mark.slow, pytest.mark.diffusion]
@@ -126,7 +126,7 @@ def _minimal_images_gen_json(omni_server: OmniServer) -> dict[str, object]:
@pytest.mark.parametrize("omni_server", _PARAMS, indirect=True)
def test_images_generations_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
body_spec: Any,
err_message: str | tuple[str, ...],
) -> None:
@@ -137,6 +137,6 @@ def test_images_generations_invalid_requests(
assert isinstance(body_spec, dict)
body = _minimal_images_gen_json(omni_server)
body.update(body_spec)
openai_client.send_images_generations_http_request(
online_client.send_images_generations_http_request(
{"json": body, "timeout": 300, "err_code": 400, "err_message": err_message}
)
tests/dfx/reliability/invalid_param_test/test_invalid_omni_chat.py
+9 -8
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Invalid inputs on Qwen3-Omni: ``POST /v1/chat/completions``, ``WS /v1/video/chat/stream``, ``WS /v1/realtime``."""
from __future__ import annotations
@@ -10,7 +10,7 @@ from typing import Any
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient
from tests.helpers.stage_config import get_deploy_config_path
pytestmark = [pytest.mark.slow, pytest.mark.omni]
@@ -140,7 +140,7 @@ def _chat_completions_request_without_expectations(omni_server: OmniServer, case
@pytest.mark.parametrize("omni_server", _QWEN3_OMNI_SERVER, indirect=True)
def test_chat_completions_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
case_id: str,
err_code: int | tuple[int, ...],
err_message: str | tuple[str, ...],
@@ -148,7 +148,7 @@ def test_chat_completions_invalid_requests(
cfg = _chat_completions_request_without_expectations(omni_server, case_id)
cfg["err_code"] = err_code
cfg["err_message"] = err_message
openai_client.send_chat_completions_http_request(cfg)[0]
online_client.send_chat_completions_http_request(cfg)[0]
# ─────────────────────────────────────────────────────────────────────────────
@@ -174,7 +174,7 @@ _VIDEO_CHAT_WS_SESSION_MODALITIES_SCALAR = object()
@pytest.mark.parametrize("omni_server", _QWEN3_OMNI_SERVER, indirect=True)
def test_video_chat_stream_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
send_frames_spec: Any,
err_message: str,
) -> None:
@@ -183,7 +183,7 @@ def test_video_chat_stream_invalid_requests(
else:
send_frames = send_frames_spec
assert isinstance(send_frames, str)
openai_client.send_video_chat_stream_ws_request(
online_client.send_video_chat_stream_ws_request(
{
"send_frames": send_frames,
"timeout": 120,
@@ -243,11 +243,12 @@ _REALTIME_WS_INVALID_AUDIO_APPEND = object()
@pytest.mark.parametrize("omni_server", _QWEN3_OMNI_SERVER, indirect=True)
def test_realtime_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
send_frames_spec: Any,
ws_error_code: str,
err_message: str | tuple[str, ...],
) -> None:
send_frames: Any
if send_frames_spec is _REALTIME_WS_MODEL_MISMATCH:
send_frames = json.dumps(
{
@@ -262,7 +263,7 @@ def test_realtime_invalid_requests(
]
else:
send_frames = send_frames_spec
openai_client.send_realtime_ws_request(
online_client.send_realtime_ws_request(
{
"send_frames": send_frames,
"timeout": 120,
tests/dfx/reliability/invalid_param_test/test_invalid_server_control.py
+11 -10
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Omni sleep/wakeup JSON validation and level-2 wake contract (live ``Qwen/Qwen-Image``)."""
from __future__ import annotations
@@ -9,7 +9,7 @@ from typing import Any, Literal
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient
pytestmark = [pytest.mark.slow, pytest.mark.diffusion, pytest.mark.omni]
@@ -86,7 +86,7 @@ _QWEN_IMAGE = [
@pytest.mark.parametrize("omni_server", _QWEN_IMAGE, indirect=True)
def test_omni_sleep_wakeup_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
endpoint: Literal["sleep", "wakeup"],
json_body: dict[str, Any],
err_message: str | tuple[str, ...],
@@ -98,9 +98,9 @@ def test_omni_sleep_wakeup_invalid_requests(
"err_message": err_message,
}
if endpoint == "sleep":
openai_client.send_omni_sleep_http_request(cfg)
online_client.send_omni_sleep_http_request(cfg)
else:
openai_client.send_omni_wakeup_http_request(cfg)
online_client.send_omni_wakeup_http_request(cfg)
# Keep this after the invalid-JSON cases: sleep(level=2) discards weights and the
@@ -108,7 +108,7 @@ def test_omni_sleep_wakeup_invalid_requests(
@pytest.mark.parametrize("omni_server", _QWEN_IMAGE, indirect=True)
def test_wakeup_after_level2_sleep_fails(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
) -> None:
"""Regression for #4473 Repro A via HTTP: wake after sleep(level=2) → 501.
@@ -116,7 +116,7 @@ def test_wakeup_after_level2_sleep_fails(
Live serve surfaces ``NotImplementedError`` as OpenAI-style JSON with code 501.
"""
assert omni_server is not None
sleep_resps = openai_client.send_omni_sleep_http_request(
sleep_resps = online_client.send_omni_sleep_http_request(
{
"json": {"stage_ids": [0], "level": 2},
"timeout": 180,
@@ -124,10 +124,11 @@ def test_wakeup_after_level2_sleep_fails(
)
sleep_resp = sleep_resps[0]
assert sleep_resp.status_code == 200, sleep_resp
assert sleep_resp.json_body is not None
assert sleep_resp.json_body.get("status") == "SUCCESS", sleep_resp.json_body
json_body = sleep_resp.json_body
assert isinstance(json_body, dict)
assert json_body.get("status") == "SUCCESS", json_body
openai_client.send_omni_wakeup_http_request(
online_client.send_omni_wakeup_http_request(
{
"json": {"stage_ids": [0]},
"timeout": 120,
tests/dfx/reliability/invalid_param_test/test_invalid_video_generation.py
+14 -14
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""HTTP video job routes (live ``Wan-AI/Wan2.2-T2V-A14B-Diffusers``).
WebSocket ``/v1/video/chat/stream`` and ``/v1/realtime`` invalid-input checks live in
@@ -13,7 +13,7 @@ from typing import Any
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient
pytestmark = [pytest.mark.slow, pytest.mark.diffusion]
@@ -54,12 +54,12 @@ def _minimal_video_form_data() -> dict[str, str]:
@pytest.mark.parametrize("omni_server", _WAN_T2V, indirect=True)
def test_post_videos_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
form_extra: dict[str, str] | None,
err_message: str | tuple[str, ...],
) -> None:
data: dict[str, str] = {} if form_extra is None else {**_minimal_video_form_data(), **form_extra}
openai_client.send_videos_create_http_request(
online_client.send_videos_create_http_request(
{"data": data, "timeout": 120, "err_code": 400, "err_message": err_message}
)
@@ -85,12 +85,12 @@ def test_post_videos_invalid_requests(
@pytest.mark.parametrize("omni_server", _WAN_T2V, indirect=True)
def test_post_videos_sync_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
form_extra: dict[str, str] | None,
err_message: str | tuple[str, ...],
) -> None:
data: dict[str, str] = {} if form_extra is None else {**_minimal_video_form_data(), **form_extra}
openai_client.send_videos_sync_http_request(
online_client.send_videos_sync_http_request(
{"data": data, "timeout": 120, "err_code": 400, "err_message": err_message}
)
@@ -111,11 +111,11 @@ def test_post_videos_sync_invalid_requests(
@pytest.mark.parametrize("omni_server", _WAN_T2V, indirect=True)
def test_get_videos_list_invalid_requests(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
params: dict[str, Any],
err_message: str | tuple[str, ...],
) -> None:
openai_client.send_videos_list_http_request(
online_client.send_videos_list_http_request(
{"params": params, "timeout": 120, "err_code": 400, "err_message": err_message}
)
@@ -126,8 +126,8 @@ def test_get_videos_list_invalid_requests(
@pytest.mark.parametrize("omni_server", _WAN_T2V, indirect=True)
def test_get_video_invalid_requests(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
openai_client.send_video_retrieve_http_request(
def test_get_video_invalid_requests(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
online_client.send_video_retrieve_http_request(
{
"video_id": "does-not-exist",
"timeout": 120,
@@ -143,8 +143,8 @@ def test_get_video_invalid_requests(omni_server: OmniServer, openai_client: Open
@pytest.mark.parametrize("omni_server", _WAN_T2V, indirect=True)
def test_delete_video_invalid_requests(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
openai_client.send_video_delete_http_request(
def test_delete_video_invalid_requests(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
online_client.send_video_delete_http_request(
{
"video_id": "does-not-exist",
"timeout": 120,
@@ -160,8 +160,8 @@ def test_delete_video_invalid_requests(omni_server: OmniServer, openai_client: O
@pytest.mark.parametrize("omni_server", _WAN_T2V, indirect=True)
def test_get_video_content_invalid_requests(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
openai_client.send_video_content_http_request(
def test_get_video_content_invalid_requests(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
online_client.send_video_content_http_request(
{
"video_id": "does-not-exist",
"timeout": 120,
tests/dfx/reliability/test_reliability_hunyuan_image.py
+18 -15
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
HunyuanImage-3.0-Instruct reliability integration tests (DiT-only deploy).
@@ -35,7 +38,7 @@ from tests.dfx.reliability.helpers import (
worker_residual_timeout_after_kill_signal,
)
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OpenAIClientHandler
from tests.helpers.runtime import OnlineOmniClient
RELIABILITY_SCENARIOS: list[dict[str, Any]] = [
{
@@ -133,8 +136,8 @@ def _image_request_config(omni_server: Any) -> dict[str, Any]:
}
def _submit_image_generation(openai_client: OpenAIClientHandler, omni_server: Any) -> None:
responses = openai_client.send_images_generations_http_request(_image_request_config(omni_server))
def _submit_image_generation(online_client: OnlineOmniClient, omni_server: Any) -> None:
responses = online_client.send_images_generations_http_request(_image_request_config(omni_server))
resp = responses[0]
if resp.status_code >= 400:
raise RuntimeError(
@@ -189,7 +192,7 @@ def _assert_post_fault_health_terminal(host: str, port: int, *, scenario: str) -
@pytest.mark.parametrize("omni_server_function", HUNYUAN_PARAMS, indirect=True)
def test_reliability_fault_gpu_oom_image_large_request_failure(
omni_server_function,
openai_client_function,
online_client_function,
) -> None:
stage_config_path = getattr(omni_server_function, "stage_config_path", None)
device_spec = resolve_oom_device_spec(OOM_INJECTION_CONFIG, stage_config_path)
@@ -202,7 +205,7 @@ def test_reliability_fault_gpu_oom_image_large_request_failure(
)
try:
try:
_submit_image_generation(openai_client_function, omni_server_function)
_submit_image_generation(online_client_function, omni_server_function)
except Exception as exc:
assert_fault_exception(exc, FAULT_ERROR_KEYWORDS)
else:
@@ -222,10 +225,10 @@ def test_reliability_fault_gpu_oom_image_large_request_failure(
@pytest.mark.parametrize("omni_server_function", HUNYUAN_PARAMS, indirect=True)
def test_reliability_fault_process_kill_request_failure(
omni_server_after_fault_function,
openai_client_function,
online_client_function,
) -> None:
try:
_submit_image_generation(openai_client_function, omni_server_after_fault_function)
_submit_image_generation(online_client_function, omni_server_after_fault_function)
except Exception as exc:
assert_fault_exception(exc, PROCESS_KILL_ERROR_KEYWORDS)
else:
@@ -244,7 +247,7 @@ def test_reliability_fault_process_kill_request_failure(
@pytest.mark.parametrize("omni_server_function", HUNYUAN_PARAMS, indirect=True)
def test_reliability_fault_process_kill_health_fast_fail_and_concurrent(
omni_server_after_fault_function,
openai_client_function,
online_client_function,
) -> None:
host = omni_server_after_fault_function.host
port = omni_server_after_fault_function.port
@@ -280,7 +283,7 @@ def test_reliability_fault_process_kill_health_fast_fail_and_concurrent(
start = time.monotonic()
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
futures = [
executor.submit(_submit_image_generation, openai_client_function, omni_server_after_fault_function)
executor.submit(_submit_image_generation, online_client_function, omni_server_after_fault_function)
for _ in range(3)
]
done, pending = concurrent.futures.wait(
@@ -335,12 +338,12 @@ def test_reliability_fault_process_kill_health_fast_fail_and_concurrent(
@pytest.mark.parametrize("omni_server_function", HUNYUAN_PARAMS, indirect=True)
def test_reliability_fault_process_kill_worker_with_load_request_failure(
omni_server_function,
openai_client_function,
online_client_function,
fault_injector: FaultInjector,
) -> None:
scenario = "kill_worker_with_load"
load_result = run_fault_injection_with_rate_load(
submit_request=lambda: _submit_image_generation(openai_client_function, omni_server_function),
submit_request=lambda: _submit_image_generation(online_client_function, omni_server_function),
inject_fault=lambda: fault_injector(omni_server_function),
num_requests=INFLIGHT_INJECTION_REQUEST_COUNT,
request_rate=INFLIGHT_INJECTION_REQUEST_RATE,
@@ -363,13 +366,13 @@ def test_reliability_fault_process_kill_worker_with_load_request_failure(
@pytest.mark.parametrize("omni_server_function", HUNYUAN_PARAMS, indirect=True)
def test_reliability_fault_process_kill_serve_root_with_load_fast_fail_and_cleanup(
omni_server_function,
openai_client_function,
online_client_function,
signal_name: str,
) -> None:
scenario = f"kill_serve_root_with_load_{signal_name.lower()}"
injector = make_server_root_kill_fault_injector(signal_name=signal_name, post_kill_wait_seconds=2.0)
load_result = run_fault_injection_with_rate_load(
submit_request=lambda: _submit_image_generation(openai_client_function, omni_server_function),
submit_request=lambda: _submit_image_generation(online_client_function, omni_server_function),
inject_fault=lambda: injector(omni_server_function),
num_requests=INFLIGHT_INJECTION_REQUEST_COUNT,
request_rate=INFLIGHT_INJECTION_REQUEST_RATE,
@@ -448,7 +451,7 @@ def test_reliability_fault_process_kill_tree_no_load_fast_fail_and_cleanup(
@pytest.mark.parametrize("omni_server_function", HUNYUAN_PARAMS, indirect=True)
def test_reliability_fault_process_kill_tree_with_load_fast_fail_and_cleanup(
omni_server_function,
openai_client_function,
online_client_function,
signal_name: str,
) -> None:
scenario = f"kill_serve_tree_with_load_{signal_name.lower()}"
@@ -458,7 +461,7 @@ def test_reliability_fault_process_kill_tree_with_load_fast_fail_and_cleanup(
inter_kill_wait_seconds=0.1,
)
load_result = run_fault_injection_with_rate_load(
submit_request=lambda: _submit_image_generation(openai_client_function, omni_server_function),
submit_request=lambda: _submit_image_generation(online_client_function, omni_server_function),
inject_fault=lambda: injector(omni_server_function),
num_requests=INFLIGHT_INJECTION_REQUEST_COUNT,
request_rate=INFLIGHT_INJECTION_REQUEST_RATE,
tests/dfx/reliability/test_reliability_qwen3_omni.py
+17 -14
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Qwen3-Omni reliability integration tests.
"""
@@ -63,7 +66,7 @@ DEPLOY_CONFIGS_DIR = Path(__file__).resolve().parent.parent.parent.parent / "vll
def _default_oom_device_spec() -> str:
"""Use currently visible CUDA ordinals to avoid invalid device index in sidecar."""
count = torch.accelerator.device_count()
count = torch.accelerator.device_count() or 0
if count <= 0:
return "0"
return ",".join(str(i) for i in range(count))
@@ -414,7 +417,7 @@ def test_reliability_fault_gpu_oom_error_contract_consistent_chat_speech(
@pytest.mark.slow
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@pytest.mark.parametrize("omni_server_function", QWEN_PARAMS, indirect=True)
def test_reliability_fault_gpu_oom_chat_large_payload_failure(omni_server_function, openai_client_function) -> None:
def test_reliability_fault_gpu_oom_chat_large_payload_failure(omni_server_function, online_client_function) -> None:
device_spec = resolve_oom_device_spec(
OOM_INJECTION_CONFIG,
_deploy_config_path_from_omni_server(omni_server_function),
@@ -444,7 +447,7 @@ def test_reliability_fault_gpu_oom_chat_large_payload_failure(omni_server_functi
"key_words": {"audio": ["test"]},
}
try:
openai_client_function.send_omni_request(request_config, request_num=1)
online_client_function.send_omni_request(request_config, request_num=1)
except Exception as exc:
assert_fault_exception(exc, FAULT_ERROR_KEYWORDS)
else:
@@ -456,7 +459,7 @@ def test_reliability_fault_gpu_oom_chat_large_payload_failure(omni_server_functi
@pytest.mark.slow
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@pytest.mark.parametrize("omni_server_function", QWEN_PARAMS, indirect=True)
def test_reliability_fault_gpu_oom_concurrent_pressure_failure(omni_server_function, openai_client_function) -> None:
def test_reliability_fault_gpu_oom_concurrent_pressure_failure(omni_server_function, online_client_function) -> None:
device_spec = resolve_oom_device_spec(
OOM_INJECTION_CONFIG,
_deploy_config_path_from_omni_server(omni_server_function),
@@ -487,7 +490,7 @@ def test_reliability_fault_gpu_oom_concurrent_pressure_failure(omni_server_funct
"key_words": {"audio": ["test"]},
}
try:
openai_client_function.send_omni_request(request_config, request_num=4)
online_client_function.send_omni_request(request_config, request_num=4)
except Exception as exc:
assert_fault_exception(exc, FAULT_ERROR_KEYWORDS)
else:
@@ -505,7 +508,7 @@ def test_reliability_fault_gpu_oom_concurrent_pressure_failure(omni_server_funct
)
@pytest.mark.parametrize("omni_server_function", QWEN_PARAMS, indirect=True)
def test_reliability_fault_process_kill_request_failure(
omni_server_after_fault_function, openai_client_function
omni_server_after_fault_function, online_client_function
) -> None:
messages = dummy_messages_from_mix_data(
system_prompt=_get_system_prompt(),
@@ -519,7 +522,7 @@ def test_reliability_fault_process_kill_request_failure(
"key_words": {"text": ["beijing"]},
}
try:
openai_client_function.send_omni_request(request_config, request_num=1)
online_client_function.send_omni_request(request_config, request_num=1)
except Exception as exc:
assert_fault_exception(exc, PROCESS_KILL_ERROR_KEYWORDS)
else:
@@ -567,7 +570,7 @@ def test_reliability_fault_process_kill_health_fast_fail_and_concurrent(
assert elapsed <= _PROCESS_KILL_WORKER_CHAT_FF_MAX_ELAPSED_SEC + 1.0, (
f"[process_kill fast_fail] request did not fail fast after fault: {elapsed:.2f}s"
)
assert ff_status >= 500, (
assert ff_status is not None and ff_status >= 500, (
f"[process_kill fast_fail] expected server-side failure after fault, "
f"got status={ff_status}, body={ff_body[:200]!r}"
)
@@ -652,13 +655,13 @@ def test_reliability_fault_process_kill_health_fast_fail_and_concurrent(
@pytest.mark.parametrize("omni_server_function", QWEN_PARAMS, indirect=True)
def test_reliability_fault_process_kill_worker_with_load_request_failure(
omni_server_function,
openai_client_function,
online_client_function,
fault_injector: FaultInjector,
) -> None:
request_config = _chat_request_config(omni_server_function)
scenario = "kill_worker_with_load"
load_result = run_fault_injection_with_rate_load(
submit_request=lambda: openai_client_function.send_omni_request(request_config, request_num=1),
submit_request=lambda: online_client_function.send_omni_request(request_config, request_num=1),
inject_fault=lambda: fault_injector(omni_server_function),
num_requests=INFLIGHT_INJECTION_REQUEST_COUNT,
request_rate=INFLIGHT_INJECTION_REQUEST_RATE,
@@ -704,14 +707,14 @@ def test_reliability_fault_process_kill_serve_root_no_load_fast_fail_and_cleanup
@pytest.mark.parametrize("omni_server_function", QWEN_PARAMS, indirect=True)
def test_reliability_fault_process_kill_serve_root_with_load_fast_fail_and_cleanup(
omni_server_function,
openai_client_function,
online_client_function,
signal_name: str,
) -> None:
request_config = _chat_request_config(omni_server_function)
scenario = f"kill_serve_root_with_load_{signal_name.lower()}"
injector = make_server_root_kill_fault_injector(signal_name=signal_name, post_kill_wait_seconds=2.0)
load_result = run_fault_injection_with_rate_load(
submit_request=lambda: openai_client_function.send_omni_request(request_config, request_num=1),
submit_request=lambda: online_client_function.send_omni_request(request_config, request_num=1),
inject_fault=lambda: injector(omni_server_function),
num_requests=INFLIGHT_INJECTION_REQUEST_COUNT,
request_rate=INFLIGHT_INJECTION_REQUEST_RATE,
@@ -764,7 +767,7 @@ def test_reliability_fault_process_kill_tree_no_load_fast_fail_and_cleanup(
@pytest.mark.parametrize("omni_server_function", QWEN_PARAMS, indirect=True)
def test_reliability_fault_process_kill_tree_with_load_fast_fail_and_cleanup(
omni_server_function,
openai_client_function,
online_client_function,
signal_name: str,
) -> None:
request_config = _chat_request_config(omni_server_function)
@@ -775,7 +778,7 @@ def test_reliability_fault_process_kill_tree_with_load_fast_fail_and_cleanup(
inter_kill_wait_seconds=0.1,
)
load_result = run_fault_injection_with_rate_load(
submit_request=lambda: openai_client_function.send_omni_request(request_config, request_num=1),
submit_request=lambda: online_client_function.send_omni_request(request_config, request_num=1),
inject_fault=lambda: injector(omni_server_function),
num_requests=INFLIGHT_INJECTION_REQUEST_COUNT,
request_rate=INFLIGHT_INJECTION_REQUEST_RATE,
tests/dfx/reliability/test_reliability_wan22.py
+15 -12
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Wan2.2 reliability integration tests.
"""
@@ -183,7 +186,7 @@ def _assert_post_fault_health_terminal(host: str, port: int, *, scenario: str) -
@pytest.mark.slow
@hardware_test(res={"cuda": "H100"}, num_cards=1)
@pytest.mark.parametrize("omni_server_function", DIFFUSION_VIDEO_PARAMS, indirect=True)
def test_reliability_fault_gpu_oom_video_large_request_failure(omni_server_function, openai_client_function) -> None:
def test_reliability_fault_gpu_oom_video_large_request_failure(omni_server_function, online_client_function) -> None:
stage_config_path = getattr(omni_server_function, "stage_config_path", None)
device_spec = resolve_oom_device_spec(OOM_INJECTION_CONFIG, stage_config_path)
handle = inject_gpu_oom(
@@ -211,7 +214,7 @@ def test_reliability_fault_gpu_oom_video_large_request_failure(omni_server_funct
"stream": False,
}
try:
openai_client_function.send_video_diffusion_request(request_config, request_num=1)
online_client_function.send_video_diffusion_request(request_config, request_num=1)
except Exception as exc:
assert_fault_exception(exc, FAULT_ERROR_KEYWORDS)
else:
@@ -230,11 +233,11 @@ def test_reliability_fault_gpu_oom_video_large_request_failure(omni_server_funct
@pytest.mark.parametrize("omni_server_function", DIFFUSION_VIDEO_PARAMS, indirect=True)
def test_reliability_fault_process_kill_request_failure(
omni_server_after_fault_function,
openai_client_function,
online_client_function,
) -> None:
request_config = _video_request_config()
try:
openai_client_function.send_video_diffusion_request(request_config, request_num=1)
online_client_function.send_video_diffusion_request(request_config, request_num=1)
except Exception as exc:
assert_fault_exception(exc, PROCESS_KILL_ERROR_KEYWORDS)
else:
@@ -251,7 +254,7 @@ def test_reliability_fault_process_kill_request_failure(
@pytest.mark.parametrize("omni_server_function", DIFFUSION_VIDEO_PARAMS, indirect=True)
def test_reliability_fault_process_kill_health_fast_fail_and_concurrent(
omni_server_after_fault_function,
openai_client_function,
online_client_function,
) -> None:
"""Black-box: after process kill, /v1/videos fails fast and concurrent calls don't hang.
@@ -308,7 +311,7 @@ def test_reliability_fault_process_kill_health_fast_fail_and_concurrent(
start = time.monotonic()
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
futures = [
executor.submit(openai_client_function.send_video_diffusion_request, request_config, 1) for _ in range(3)
executor.submit(online_client_function.send_video_diffusion_request, request_config, 1) for _ in range(3)
]
done, pending = concurrent.futures.wait(
futures,
@@ -367,13 +370,13 @@ def test_reliability_fault_process_kill_health_fast_fail_and_concurrent(
@pytest.mark.parametrize("omni_server_function", DIFFUSION_VIDEO_PARAMS, indirect=True)
def test_reliability_fault_process_kill_worker_with_load_request_failure(
omni_server_function,
openai_client_function,
online_client_function,
fault_injector: FaultInjector,
) -> None:
request_config = _video_request_config()
scenario = "kill_worker_with_load"
load_result = run_fault_injection_with_rate_load(
submit_request=lambda: openai_client_function.send_video_diffusion_request(request_config, request_num=1),
submit_request=lambda: online_client_function.send_video_diffusion_request(request_config, request_num=1),
inject_fault=lambda: fault_injector(omni_server_function),
num_requests=INFLIGHT_INJECTION_REQUEST_COUNT,
request_rate=INFLIGHT_INJECTION_REQUEST_RATE,
@@ -516,7 +519,7 @@ def test_reliability_video_oom_recovers_after_fault_removed(
@pytest.mark.parametrize("omni_server_function", DIFFUSION_VIDEO_PARAMS, indirect=True)
def test_reliability_fault_process_kill_serve_root_with_load_fast_fail_and_cleanup(
omni_server_function,
openai_client_function,
online_client_function,
signal_name: str,
) -> None:
"""Black-box: during in-flight video requests, kill serve root and verify post-fault behavior/cleanup."""
@@ -524,7 +527,7 @@ def test_reliability_fault_process_kill_serve_root_with_load_fast_fail_and_clean
scenario = f"kill_serve_root_with_load_{signal_name.lower()}"
injector = make_server_root_kill_fault_injector(signal_name=signal_name, post_kill_wait_seconds=2.0)
load_result = run_fault_injection_with_rate_load(
submit_request=lambda: openai_client_function.send_video_diffusion_request(request_config, request_num=1),
submit_request=lambda: online_client_function.send_video_diffusion_request(request_config, request_num=1),
inject_fault=lambda: injector(omni_server_function),
num_requests=INFLIGHT_INJECTION_REQUEST_COUNT,
request_rate=INFLIGHT_INJECTION_REQUEST_RATE,
@@ -601,7 +604,7 @@ def test_reliability_fault_process_kill_tree_no_load_fast_fail_and_cleanup(
@pytest.mark.parametrize("omni_server_function", DIFFUSION_VIDEO_PARAMS, indirect=True)
def test_reliability_fault_process_kill_tree_with_load_fast_fail_and_cleanup(
omni_server_function,
openai_client_function,
online_client_function,
signal_name: str,
) -> None:
"""Black-box: during in-flight video requests, kill server process tree and verify post-fault behavior/cleanup."""
@@ -613,7 +616,7 @@ def test_reliability_fault_process_kill_tree_with_load_fast_fail_and_cleanup(
inter_kill_wait_seconds=0.1,
)
load_result = run_fault_injection_with_rate_load(
submit_request=lambda: openai_client_function.send_video_diffusion_request(request_config, request_num=1),
submit_request=lambda: online_client_function.send_video_diffusion_request(request_config, request_num=1),
inject_fault=lambda: injector(omni_server_function),
num_requests=INFLIGHT_INJECTION_REQUEST_COUNT,
request_rate=INFLIGHT_INJECTION_REQUEST_RATE,
tests/diffusion/cache/test_cache_dit.py
Vendored
+17 -2
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Model specific tests for CacheDiT enablement.
@@ -13,6 +13,7 @@ from unittest.mock import Mock, patch
import pytest
import torch
from cache_dit.caching.cache_blocks.pattern_0_1_2 import CachedBlocks_Pattern_0_1_2
from vllm.distributed import parallel_state
import vllm_omni.diffusion.cache.cachedit as cd_backend
import vllm_omni.diffusion.cache.cachedit.model_specific as cd_model_specific
@@ -61,6 +62,18 @@ def test_custom_cache_dit_enablers_are_registered_explicitly():
assert cd_backend.CUSTOM_DIT_ENABLERS == expected_enablers
@pytest.fixture()
def init_fake_tp_group(mocker):
"""Provide a fake TP group so vLLM linear layers can be instantiated."""
mock_tp = mocker.MagicMock()
mock_tp.world_size = 1
mock_tp.rank_in_group = 0
old = parallel_state._TP
parallel_state._TP = mock_tp
yield
parallel_state._TP = old
def test_wan22_vace_uses_wan22_custom_cache_dit_enabler():
assert cd_backend.CUSTOM_DIT_ENABLERS["Wan22VACEPipeline"] is cd_model_specific.enable_cache_for_wan22
@@ -110,7 +123,9 @@ def test_cachedit_consumers_use_package_api():
for module in modules:
if module == legacy_module or module.startswith(internal_prefix):
invalid_imports.append(f"{source_path.relative_to(package_root)}:{node.lineno}: {module}")
invalid_imports.append(
f"{source_path.relative_to(package_root)}:{getattr(node, 'lineno', '?')}: {module}"
)
assert not invalid_imports, "Cache-DiT consumers must use the package API:\n" + "\n".join(invalid_imports)
tests/diffusion/diffusion_backend/test_diffusers_backend.py
+7 -5
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
from collections import namedtuple
from types import SimpleNamespace
@@ -10,7 +10,7 @@ from diffusers import DiffusionPipeline # pyright: ignore[reportPrivateImportUs
from PIL import Image
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
from vllm_omni.diffusion.data import (
DiffusionOutput,
DiffusionParallelConfig,
@@ -651,7 +651,7 @@ class TestDiffusersBackendEndToEndExecution:
def test_t2i_random_weights(
self,
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
):
messages = dummy_messages_from_mix_data(content_text="a photo of an astronaut riding a horse on mars")
@@ -667,8 +667,10 @@ class TestDiffusersBackendEndToEndExecution:
},
}
response = openai_client.send_diffusion_request(request_config)
image: Image.Image = response[0].images[0] # pyright: ignore[reportOptionalSubscript]
response = online_client.send_diffusion_request(request_config)
images = response[0].images
assert images is not None
image: Image.Image = images[0]
# Request config has incomplete width/height, so internal assertion in `send_diffusion_request` is incomplete.
assert image.size == (512, 512)
tests/diffusion/distributed/test_tensor_parallel.py
+2 -2
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""DiT tensor-parallel parity and perf checks for diffusion models.
@@ -18,8 +18,8 @@ import pytest
import torch
from PIL import Image
from tests.helpers.env import DeviceMemoryMonitor
from tests.helpers.mark import hardware_test
from tests.helpers.monitor import DeviceMemoryMonitor
from tests.helpers.runtime import OmniRunner
from vllm_omni.diffusion.data import DiffusionParallelConfig
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
tests/diffusion/offloader/test_diffusion_cpu_offload.py
+12 -7
View File
@@ -1,25 +1,29 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
import gc
from typing import Any
import numpy as np
import pytest
import torch
from vllm.distributed.parallel_state import cleanup_dist_env_and_memory
from tests.helpers.env import DeviceMemoryMonitor
from tests.helpers.mark import hardware_test
from tests.helpers.monitor import DeviceMemoryMonitor
from tests.helpers.runtime import OmniRunner
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.platforms import current_omni_platform
AUDIO_MODEL = {
AUDIO_MODEL: dict[str, dict[str, int | None]] = {
"stabilityai/stable-audio-open-1.0": {"cuda": 100, "rocm": None},
}
IMAGE_MODELS = {
IMAGE_MODELS: dict[str, dict[str, int | None]] = {
"riverclouds/qwen_image_random": {"cuda": 2200, "rocm": 2100},
}
MODELS = {**AUDIO_MODEL, **IMAGE_MODELS}
MODELS: dict[str, dict[str, int | None]] = {**AUDIO_MODEL, **IMAGE_MODELS}
MODEL_MARKS = {
"riverclouds/qwen_image_random": pytest.mark.core_model,
@@ -29,12 +33,12 @@ MODEL_MARKS = {
_GATED_MODELS = {"stabilityai/stable-audio-open-1.0"}
AUDIO_MODEL_PARAMS = {
AUDIO_MODEL_PARAMS: dict[str, dict[str, Any]] = {
"runner_params": {},
"sampler_params": {},
}
IMAGE_MODEL_PARAMS = {
IMAGE_MODEL_PARAMS: dict[str, dict[str, Any]] = {
"runner_params": {},
"sampler_params": {
"height": 256,
@@ -43,7 +47,7 @@ IMAGE_MODEL_PARAMS = {
}
def inference(model_name: str, offload: bool = True):
def inference(model_name: str, offload: bool = True) -> tuple[float, Any]:
gc.collect()
current_omni_platform.empty_cache()
device_index = current_omni_platform.current_device()
@@ -133,6 +137,7 @@ def test_cpu_offload_diffusion_model(model_name: str):
threshold = MODELS[model_name][platform]
if threshold is None:
pytest.skip(f"Threshold not defined for {platform} on {model_name}")
assert threshold is not None
assert offload_peak_memory + threshold < no_offload_peak_memory, (
f"Offload peak memory {offload_peak_memory} MB should be less than "
tests/diffusion/offloader/test_diffusion_layerwise_offload.py
+13 -7
View File
@@ -1,36 +1,41 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
from typing import Any
import numpy as np
import pytest
import torch
from vllm.distributed.parallel_state import cleanup_dist_env_and_memory
from tests.helpers.env import DeviceMemoryMonitor
from tests.helpers.mark import hardware_test
from tests.helpers.monitor import DeviceMemoryMonitor
from tests.helpers.runtime import OmniRunner
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.platforms import current_omni_platform
AUDIO_MODEL = {
AUDIO_MODEL: dict[str, dict[str, int | None]] = {
"stabilityai/stable-audio-open-1.0": {"cuda": 1500, "rocm": 1500},
}
IMAGE_VIDEO_MODELS = {
IMAGE_VIDEO_MODELS: dict[str, dict[str, int | None]] = {
"riverclouds/qwen_image_random": {"cuda": 4500, "rocm": None},
# "Wan-AI/Wan2.2-T2V-A14B-Diffusers": {"cuda": 45000, "rocm": None},
}
MODELS = {**AUDIO_MODEL, **IMAGE_VIDEO_MODELS}
MODELS: dict[str, dict[str, int | None]] = {**AUDIO_MODEL, **IMAGE_VIDEO_MODELS}
MODEL_MARKS = {
"riverclouds/qwen_image_random": pytest.mark.core_model,
"stabilityai/stable-audio-open-1.0": pytest.mark.full_model,
}
AUDIO_MODEL_PARAMS = {
AUDIO_MODEL_PARAMS: dict[str, dict[str, Any]] = {
"runner_params": {},
"sampler_params": {},
}
IMAGE_VIDEO_MODELS_PARAMS = {
IMAGE_VIDEO_MODELS_PARAMS: dict[str, dict[str, Any]] = {
"runner_params": {"boundary_ratio": 0.875, "flow_shift": 5.0},
"sampler_params": {"height": 480, "width": 640, "num_frames": 5},
}
@@ -55,7 +60,7 @@ def run_inference(
model_name: str,
layerwise_offload: bool = False,
num_inference_steps: int = 3,
) -> float:
) -> tuple[float, Any]:
current_omni_platform.empty_cache()
device_index = current_omni_platform.current_device()
monitor = DeviceMemoryMonitor(device_index=device_index, interval=0.02)
@@ -140,6 +145,7 @@ def test_layerwise_offload_diffusion_model(model_name: str):
if expected_saved_memory is None:
pytest.skip(f"Threshold not defined for {platform} on {model_name}")
assert expected_saved_memory is not None
# Verify that layerwise offloading significantly reduces memory usage
# Passes only if the actual savings exceeds the expected savings
tests/diffusion/quantization/test_gguf_diffusion.py
+2 -2
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""End-to-end tests for GGUF quantization on diffusion models.
Validates that GGUF-quantized diffusion models generate valid images and
@@ -19,8 +19,8 @@ import numpy as np
import pytest
import torch
from tests.helpers.env import DeviceMemoryMonitor
from tests.helpers.mark import hardware_test
from tests.helpers.monitor import DeviceMemoryMonitor
from tests.helpers.runtime import OmniRunner
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.outputs import OmniRequestOutput
tests/e2e/accuracy/helpers.py
+25 -17
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
import json
import os
import re
@@ -14,7 +17,7 @@ from hashlib import sha1
from io import BytesIO
from mimetypes import guess_type
from pathlib import Path
from typing import Any
from typing import IO, Any
from urllib.parse import urlparse
import numpy as np
@@ -35,11 +38,14 @@ class IdentityFtfy:
def apply_ftfy_mock(*, wan_i2v_module: Any | None = None) -> None:
"""Mock ftfy library for test cases in the main process"""
if wan_i2v_module is None:
from diffusers.pipelines.wan import pipeline_wan_i2v as wan_i2v_module
module: Any = wan_i2v_module
if module is None:
from diffusers.pipelines.wan import pipeline_wan_i2v as wan_pipeline
if not hasattr(wan_i2v_module, "ftfy"):
wan_i2v_module.ftfy = IdentityFtfy()
module = wan_pipeline
if not hasattr(module, "ftfy"):
module.ftfy = IdentityFtfy()
print("ftfy (text encoding sanitizer) is not installed. Using mock ftfy implementation (identity function)")
else:
print("ftfy (text encoding sanitizer) is installed. Using actual ftfy implementation.")
@@ -118,6 +124,7 @@ def probe_binary(binary: str, *, test_name: str = "video similarity e2e test") -
resolved = shutil.which(binary)
if resolved is None:
pytest.skip(f"{binary} is required for {test_name}.")
assert resolved is not None
return resolved
@@ -235,7 +242,7 @@ def video_artifact_dir(result_root: Path, source: str) -> Path:
def send_video_request_with_timeout(
openai_client,
online_client,
request_config: dict[str, Any],
*,
timeout_seconds: int,
@@ -266,7 +273,7 @@ def send_video_request_with_timeout(
normalized_form_data["image_reference"] = json.dumps({"image_url": image_reference})
start_time = time.perf_counter()
create_url = openai_client._build_url("/v1/videos")
create_url = online_client._build_url("/v1/videos")
response = requests.post(
create_url,
data=normalized_form_data,
@@ -280,8 +287,8 @@ def send_video_request_with_timeout(
raise requests.HTTPError(f"{exc}; response body: {response.text}", response=response) from exc
job_data = response.json()
video_id = job_data["id"]
openai_client._wait_until_video_completed(video_id, timeout_seconds=timeout_seconds)
video_content = openai_client._download_video_content(video_id)
online_client._wait_until_video_completed(video_id, timeout_seconds=timeout_seconds)
video_content = online_client._download_video_content(video_id)
print(f"online_video_e2e_latency_s={time.perf_counter() - start_time:.3f}")
return video_content
@@ -571,11 +578,11 @@ class SemanticSimilarityScorer:
raise ImportError("FlagEmbedding not installed. Install with: pip install FlagEmbedding")
return self._text_model
def text_similarity(self, text1: str, text2: str) -> float:
def text_similarity(self, text1: str, text2: str) -> dict[str, float | int]:
"""
Compute semantic similarity between two texts using BGE-M3
"""
results = {}
results: dict[str, float | int] = {}
# 1. Text prefix matching (maximum matching character length)
match_count = 0
for char_vllm, char_ref in zip(text1, text2):
@@ -653,7 +660,7 @@ def dreamzero_cfg_parallel_size() -> int:
def _dreamzero_parity_vllm_env() -> dict[str, str]:
from tests.helpers.env import pick_least_used_device_indices
from tests.helpers.clean import pick_least_used_device_indices
from tests.helpers.runtime import get_open_port
parallel_size = dreamzero_cfg_parallel_size()
@@ -683,8 +690,9 @@ def dreamzero_parity_vllm_params(model: str | None = None):
]
if parallel_size > 1:
server_args.extend(["--cfg-parallel-size", str(parallel_size)])
resolved_model = model or os.environ.get("VLLM_DREAMZERO_MODEL") or _DREAMZERO_DEFAULT_MODEL
return OmniServerParams(
model=model or os.environ.get("VLLM_DREAMZERO_MODEL", _DREAMZERO_DEFAULT_MODEL),
model=resolved_model,
stage_config_path=get_deploy_config_path("dreamzero_tp1_cfg2.yaml"),
server_args=server_args,
env_dict=_dreamzero_parity_vllm_env(),
@@ -725,9 +733,9 @@ def dreamzero_parity_vllm_client(
def _dreamzero_create_openpi_vllm_client(host: str, port: int) -> Any:
from openpi_client import msgpack_numpy
_, WebsocketClientPolicy = load_dreamzero_upstream_modules()
_, websocket_client_policy_cls = load_dreamzero_upstream_modules()
class _OpenPIWebsocketClientPolicy(WebsocketClientPolicy):
class _OpenPIWebsocketClientPolicy(websocket_client_policy_cls): # type: ignore[misc, valid-type]
def __init__(
self,
host: str = "127.0.0.1",
@@ -869,7 +877,7 @@ class _DreamZeroParityServer(ABC):
self.port = get_open_port() if port is None else port
self.ready_timeout_s = ready_timeout_s
self.proc: subprocess.Popen[str] | None = None
self._log_file = None
self._log_file: IO[str] | None = None
@abstractmethod
def _build_argv(self) -> list[str]:
@@ -932,7 +940,7 @@ class DreamZeroUpstreamServer(_DreamZeroParityServer):
]
def _build_env(self) -> dict[str, str]:
from tests.helpers.env import pick_least_used_device_indices
from tests.helpers.clean import pick_least_used_device_indices
dreamzero_repo, _ = require_dreamzero_parity_env()
env = os.environ.copy()
tests/e2e/accuracy/hunyuanvideo15_i2v/test_hunyuanvideo15_i2v_video_similarity.py
+7 -4
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
from __future__ import annotations
import os
@@ -149,7 +152,7 @@ def _generate_offline_video(*, image_source: str) -> Path:
def _generate_online_video(
*,
omni_server,
openai_client,
online_client,
image_source: str,
timeout_seconds: int,
) -> Path:
@@ -171,7 +174,7 @@ def _generate_online_video(
"image_reference": build_online_image_reference(online_image_source),
}
online_video_bytes = send_video_request_with_timeout(
openai_client,
online_client,
request_config,
timeout_seconds=timeout_seconds,
)
@@ -204,7 +207,7 @@ def test_hunyuanvideo15_i2v_diffusers_offline_generates_video(
@pytest.mark.parametrize("omni_server", SERVER_CASES, indirect=True)
def test_hunyuanvideo15_i2v_online_serving_generates_video(
omni_server,
openai_client,
online_client,
hunyuanvideo15_i2v_image_source: str | None,
hunyuanvideo15_online_timeout_seconds: int,
) -> None:
@@ -216,7 +219,7 @@ def test_hunyuanvideo15_i2v_online_serving_generates_video(
validate_image_source(image_source)
online_path = _generate_online_video(
omni_server=omni_server,
openai_client=openai_client,
online_client=online_client,
image_source=image_source,
timeout_seconds=hunyuanvideo15_online_timeout_seconds,
)
tests/e2e/accuracy/hunyuanvideo15_t2v/test_hunyuanvideo15_t2v_video_similarity.py
+7 -4
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
from __future__ import annotations
import os
@@ -115,7 +118,7 @@ def _generate_offline_video() -> Path:
return offline_path
def _generate_online_video(*, omni_server, openai_client, timeout_seconds: int) -> Path:
def _generate_online_video(*, omni_server, online_client, timeout_seconds: int) -> Path:
online_path, _ = _artifact_paths()
request_config = {
"model": omni_server.model,
@@ -132,7 +135,7 @@ def _generate_online_video(*, omni_server, openai_client, timeout_seconds: int)
},
}
online_video_bytes = send_video_request_with_timeout(
openai_client,
online_client,
request_config,
timeout_seconds=timeout_seconds,
)
@@ -161,7 +164,7 @@ def test_hunyuanvideo15_t2v_diffusers_offline_generates_video() -> None:
@pytest.mark.parametrize("omni_server", SERVER_CASES, indirect=True)
def test_hunyuanvideo15_t2v_online_serving_generates_video(
omni_server,
openai_client,
online_client,
hunyuanvideo15_online_timeout_seconds: int,
) -> None:
if not torch.cuda.is_available():
@@ -170,7 +173,7 @@ def test_hunyuanvideo15_t2v_online_serving_generates_video(
probe_binary("ffprobe")
online_path = _generate_online_video(
omni_server=omni_server,
openai_client=openai_client,
online_client=online_client,
timeout_seconds=hunyuanvideo15_online_timeout_seconds,
)
assert online_path.exists(), f"Expected online video artifact at {online_path}"
tests/e2e/accuracy/qwen3_omni/qwen3_omni_acc_bench_core.py
+2 -1
View File
@@ -1,4 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Shared helpers for Qwen3-Omni Daily-Omni / Seed-TTS ``vllm bench serve --omni`` accuracy runs.
Local dataset paths are **optional**. When ``VLLM_DAILY_OMNI_QA_JSON`` + ``VLLM_DAILY_OMNI_VIDEO_DIR``
@@ -28,7 +29,7 @@ DEFAULT_SEED_TTS_HF_REPO = "zhaochenyang20/seed-tts-eval"
class OmniBenchServerEndpoint(Protocol):
"""Anything with ``host`` / ``port`` / ``model`` (e.g. :class:`tests.conftest.OmniServer`)."""
"""Anything with ``host`` / ``port`` / ``model`` (e.g. :class:`tests.helpers.runtime.OmniServer`)."""
host: str
port: int
tests/e2e/accuracy/qwen3_omni/test_qwen3_omni.py
+3 -6
View File
@@ -1,4 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Qwen3-Omni accuracy benchmarks (Daily-Omni MCQ + Seed-TTS WER) via ``vllm bench serve --omni``.
Starts a **module-scoped** Omni OpenAI-compatible server (same pattern as ``tests/dfx/perf`` and
@@ -67,12 +68,8 @@ def get_chunk_config(config_path: str | None = None):
"""Load the qwen3_omni CI deploy yaml with async_chunk modifications for streaming mode."""
if config_path is None:
config_path = _CI_DEPLOY
# TODO: remove this workaround once legacy `stage_args` path is deleted.
# The pipeline (qwen3_omni/pipeline.py) already wires
# thinker2talker_async_chunk / talker2code2wav_async_chunk on stage 0/1,
# so only async_chunk needs flipping. Writing nested `engine_args:` into
# the new-schema overlay trips _parse_stage_deploy's legacy branch and
# drops flat fields (load_format, max_num_seqs, ...).
# Pipeline already wires thinker2talker_async_chunk / talker2code2wav_async_chunk
# on stage 0/1; only the top-level async_chunk flag needs flipping here.
return modify_stage_config(config_path, updates={"async_chunk": True})
tests/e2e/accuracy/test_diffusers_backend_similarity.py
+14 -10
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Run vLLM-Omni with diffusers backend, then run diffusers directly. Compare the output similarity.
@@ -39,9 +39,9 @@ from tests.e2e.accuracy.helpers import (
from tests.e2e.accuracy.helpers import (
run_ffmpeg_similarity as _run_ffmpeg_similarity,
)
from tests.helpers.env import run_post_test_cleanup, run_pre_test_cleanup
from tests.helpers.clean import cleanup_test_environment
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniServer, OpenAIClientHandler
from tests.helpers.runtime import OmniServer, OnlineOmniClient
from vllm_omni.diffusion.models.diffusers_adapter.pipeline_diffusers_adapter import (
CUDA_FLASH_ATTENTION_BACKEND_ATTEMPTS,
)
@@ -128,7 +128,7 @@ def _run_vllm_omni_wan22_i2v(
"seed": SEED,
}
with OmniServer(model, server_args, env_dict=env_to_apply_ftfy_mock_in_subproc(), use_omni=True) as omni_server:
client = OpenAIClientHandler(
client = OnlineOmniClient(
host=omni_server.host,
port=omni_server.port,
run_level="full_model",
@@ -139,16 +139,20 @@ def _run_vllm_omni_wan22_i2v(
"image_reference": f"data:image/png;base64,{pil_to_base64(conditioning_image, 'png')}",
}
result = client.send_video_diffusion_request(request_config)[0]
video_bytes = result.videos[0] # pyright: ignore[reportOptionalSubscript] # Guaranteed not None
videos = result.videos
assert videos is not None
video_bytes = videos[0]
output_path.write_bytes(video_bytes)
return result.e2e_latency # pyright: ignore[reportReturnType] # Guaranteed not None
latency = result.e2e_latency
assert latency is not None
return latency
def _run_diffusers_wan22_i2v(*, model: str, output_path: Path, conditioning_image: Image.Image) -> float:
from diffusers import WanImageToVideoPipeline # pyright: ignore[reportPrivateImportUsage]
from diffusers.schedulers.scheduling_unipc_multistep import UniPCMultistepScheduler
run_pre_test_cleanup()
cleanup_test_environment()
apply_ftfy_mock()
pipe: WanImageToVideoPipeline | None = None
try:
@@ -190,7 +194,7 @@ def _run_diffusers_wan22_i2v(*, model: str, output_path: Path, conditioning_imag
gc.collect()
if torch.cuda.is_available():
torch.accelerator.empty_cache()
run_post_test_cleanup()
cleanup_test_environment()
def _run_vllm_omni_qwen_image(*, model: str, output_path: Path) -> tuple[Image.Image, float]:
@@ -248,7 +252,7 @@ def _run_vllm_omni_qwen_image(*, model: str, output_path: Path) -> tuple[Image.I
def _run_diffusers_qwen_image(*, model: str, output_path: Path) -> tuple[Image.Image, float]:
run_pre_test_cleanup()
cleanup_test_environment()
pipe: DiffusionPipeline | None = None
try:
pipe = DiffusionPipeline.from_pretrained(
@@ -288,7 +292,7 @@ def _run_diffusers_qwen_image(*, model: str, output_path: Path) -> tuple[Image.I
gc.collect()
if torch.cuda.is_available():
torch.accelerator.empty_cache()
run_post_test_cleanup()
cleanup_test_environment()
@pytest.mark.benchmark
tests/e2e/accuracy/test_qwen_image.py
+8 -5
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
from __future__ import annotations
import base64
@@ -13,7 +16,7 @@ from diffusers.pipelines.pipeline_utils import DiffusionPipeline
from PIL import Image
from tests.e2e.accuracy.helpers import assert_images_pixel_close, assert_similarity, model_output_dir
from tests.helpers.env import run_post_test_cleanup, run_pre_test_cleanup
from tests.helpers.clean import cleanup_test_environment
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniServer
@@ -149,7 +152,7 @@ def _run_vllm_omni_qwen_image(*, model: str, output_path: Path) -> Image.Image:
def _run_diffusers_qwen_image(*, model: str, output_path: Path) -> Image.Image:
run_pre_test_cleanup()
cleanup_test_environment()
pipe: DiffusionPipeline | None = None
try:
pipe = DiffusionPipeline.from_pretrained(
@@ -179,7 +182,7 @@ def _run_diffusers_qwen_image(*, model: str, output_path: Path) -> Image.Image:
gc.collect()
if torch.cuda.is_available():
torch.accelerator.empty_cache()
run_post_test_cleanup()
cleanup_test_environment()
def _run_vllm_omni_qwen_image_2512(*, model: str, output_path: Path) -> Image.Image:
@@ -211,7 +214,7 @@ def _run_vllm_omni_qwen_image_2512(*, model: str, output_path: Path) -> Image.Im
def _run_diffusers_qwen_image_2512(*, model: str, output_path: Path) -> Image.Image:
run_pre_test_cleanup()
cleanup_test_environment()
pipe: DiffusionPipeline | None = None
try:
pipe = DiffusionPipeline.from_pretrained(
@@ -241,7 +244,7 @@ def _run_diffusers_qwen_image_2512(*, model: str, output_path: Path) -> Image.Im
gc.collect()
if torch.cuda.is_available():
torch.accelerator.empty_cache()
run_post_test_cleanup()
cleanup_test_environment()
@pytest.mark.benchmark
tests/e2e/accuracy/test_qwen_image_edit.py
+8 -5
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
from __future__ import annotations
import gc
@@ -12,7 +15,7 @@ from PIL import Image
from benchmarks.accuracy.common import decode_base64_image, pil_to_png_bytes
from tests.e2e.accuracy.helpers import assert_images_pixel_close, assert_similarity, model_output_dir
from tests.helpers.env import run_post_test_cleanup, run_pre_test_cleanup
from tests.helpers.clean import cleanup_test_environment
from tests.helpers.mark import hardware_test
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServer
@@ -109,7 +112,7 @@ def _run_diffusers_image_edit(
input_images: list[Image.Image],
output_path: Path,
) -> Image.Image:
run_pre_test_cleanup()
cleanup_test_environment()
pipe: QwenImageEditPipeline | QwenImageEditPlusPipeline | None = None
try:
images = input_images[0] if len(input_images) == 1 else input_images
@@ -140,7 +143,7 @@ def _run_diffusers_image_edit(
gc.collect()
if torch.cuda.is_available():
torch.accelerator.empty_cache()
run_post_test_cleanup()
cleanup_test_environment()
def _load_edit_2511_input_images() -> list[Image.Image]:
@@ -189,7 +192,7 @@ def _run_diffusers_image_edit_2511(
input_images: list[Image.Image],
output_path: Path,
) -> Image.Image:
run_pre_test_cleanup()
cleanup_test_environment()
pipe: QwenImageEditPlusPipeline | None = None
try:
pipe = QwenImageEditPlusPipeline.from_pretrained(
@@ -222,7 +225,7 @@ def _run_diffusers_image_edit_2511(
gc.collect()
if torch.cuda.is_available():
torch.accelerator.empty_cache()
run_post_test_cleanup()
cleanup_test_environment()
def _vllm_omni_output_single_image(
tests/e2e/accuracy/test_qwen_image_layered.py
+6 -3
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
from __future__ import annotations
import base64
@@ -13,7 +16,7 @@ from diffusers.pipelines.pipeline_utils import DiffusionPipeline
from PIL import Image
from tests.e2e.accuracy.helpers import assert_image_sequence_similarity, model_output_dir
from tests.helpers.env import run_post_test_cleanup, run_pre_test_cleanup
from tests.helpers.clean import cleanup_test_environment
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniServer
@@ -93,7 +96,7 @@ def _run_vllm_omni_qwen_image_layered(*, model: str, input_image: Image.Image, o
def _run_diffusers_qwen_image_layered(*, model: str, input_image: Image.Image, output_dir: Path) -> list[Image.Image]:
run_pre_test_cleanup()
cleanup_test_environment()
pipe: DiffusionPipeline | None = None
try:
pipe = DiffusionPipeline.from_pretrained(
@@ -126,7 +129,7 @@ def _run_diffusers_qwen_image_layered(*, model: str, input_image: Image.Image, o
gc.collect()
if torch.cuda.is_available():
torch.accelerator.empty_cache()
run_post_test_cleanup()
cleanup_test_environment()
@pytest.mark.benchmark
tests/e2e/accuracy/test_video_streaming_output_similarity.py
+7 -5
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Online serving smoke test for streaming video output similarity.
"""
@@ -9,7 +12,7 @@ import pytest
from tests.e2e.accuracy.helpers import assert_video_similarity_metrics, probe_video
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import DiffusionResponse, OmniServer, OpenAIClientHandler
from tests.helpers.runtime import DiffusionResponse, OmniServer, OnlineOmniClient, get_model_prefix
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -67,8 +70,8 @@ def _request_config(model: str, *, serialize_extra_params: bool) -> dict:
}
def _client_for(server: OmniServer, *, run_level: str) -> OpenAIClientHandler:
return OpenAIClientHandler(
def _client_for(server: OmniServer, *, run_level: str) -> OnlineOmniClient:
return OnlineOmniClient(
host=server.host,
port=server.port,
api_key="EMPTY",
@@ -95,12 +98,11 @@ def _write_video_artifact(response: DiffusionResponse, output_path: Path) -> Pat
)
def test_helios_streaming_output_matches_non_streaming(
model: str,
model_prefix: str,
run_level: str,
tmp_path: Path,
record_property,
):
model = model_prefix + model
model = get_model_prefix() + model
with OmniServer(model, _server_args(streaming_output=True), use_omni=True) as server:
client = _client_for(server, run_level=run_level)
tests/e2e/accuracy/wan22_i2v/test_wan22_i2v_video_similarity.py
+7 -4
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
from __future__ import annotations
import os
@@ -338,7 +341,7 @@ def _artifact_paths(image_source: str) -> tuple[Path, Path, Path]:
def _generate_online_video(
*,
omni_server,
openai_client,
online_client,
image_source: str,
online_timeout_seconds: int,
) -> Path:
@@ -360,7 +363,7 @@ def _generate_online_video(
"image_reference": _build_online_image_reference(image_source),
}
online_video_bytes = _send_video_request_with_timeout(
openai_client,
online_client,
request_config,
timeout_seconds=_online_timeout_seconds(online_timeout_seconds),
)
@@ -412,7 +415,7 @@ def test_wan22_i2v_diffusers_offline_generates_video(
@pytest.mark.parametrize("omni_server", SERVER_CASES, indirect=True)
def test_wan22_i2v_online_serving_generates_video(
omni_server,
openai_client,
online_client,
wan22_i2v_image_source: str | None,
wan22_i2v_online_timeout_seconds: int,
) -> None:
@@ -424,7 +427,7 @@ def test_wan22_i2v_online_serving_generates_video(
_validate_image_source(image_source)
online_path = _generate_online_video(
omni_server=omni_server,
openai_client=openai_client,
online_client=online_client,
image_source=image_source,
online_timeout_seconds=wan22_i2v_online_timeout_seconds,
)
tests/e2e/features/comfyui/test_comfyui_integration.py
+10 -5
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Integration tests for ComfyUI nodes that use the Omni API client, with a mocked AsyncOmni and a real API server running in a background process.
These tests cover the integration between ComfyUI node and the API server, without actual model inference logic.
@@ -39,6 +42,7 @@ from pytest_mock import MockerFixture
from vllm import SamplingParams
from vllm.outputs import CompletionOutput, RequestOutput
from tests.helpers.runtime import get_open_port
from vllm_omni.entrypoints.async_omni import AsyncOmni as RealAsyncOmni
from vllm_omni.entrypoints.cli.serve import OmniServeCommand
from vllm_omni.inputs.data import OmniDiffusionSamplingParams, OmniSamplingParams
@@ -515,7 +519,7 @@ def mock_async_omni(
@pytest.fixture
def api_server(unused_tcp_port_factory, server_case: ServerCase, mock_async_omni, tmp_path):
def api_server(server_case: ServerCase, mock_async_omni, tmp_path):
"""Set up a API server in background process from command line with parametrized model name and mocked AsyncOmni."""
# Override the STORAGE_MANAGER path to a writable temp directory before
# forking the server subprocess. The default /tmp/storage may be owned
@@ -533,7 +537,7 @@ def api_server(unused_tcp_port_factory, server_case: ServerCase, mock_async_omni
cmd = OmniServeCommand()
cmd.subparser_init(subparsers)
port = unused_tcp_port_factory()
port = get_open_port()
args = parser.parse_args(["serve", server_case.served_model, "--omni", "--port", str(port)])
def run_server():
@@ -550,11 +554,12 @@ def api_server(unused_tcp_port_factory, server_case: ServerCase, mock_async_omni
wait_poll_interval = 1
for _ in range(wait_time // wait_poll_interval):
try:
response = requests.get(f"http://127.0.0.1:{port}/health")
response = requests.get(f"http://127.0.0.1:{port}/health", timeout=wait_poll_interval)
if response.status_code == 200:
break
except requests.ConnectionError:
time.sleep(wait_poll_interval)
except requests.RequestException:
pass
time.sleep(wait_poll_interval)
else:
if server_process.is_alive():
server_process.terminate()
tests/e2e/offline_inference/test_covo_audio_expansion.py
+5 -2
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E tests for Covo-Audio-Chat model with audio input and audio/text output.
"""
@@ -25,7 +28,7 @@ def get_question():
@pytest.mark.omni
@hardware_test(res={"cuda": "H100"}, num_cards={"cuda": 1})
@pytest.mark.parametrize("omni_runner", test_params, indirect=True)
def test_audio_to_audio(omni_runner, omni_runner_handler) -> None:
def test_audio_to_audio(omni_runner, offline_client) -> None:
"""
Test audio input → text + audio output.
Deploy Setting: default yaml
@@ -44,4 +47,4 @@ def test_audio_to_audio(omni_runner, omni_runner_handler) -> None:
"audios": (audio, 16000),
"modalities": ["audio"],
}
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
tests/e2e/offline_inference/test_ernie_image_expansion.py
+5 -5
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
End-to-end test for ErnieImage text-to-image generation.
@@ -11,7 +11,7 @@ Equivalent to running:
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniRunnerHandler
from tests.helpers.runtime import OfflineOmniClient
from vllm_omni.diffusion.data import DiffusionParallelConfig
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
@@ -37,9 +37,9 @@ pytestmark = [
@hardware_test(res={"cuda": "L4"}, num_cards=2)
def test_ernie_image_text_to_image(omni_runner_handler: OmniRunnerHandler) -> None:
def test_ernie_image_text_to_image(offline_client: OfflineOmniClient) -> None:
request_config = {
"model": omni_runner_handler.runner.model_name,
"model": offline_client.runner.model_name,
"prompt": "A photo of a cat sitting on a laptop",
"sampling_params": OmniDiffusionSamplingParams(
height=512,
@@ -49,4 +49,4 @@ def test_ernie_image_text_to_image(omni_runner_handler: OmniRunnerHandler) -> No
seed=42,
),
}
omni_runner_handler.send_diffusion_request(request_config)
offline_client.send_diffusion_request(request_config)
tests/e2e/offline_inference/test_flux1_dev_autoround_w4a16_expansion.py
+2 -2
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""E2E tests for FLUX AutoRound W4A16 quantized inference.
These tests require:
@@ -19,8 +19,8 @@ import pytest
import torch
from vllm.distributed.parallel_state import cleanup_dist_env_and_memory
from tests.helpers.env import DeviceMemoryMonitor
from tests.helpers.mark import hardware_test
from tests.helpers.monitor import DeviceMemoryMonitor
from tests.helpers.runtime import OmniRunner
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.outputs import OmniRequestOutput
tests/e2e/offline_inference/test_flux1_kontext_expansion.py
+6 -6
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Tests for FluxKontext model pipeline.
@@ -15,7 +15,7 @@ import pytest
from vllm.assets.image import ImageAsset
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniRunnerHandler
from tests.helpers.runtime import OfflineOmniClient
from vllm_omni.diffusion.data import DiffusionParallelConfig
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
@@ -47,18 +47,18 @@ def _sampling_512() -> OmniDiffusionSamplingParams:
@hardware_test(res={"cuda": "H100"}, num_cards=2)
def test_flux_kontext_text_to_image(omni_runner_handler: OmniRunnerHandler):
def test_flux_kontext_text_to_image(offline_client: OfflineOmniClient):
"""Test FluxKontext text-to-image generation with real model."""
request_config = {
"model": MODEL,
"prompt": "A photo of a cat sitting on a laptop",
"sampling_params": _sampling_512(),
}
omni_runner_handler.send_diffusion_request(request_config)
offline_client.send_diffusion_request(request_config)
@hardware_test(res={"cuda": "H100"}, num_cards=2)
def test_flux_kontext_image_edit(omni_runner_handler: OmniRunnerHandler):
def test_flux_kontext_image_edit(offline_client: OfflineOmniClient):
"""Test FluxKontext image-to-image editing with real model."""
input_image = ImageAsset("2560px-Gfp-wisconsin-madison-the-nature-boardwalk").pil_image.convert("RGB")
request_config = {
@@ -67,4 +67,4 @@ def test_flux_kontext_image_edit(omni_runner_handler: OmniRunnerHandler):
"modalities": ["img2img"],
"sampling_params": _sampling_512(),
}
omni_runner_handler.send_diffusion_request(request_config)
offline_client.send_diffusion_request(request_config)
tests/e2e/offline_inference/test_flux1_schnell_expansion.py
+4 -4
View File
@@ -1,12 +1,12 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Tests for Flux1 Schnell."""
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniRunnerHandler
from tests.helpers.runtime import OfflineOmniClient
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
MODEL = "black-forest-labs/FLUX.1-schnell"
@@ -21,7 +21,7 @@ pytestmark = [
@hardware_test(res={"cuda": "H100"}, num_cards=1)
def test_flux_schnell_text_to_image(omni_runner_handler: OmniRunnerHandler):
def test_flux_schnell_text_to_image(offline_client: OfflineOmniClient):
"""Test FLUX.1-schnell text-to-image generation."""
request_config = {
"model": MODEL,
@@ -33,4 +33,4 @@ def test_flux_schnell_text_to_image(omni_runner_handler: OmniRunnerHandler):
seed=42,
),
}
omni_runner_handler.send_diffusion_request(request_config)
offline_client.send_diffusion_request(request_config)
tests/e2e/offline_inference/test_flux2_klein.py
+17 -15
View File
@@ -1,12 +1,12 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
End-to-end test for Flux2 Klein inpainting.
Inpainting uses ``omni_runner_handler.send_diffusion_request`` with
Inpainting uses ``offline_client.send_diffusion_request`` with
``multi_modal_data`` containing ``image`` and ``mask_image``; see
:meth:`OmniRunnerHandler.send_diffusion_request` in ``tests.helpers.runtime``.
:meth:`OfflineOmniClient.send_diffusion_request` in ``tests.helpers.runtime``.
"""
import pytest
@@ -14,7 +14,7 @@ import torch
from PIL import Image, ImageDraw
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import DiffusionResponse, OmniRunnerHandler
from tests.helpers.runtime import DiffusionResponse, OfflineOmniClient
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.platforms import current_omni_platform
@@ -47,15 +47,17 @@ def _create_test_inputs(color: tuple = (100, 150, 200)):
def _images_from_response(response: DiffusionResponse) -> list[Image.Image]:
if isinstance(response.images[0], list):
return [f for fr in response.images for f in fr]
return list(response.images)
images = response.images
assert images is not None
if images and isinstance(images[0], list):
return [frame for frames in images for frame in frames]
return list(images)
def _send_inpaint_with_generator(
omni_runner_handler: OmniRunnerHandler, prompt: str, input_image, mask_image, generator: torch.Generator
offline_client: OfflineOmniClient, prompt: str, input_image, mask_image, generator: torch.Generator
) -> DiffusionResponse:
return omni_runner_handler.send_diffusion_request(
return offline_client.send_diffusion_request(
{
"model": MODEL,
"prompt": prompt,
@@ -75,15 +77,15 @@ def _send_inpaint_with_generator(
@pytest.mark.slow
@pytest.mark.diffusion
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_flux2_klein_inpaint_deterministic(omni_runner_handler: OmniRunnerHandler):
def test_flux2_klein_inpaint_deterministic(offline_client: OfflineOmniClient):
input_image, mask_image = _create_test_inputs()
seed = 12345
gen1 = torch.Generator(current_omni_platform.device_type).manual_seed(seed)
gen2 = torch.Generator(current_omni_platform.device_type).manual_seed(seed)
r1 = _send_inpaint_with_generator(omni_runner_handler, "A red flower in a field", input_image, mask_image, gen1)
r2 = _send_inpaint_with_generator(omni_runner_handler, "A red flower in a field", input_image, mask_image, gen2)
r1 = _send_inpaint_with_generator(offline_client, "A red flower in a field", input_image, mask_image, gen1)
r2 = _send_inpaint_with_generator(offline_client, "A red flower in a field", input_image, mask_image, gen2)
images1 = _images_from_response(r1)
images2 = _images_from_response(r2)
@@ -96,14 +98,14 @@ def test_flux2_klein_inpaint_deterministic(omni_runner_handler: OmniRunnerHandle
@pytest.mark.slow
@pytest.mark.diffusion
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_flux2_klein_inpaint_different_seeds_different_output(omni_runner_handler: OmniRunnerHandler):
def test_flux2_klein_inpaint_different_seeds_different_output(offline_client: OfflineOmniClient):
input_image, mask_image = _create_test_inputs()
gen1 = torch.Generator(current_omni_platform.device_type).manual_seed(42)
gen2 = torch.Generator(current_omni_platform.device_type).manual_seed(99999)
r1 = _send_inpaint_with_generator(omni_runner_handler, "A beautiful landscape", input_image, mask_image, gen1)
r2 = _send_inpaint_with_generator(omni_runner_handler, "A beautiful landscape", input_image, mask_image, gen2)
r1 = _send_inpaint_with_generator(offline_client, "A beautiful landscape", input_image, mask_image, gen1)
r2 = _send_inpaint_with_generator(offline_client, "A beautiful landscape", input_image, mask_image, gen2)
images1 = _images_from_response(r1)
images2 = _images_from_response(r2)
tests/e2e/offline_inference/test_glm_image_autoround_w4a16_expansion.py
+7 -7
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""E2E tests for GLM-Image AutoRound W4A16 quantized inference.
These tests cover text-to-image and image-to-image generation with
@@ -19,10 +19,10 @@ import pytest
from PIL import Image
from vllm import SamplingParams
from tests.helpers.env import DeviceMemoryMonitor
from tests.helpers.mark import hardware_test
from tests.helpers.media import generate_synthetic_image
from tests.helpers.runtime import OmniRunnerHandler
from tests.helpers.monitor import DeviceMemoryMonitor
from tests.helpers.runtime import OfflineOmniClient
from tests.helpers.stage_config import get_deploy_config_path, modify_stage_config
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.outputs import OmniRequestOutput
@@ -121,7 +121,7 @@ pytestmark = [
@pytest.mark.parametrize("omni_runner", [_OMNI_RUNNER_PARAM], indirect=True)
@hardware_test(res={"cuda": "H100"}, num_cards=2)
def test_glm_image_autoround_w4a16_generates_image(omni_runner_handler: OmniRunnerHandler):
def test_glm_image_autoround_w4a16_generates_image(offline_client: OfflineOmniClient):
"""Load the W4A16 quantized GLM-Image model and verify it produces a valid image."""
gc.collect()
current_omni_platform.empty_cache()
@@ -152,7 +152,7 @@ def test_glm_image_autoround_w4a16_generates_image(omni_runner_handler: OmniRunn
seed=42,
)
outputs = omni_runner_handler.runner.generate(
outputs = offline_client.runner.generate(
[prompt_dict],
[ar_params, diffusion_params],
)
@@ -186,7 +186,7 @@ def test_glm_image_autoround_w4a16_generates_image(omni_runner_handler: OmniRunn
@pytest.mark.parametrize("omni_runner", [_OMNI_RUNNER_PARAM], indirect=True)
@hardware_test(res={"cuda": "H100"}, num_cards=2)
def test_glm_image_autoround_w4a16_image_to_image(omni_runner_handler: OmniRunnerHandler):
def test_glm_image_autoround_w4a16_image_to_image(offline_client: OfflineOmniClient):
"""Load the W4A16 quantized GLM-Image and verify image-to-image generation works."""
ref_image_arr = generate_synthetic_image(WIDTH, HEIGHT)["np_array"]
@@ -217,7 +217,7 @@ def test_glm_image_autoround_w4a16_image_to_image(omni_runner_handler: OmniRunne
seed=42,
)
outputs = omni_runner_handler.runner.generate(
outputs = offline_client.runner.generate(
[prompt_dict],
[ar_params, diffusion_params],
)
tests/e2e/offline_inference/test_higgs_audio_v2_expansion.py
+2 -2
View File
@@ -1,12 +1,12 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Offline expansion tests for higgs-audio v2.
Mirrors examples/offline_inference/text_to_speech/higgs_audio_v2/end2end.py:
build Stage-0 prompt_token_ids with the model's own processor + plain-text
chat template, drive the engine via ``Omni.generate``, and inspect the
Stage-1 PCM payload directly. The OmniRunnerHandler.send_audio_speech_request
Stage-1 PCM payload directly. The OfflineOmniClient.send_audio_speech_request
helper applies a Qwen-style ChatML wrap that is not valid for higgs, so this
test bypasses it (same approach as test_voxtral_tts.py).
tests/e2e/offline_inference/test_indextts2_expansion.py
+9 -8
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""E2E offline inference tests for IndexTTS2 two-stage pipeline.
Stage 0 (GPT AR) → Stage 1 (S2Mel + BigVGAN). Output is 22050 Hz mono WAV.
@@ -8,12 +8,11 @@ Covers: basic TTS.
from __future__ import annotations
from pathlib import Path
import pytest
import torch
from tests.helpers.mark import hardware_test
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniRunner
from tests.helpers.stage_config import get_deploy_config_path
from vllm_omni import Omni
@@ -29,8 +28,7 @@ _OMNI_RUNNER_PARAM = (
STAGE_CONFIG,
)
ASSETS_DIR = Path(__file__).resolve().parents[2] / "assets" / "indextts2"
REF_AUDIO = str(ASSETS_DIR / "ref_audio.wav")
REF_AUDIO = str(get_asset_path("indextts2/ref_audio.wav"))
pytestmark = [
pytest.mark.slow,
@@ -55,9 +53,12 @@ def _sample_rate_from_mm(mm: dict) -> int:
sr = mm.get("sr")
if isinstance(sr, list):
sr = sr[-1] if sr else None
if hasattr(sr, "item"):
return int(sr.item())
return int(sr) if sr is not None else SAMPLE_RATE
if sr is None:
return SAMPLE_RATE
item = getattr(sr, "item", None)
if callable(item):
return int(item())
return int(sr)
def _build_request(text: str, ref_audio: str, **extra) -> dict:
tests/e2e/offline_inference/test_krea2_expansion.py
+21 -19
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""L4 functionality tests for the Krea 2 text-to-image diffusion pipeline (``Krea2Pipeline``).
@@ -15,7 +15,7 @@ import numpy as np
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniRunnerHandler
from tests.helpers.runtime import OfflineOmniClient
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.lora.request import LoRARequest
from vllm_omni.lora.utils import stable_lora_int_id
@@ -49,9 +49,9 @@ def _sampling() -> OmniDiffusionSamplingParams:
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_runner", [(MODEL, None)], indirect=True)
def test_krea2_text_to_image_001(omni_runner_handler: OmniRunnerHandler) -> None:
def test_krea2_text_to_image_001(offline_client: OfflineOmniClient) -> None:
"""Basic functional smoke: a single prompt produces a decoded image."""
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
offline_client.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
@hardware_test(res={"cuda": "H100"})
@@ -60,22 +60,22 @@ def test_krea2_text_to_image_001(omni_runner_handler: OmniRunnerHandler) -> None
[(MODEL, None, {"enable_layerwise_offload": True})],
indirect=True,
)
def test_krea2_layerwise_offload(omni_runner_handler: OmniRunnerHandler) -> None:
def test_krea2_layerwise_offload(offline_client: OfflineOmniClient) -> None:
"""Exercise layerwise CPU offload on the DiT (SupportsComponentDiscovery + _layerwise_offload_blocks_attrs)."""
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
offline_client.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
@hardware_test(res={"cuda": "H100"})
@pytest.mark.cache
@pytest.mark.parametrize("omni_runner", [(MODEL, None, {"cache_backend": "cache_dit"})], indirect=True)
def test_krea2_cache_dit(omni_runner_handler: OmniRunnerHandler) -> None:
def test_krea2_cache_dit(offline_client: OfflineOmniClient) -> None:
"""Exercise Cache-DiT on Krea 2 via the custom Krea2Pipeline enabler (``enable_cache_for_krea2``).
Validates the docs' Cache-DiT support claim for Krea 2. ``has_separate_cfg`` is checkpoint-aware
(False for the distilled Turbo no-CFG path, True for the Raw CFG path); the default Turbo checkpoint
exercises the no-CFG branch.
"""
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
offline_client.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@@ -88,13 +88,13 @@ def test_krea2_cache_dit(omni_runner_handler: OmniRunnerHandler) -> None:
[(MODEL, None, {"use_hsdp": True, "hsdp_shard_size": 2})],
indirect=True,
)
def test_krea2_hsdp(omni_runner_handler: OmniRunnerHandler) -> None:
def test_krea2_hsdp(offline_client: OfflineOmniClient) -> None:
"""Exercise Hybrid Sharded Data Parallel (HSDP) weight sharding on the DiT across 2 GPUs.
Validates the docs' HSDP-support claim for Krea 2. Scheduled on the multi-GPU (distributed_cuda)
shard by ``num_cards=2``; skipped automatically on boxes with fewer than 2 CUDA devices.
"""
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
offline_client.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@@ -107,27 +107,29 @@ def test_krea2_hsdp(omni_runner_handler: OmniRunnerHandler) -> None:
[(MODEL, None, {"tensor_parallel_size": 2, "vae_patch_parallel_size": 2})],
indirect=True,
)
def test_krea2_vae_patch_parallel(omni_runner_handler: OmniRunnerHandler) -> None:
def test_krea2_vae_patch_parallel(offline_client: OfflineOmniClient) -> None:
"""Exercise VAE patch-parallel (decode) across 2 GPUs.
Validates the docs' VAE-patch-parallel (decode) support claim for Krea 2. Scheduled on the
multi-GPU (distributed_cuda) shard by ``num_cards=2``; skipped automatically on boxes with
fewer than 2 CUDA devices.
"""
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
offline_client.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
def _generate(handler: OmniRunnerHandler, lora_request: LoRARequest | None, lora_scale: float = 1.0) -> np.ndarray:
def _generate(handler: OfflineOmniClient, lora_request: LoRARequest | None, lora_scale: float = 1.0) -> np.ndarray:
sp = _sampling()
sp.lora_request = lora_request
sp.lora_scale = lora_scale
resp = handler.send_diffusion_request({"model": MODEL, "prompt": LORA_PROMPT, "sampling_params": sp})
return np.asarray(resp.images[0].convert("RGB"), dtype=np.int16)
images = resp.images
assert images is not None
return np.asarray(images[0].convert("RGB"), dtype=np.int16)
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_runner", [(MODEL, None)], indirect=True)
def test_krea2_lora(omni_runner_handler: OmniRunnerHandler) -> None:
def test_krea2_lora(offline_client: OfflineOmniClient) -> None:
"""Validate diffusion LoRA on Krea 2: visible effect, scale sensitivity, clean deactivation.
Uses ``LORA`` (a vLLM-Omni-compatible PEFT repackaging of ``krea/Krea-2-LoRA-darkbrush``,
@@ -137,10 +139,10 @@ def test_krea2_lora(omni_runner_handler: OmniRunnerHandler) -> None:
"""
lora_request = LoRARequest(lora_name="darkbrush", lora_int_id=stable_lora_int_id(LORA), lora_path=LORA)
baseline = _generate(omni_runner_handler, None)
img_1x = _generate(omni_runner_handler, lora_request, lora_scale=1.0)
img_2x = _generate(omni_runner_handler, lora_request, lora_scale=2.0)
restored = _generate(omni_runner_handler, None)
baseline = _generate(offline_client, None)
img_1x = _generate(offline_client, lora_request, lora_scale=1.0)
img_2x = _generate(offline_client, lora_request, lora_scale=2.0)
restored = _generate(offline_client, None)
diff_1x = np.abs(baseline - img_1x).mean()
diff_2x = np.abs(baseline - img_2x).mean()
tests/e2e/offline_inference/test_ming_flash_omni_expansion.py
+15 -15
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
import os
@@ -53,7 +53,7 @@ test_params = [(m, c) for m in models for c in [get_eager_config()]]
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", test_params_thinker, indirect=True)
def test_text_to_text(omni_runner, omni_runner_handler) -> None:
def test_text_to_text(omni_runner, offline_client) -> None:
"""
Test text-only input processing and text output generation.
Input Modal: text
@@ -62,14 +62,14 @@ def test_text_to_text(omni_runner, omni_runner_handler) -> None:
prompt = build_prompt("请详细介绍鹦鹉的生活习性。")
request_config = {"prompts": prompt, "modalities": ["text"]}
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
@pytest.mark.slow
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", test_params_thinker, indirect=True)
def test_image_to_text(omni_runner, omni_runner_handler) -> None:
def test_image_to_text(omni_runner, offline_client) -> None:
"""
Test image understanding with text output.
Input Modal: image + text
@@ -79,14 +79,14 @@ def test_image_to_text(omni_runner, omni_runner_handler) -> None:
prompt = build_prompt(f"{IMAGE_TOKEN}Describe this image briefly.")
request_config = {"prompts": prompt, "images": image, "modalities": ["text"]}
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
@pytest.mark.slow
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", test_params_thinker, indirect=True)
def test_audio_to_text(omni_runner, omni_runner_handler) -> None:
def test_audio_to_text(omni_runner, offline_client) -> None:
"""
Test audio understanding with text output.
Input Modal: audio + text
@@ -98,14 +98,14 @@ def test_audio_to_text(omni_runner, omni_runner_handler) -> None:
prompt = build_prompt(f"{AUDIO_TOKEN}Please recognize the language of this speech and transcribe it. Format: oral.")
request_config = {"prompts": prompt, "audios": audio, "modalities": ["text"]}
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
@pytest.mark.slow
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", test_params_thinker, indirect=True)
def test_video_to_text(omni_runner, omni_runner_handler) -> None:
def test_video_to_text(omni_runner, offline_client) -> None:
"""
Test video understanding with text output.
Input Modal: video + text
@@ -115,14 +115,14 @@ def test_video_to_text(omni_runner, omni_runner_handler) -> None:
prompt = build_prompt(f"{VIDEO_TOKEN}Describe what is happening in this video.")
request_config = {"prompts": prompt, "videos": video, "modalities": ["text"]}
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
@pytest.mark.slow
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", test_params_thinker, indirect=True)
def test_mixed_to_text(omni_runner, omni_runner_handler) -> None:
def test_mixed_to_text(omni_runner, offline_client) -> None:
"""
Test mixed modality input (image + audio) with text output.
Input Modal: image + audio + text
@@ -135,14 +135,14 @@ def test_mixed_to_text(omni_runner, omni_runner_handler) -> None:
prompt = build_prompt(f"{IMAGE_TOKEN}{AUDIO_TOKEN}Describe the image and transcribe the audio.")
request_config = {"prompts": prompt, "images": image, "audios": audio, "modalities": ["text"]}
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
@pytest.mark.slow
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", test_params, indirect=True)
def test_text_to_audio(omni_runner, omni_runner_handler) -> None:
def test_text_to_audio(omni_runner, offline_client) -> None:
"""
Test text input with audio output via the thinker+talker pipeline.
Input Modal: text
@@ -151,14 +151,14 @@ def test_text_to_audio(omni_runner, omni_runner_handler) -> None:
prompt = build_prompt("请简单介绍一下北京。")
request_config = {"prompts": prompt, "modalities": ["audio"]}
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
@pytest.mark.slow
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", test_params, indirect=True)
def test_image_to_audio(omni_runner, omni_runner_handler) -> None:
def test_image_to_audio(omni_runner, offline_client) -> None:
"""
Test image + text input with audio output via the thinker+talker pipeline.
Input Modal: image + text
@@ -168,4 +168,4 @@ def test_image_to_audio(omni_runner, omni_runner_handler) -> None:
prompt = build_prompt(f"{IMAGE_TOKEN}Describe this image briefly.")
request_config = {"prompts": prompt, "images": image, "modalities": ["audio"]}
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
tests/e2e/offline_inference/test_qwen2_5_omni_autoround_w4a16_expansion.py
+11 -11
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""E2E tests for Qwen2.5-Omni AutoRound W4A16 quantized inference.
These tests cover text, audio, image, video, and mixed-modality inputs
@@ -68,13 +68,13 @@ quant_params = [(QUANTIZED_MODEL, stage_config)]
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "L4"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_text_to_text(omni_runner, omni_runner_handler):
def test_text_to_text(omni_runner, offline_client):
"""Text input → text output with W4A16 quantized Qwen2.5-Omni."""
request_config = {
"prompts": "What is the capital of China?",
"modalities": ["text"],
}
response = omni_runner_handler.send_omni_request(request_config)
response = offline_client.send_omni_request(request_config)
assert response.success, "Request failed"
assert response.text_content and len(response.text_content.strip()) > 0
@@ -87,7 +87,7 @@ def test_text_to_text(omni_runner, omni_runner_handler):
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "L4"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_audio_to_text(omni_runner, omni_runner_handler):
def test_audio_to_text(omni_runner, offline_client):
"""Audio input → text output with W4A16 quantized Qwen2.5-Omni."""
audio = generate_synthetic_audio(1, 1, 16000)["np_array"]
if len(audio.shape) == 2:
@@ -98,7 +98,7 @@ def test_audio_to_text(omni_runner, omni_runner_handler):
"audios": (audio, 16000),
"modalities": ["text"],
}
response = omni_runner_handler.send_omni_request(request_config)
response = offline_client.send_omni_request(request_config)
assert response.success, "Request failed"
assert response.text_content and len(response.text_content.strip()) > 0
@@ -111,7 +111,7 @@ def test_audio_to_text(omni_runner, omni_runner_handler):
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "L4"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_image_to_text(omni_runner, omni_runner_handler):
def test_image_to_text(omni_runner, offline_client):
"""Image input → text output with W4A16 quantized Qwen2.5-Omni."""
image = generate_synthetic_image(16, 16)["np_array"]
@@ -120,7 +120,7 @@ def test_image_to_text(omni_runner, omni_runner_handler):
"images": image,
"modalities": ["text"],
}
response = omni_runner_handler.send_omni_request(request_config)
response = offline_client.send_omni_request(request_config)
assert response.success, "Request failed"
assert response.text_content and len(response.text_content.strip()) > 0
@@ -133,7 +133,7 @@ def test_image_to_text(omni_runner, omni_runner_handler):
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "L4"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_video_to_text(omni_runner, omni_runner_handler):
def test_video_to_text(omni_runner, offline_client):
"""Video input → text output with W4A16 quantized Qwen2.5-Omni."""
video = generate_synthetic_video(16, 16, 30)["np_array"]
@@ -142,7 +142,7 @@ def test_video_to_text(omni_runner, omni_runner_handler):
"videos": video,
"modalities": ["text"],
}
response = omni_runner_handler.send_omni_request(request_config)
response = offline_client.send_omni_request(request_config)
assert response.success, "Request failed"
assert response.text_content and len(response.text_content.strip()) > 0
@@ -155,7 +155,7 @@ def test_video_to_text(omni_runner, omni_runner_handler):
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "L4"}, num_cards=4)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_mix_to_audio(omni_runner, omni_runner_handler):
def test_mix_to_audio(omni_runner, offline_client):
"""Mixed-modality input → audio output with W4A16 quantized Qwen2.5-Omni."""
video = generate_synthetic_video(16, 16, 30)["np_array"]
image = generate_synthetic_image(16, 16)["np_array"]
@@ -170,5 +170,5 @@ def test_mix_to_audio(omni_runner, omni_runner_handler):
"audios": (audio, 16000),
"modalities": ["audio"],
}
response = omni_runner_handler.send_omni_request(request_config)
response = offline_client.send_omni_request(request_config)
assert response.success, "Request failed"
tests/e2e/offline_inference/test_qwen2_5_omni_expansion.py
+7 -4
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E tests for Qwen2.5-Omni model with mixed modality inputs, audio and text output.
"""
@@ -55,7 +58,7 @@ def get_question(prompt_type="mix"):
@pytest.mark.omni
@hardware_test(res={"cuda": "L4", "rocm": "MI325", "xpu": "B60"}, num_cards={"cuda": 4, "rocm": 2, "xpu": 3})
@pytest.mark.parametrize("omni_runner", test_params, indirect=True)
def test_mix_to_audio(omni_runner, omni_runner_handler) -> None:
def test_mix_to_audio(omni_runner, offline_client) -> None:
"""
Test multi-modal input processing and text/audio output generation via OpenAI API.
Deploy Setting: default yaml
@@ -79,14 +82,14 @@ def test_mix_to_audio(omni_runner, omni_runner_handler) -> None:
}
# Test single completion
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
@pytest.mark.slow
@pytest.mark.omni
@hardware_test(res={"cuda": "L4", "rocm": "MI325", "xpu": "B60"}, num_cards={"cuda": 4, "rocm": 2, "xpu": 3})
@pytest.mark.parametrize("omni_runner", test_params, indirect=True)
def test_text_to_text(omni_runner, omni_runner_handler) -> None:
def test_text_to_text(omni_runner, offline_client) -> None:
"""
Test text input processing and text output generation via OpenAI API.
Deploy Setting: default yaml
@@ -99,4 +102,4 @@ def test_text_to_text(omni_runner, omni_runner_handler) -> None:
request_config = {"prompts": get_question("text_only"), "modalities": ["text"]}
# Test single completion
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
tests/e2e/offline_inference/test_qwen3_omni.py
+7 -4
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E offline tests for Omni model with video input and audio output.
"""
@@ -57,23 +60,23 @@ def get_question(prompt_type="video"):
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_runner", test_params, indirect=True)
def test_video_to_audio(omni_runner, omni_runner_handler) -> None:
def test_video_to_audio(omni_runner, offline_client) -> None:
"""Test processing video, generating audio output."""
video = generate_synthetic_video(224, 224, 300)["np_array"]
request_config = {"prompts": get_question(), "videos": video, "modalities": ["audio"]}
# Test single completion
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
@pytest.mark.advanced_model
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=1)
@pytest.mark.parametrize("omni_runner", thinker_test_params, indirect=True)
def test_thinker_only_model_request(omni_runner, omni_runner_handler) -> None:
def test_thinker_only_model_request(omni_runner, offline_client) -> None:
"""Test that we can load and run a request through a model that only has the thinker stage."""
request_config = {"prompts": "what color is the sky?", "modalities": ["text"]}
# Test single completion
omni_runner_handler.send_omni_request(request_config)
offline_client.send_omni_request(request_config)
tests/e2e/offline_inference/test_qwen3_omni_autoround_w4a16_expansion.py
+13 -13
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""E2E tests for Qwen3-Omni AutoRound W4A16 quantized inference.
These tests cover text, audio, image, video, and mixed-modality inputs
@@ -80,13 +80,13 @@ quant_params = [(QUANTIZED_MODEL, stage_config)]
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_text_to_text(omni_runner, omni_runner_handler):
def test_text_to_text(omni_runner, offline_client):
"""Text input → text output with W4A16 quantized Qwen3-Omni."""
request_config = {
"prompts": "What is the capital of France?",
"modalities": ["text"],
}
response = omni_runner_handler.send_request(request_config)
response = offline_client.send_request(request_config)
assert response.success, "Request failed"
assert response.text_content and len(response.text_content.strip()) > 0
@@ -99,7 +99,7 @@ def test_text_to_text(omni_runner, omni_runner_handler):
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_audio_to_text(omni_runner, omni_runner_handler):
def test_audio_to_text(omni_runner, offline_client):
"""Audio input → text output with W4A16 quantized Qwen3-Omni."""
audio = generate_synthetic_audio(1, 1, 16000)["np_array"]
if len(audio.shape) == 2:
@@ -110,7 +110,7 @@ def test_audio_to_text(omni_runner, omni_runner_handler):
"audios": (audio, 16000),
"modalities": ["text"],
}
response = omni_runner_handler.send_request(request_config)
response = offline_client.send_request(request_config)
assert response.success, "Request failed"
assert response.text_content and len(response.text_content.strip()) > 0
@@ -123,7 +123,7 @@ def test_audio_to_text(omni_runner, omni_runner_handler):
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_image_to_text(omni_runner, omni_runner_handler):
def test_image_to_text(omni_runner, offline_client):
"""Image input → text output with W4A16 quantized Qwen3-Omni."""
image = generate_synthetic_image(16, 16)["np_array"]
@@ -132,7 +132,7 @@ def test_image_to_text(omni_runner, omni_runner_handler):
"images": image,
"modalities": ["text"],
}
response = omni_runner_handler.send_request(request_config)
response = offline_client.send_request(request_config)
assert response.success, "Request failed"
assert response.text_content and len(response.text_content.strip()) > 0
@@ -145,7 +145,7 @@ def test_image_to_text(omni_runner, omni_runner_handler):
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_video_to_text(omni_runner, omni_runner_handler):
def test_video_to_text(omni_runner, offline_client):
"""Video input → text output with W4A16 quantized Qwen3-Omni."""
video = generate_synthetic_video(224, 224, 300)["np_array"]
@@ -154,7 +154,7 @@ def test_video_to_text(omni_runner, omni_runner_handler):
"videos": video,
"modalities": ["text"],
}
response = omni_runner_handler.send_request(request_config)
response = offline_client.send_request(request_config)
assert response.success, "Request failed"
assert response.text_content and len(response.text_content.strip()) > 0
@@ -167,7 +167,7 @@ def test_video_to_text(omni_runner, omni_runner_handler):
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_video_to_audio(omni_runner, omni_runner_handler):
def test_video_to_audio(omni_runner, offline_client):
"""Video input → audio output with W4A16 quantized Qwen3-Omni."""
video = generate_synthetic_video(224, 224, 300)["np_array"]
@@ -176,7 +176,7 @@ def test_video_to_audio(omni_runner, omni_runner_handler):
"videos": video,
"modalities": ["audio"],
}
response = omni_runner_handler.send_request(request_config)
response = offline_client.send_request(request_config)
assert response.success, "Request failed"
@@ -188,7 +188,7 @@ def test_video_to_audio(omni_runner, omni_runner_handler):
@_SKIP_ISSUE_5652
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_mix_to_audio(omni_runner, omni_runner_handler):
def test_mix_to_audio(omni_runner, offline_client):
"""Mixed-modality input → audio output with W4A16 quantized Qwen3-Omni."""
video = generate_synthetic_video(224, 224, 300)["np_array"]
image = generate_synthetic_image(16, 16)["np_array"]
@@ -203,5 +203,5 @@ def test_mix_to_audio(omni_runner, omni_runner_handler):
"audios": (audio, 16000),
"modalities": ["audio"],
}
response = omni_runner_handler.send_request(request_config)
response = offline_client.send_request(request_config)
assert response.success, "Request failed"
tests/e2e/offline_inference/test_qwen3_omni_modelopt_nvfp4_w4a4.py
+3 -3
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""E2E tests for Qwen3-Omni ModelOpt NVFP4 W4A4 — Marlin fallback path.
Loads the published full-thinker NVFP4 W4A4 checkpoint on H100 (sm_90),
@@ -74,7 +74,7 @@ quant_params = [(QUANTIZED_MODEL, _stage_config())]
@hardware_test(res={"cuda": "H100"}, num_cards=2)
@pytest.mark.parametrize("omni_runner", quant_params, indirect=True)
def test_marlin_fallback_no_nan_collapse(omni_runner, omni_runner_handler):
def test_marlin_fallback_no_nan_collapse(omni_runner, offline_client):
"""H100 Marlin FP4 fallback: text in -> coherent text out.
On H100 the upstream NVFP4 PWAL routes through
@@ -89,7 +89,7 @@ def test_marlin_fallback_no_nan_collapse(omni_runner, omni_runner_handler):
degenerate-output signature that indicates NaN propagation through
the FP4 GEMM.
"""
response = omni_runner_handler.send_omni_request({"prompts": "Why is the sky blue?", "modalities": ["text"]})
response = offline_client.send_omni_request({"prompts": "Why is the sky blue?", "modalities": ["text"]})
assert response.success, "request failed"
text = (response.text_content or "").strip()
assert text, "empty response — likely server returned only EOS"
tests/e2e/offline_inference/test_qwen3_tts_base.py
+5 -5
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E offline tests for Qwen3-TTS Base model with text input and audio output.
@@ -15,12 +15,12 @@ os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.media import get_asset_path
from tests.helpers.stage_config import get_deploy_config_path, modify_stage_config
MODEL = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
# See tests/e2e/online_serving/test_qwen3_tts_base.py for the vendored-asset rationale.
REF_AUDIO_URL = load_test_audio_data_url("qwen3_tts/clone_2.wav")
REF_AUDIO_URL = get_asset_path("qwen3_tts/clone_2.wav", as_data_url=True)
REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
@@ -70,7 +70,7 @@ def get_prompt():
@pytest.mark.tts
@hardware_test(res={"cuda": "L4", "xpu": "B60"}, num_cards=1)
@pytest.mark.parametrize("omni_runner", tts_server_params, indirect=True)
def test_text_to_audio_001(omni_runner, omni_runner_handler) -> None:
def test_text_to_audio_001(omni_runner, offline_client) -> None:
"""
Test text input processing and audio output via offline Omni runner.
Deploy Setting: qwen3_tts_no_async_chunk.yaml + enforce_eager=true
@@ -87,4 +87,4 @@ def test_text_to_audio_001(omni_runner, omni_runner_handler) -> None:
"ref_audio": REF_AUDIO_URL,
"ref_text": REF_TEXT,
}
omni_runner_handler.send_audio_speech_request(request_config)
offline_client.send_audio_speech_request(request_config)
tests/e2e/offline_inference/test_qwen3_tts_customvoice.py
+3 -3
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E offline tests for Qwen3-TTS CustomVoice model with text input and audio output.
@@ -64,7 +64,7 @@ def get_prompt():
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_runner", tts_server_params, indirect=True)
def test_text_to_audio_001(omni_runner, omni_runner_handler) -> None:
def test_text_to_audio_001(omni_runner, offline_client) -> None:
"""
Test text input processing and audio output via offline Omni runner.
Deploy Setting: qwen3_tts_no_async_chunk.yaml + enforce_eager=true
@@ -74,4 +74,4 @@ def test_text_to_audio_001(omni_runner, omni_runner_handler) -> None:
Datasets: few requests
"""
request_config = {"input": get_prompt(), "voice": "vivian"}
omni_runner_handler.send_audio_speech_request(request_config)
offline_client.send_audio_speech_request(request_config)
tests/e2e/offline_inference/test_stable_audio_expansion.py
+1 -5
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Stable Audio offline e2e: real weights, FP8 + TeaCache (single job to save GPU).
@@ -15,7 +15,6 @@ import numpy as np
import pytest
import torch
from tests.helpers import skip_if_gated_repo_inaccessible
from tests.helpers.assertions import assert_audio_valid
from tests.helpers.mark import hardware_test
from vllm_omni import Omni
@@ -25,8 +24,6 @@ from vllm_omni.platforms import current_omni_platform
pytestmark = [pytest.mark.slow, pytest.mark.diffusion]
_MODEL_REPO = "stabilityai/stable-audio-open-1.0"
_SAMPLE_RATE = 44100
_CLIP_DURATION_S = 2.0
@@ -80,7 +77,6 @@ def test_stable_audio_quantization_and_teacache() -> None:
CI should provide ``HF_TOKEN`` if the checkpoint is gated.
"""
skip_if_gated_repo_inaccessible(_MODEL_REPO)
# ``model_class_name`` must be passed explicitly: the default-stage-cfg
# factory in ``async_omni_engine.py`` reads it out of ``kwargs`` when
# deciding ``final_output_type`` (#2077), and at construction time the
tests/e2e/offline_inference/test_wan22_autoround_w4a16_expansion.py
+20 -20
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""E2E tests for Wan2.2 AutoRound W4A16 quantized inference.
These tests cover I2V (image-to-video) and T2V (text-to-video) generation
@@ -19,8 +19,8 @@ import pytest
import torch
from PIL import Image
from tests.helpers.env import DeviceMemoryMonitor
from tests.helpers.mark import hardware_test
from tests.helpers.monitor import DeviceMemoryMonitor
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
from vllm_omni.platforms import current_omni_platform
@@ -91,7 +91,7 @@ def _create_test_image(width: int = WIDTH, height: int = HEIGHT) -> Image.Image:
return Image.fromarray(arr)
def _generate_i2v_video(omni_runner_handler, prompt: str = "A cat sitting on a table, smooth motion") -> tuple:
def _generate_i2v_video(offline_client, prompt: str = "A cat sitting on a table, smooth motion") -> tuple:
"""Generate one I2V video, return (frames, peak_memory_mb)."""
gc.collect()
current_omni_platform.empty_cache()
@@ -101,7 +101,7 @@ def _generate_i2v_video(omni_runner_handler, prompt: str = "A cat sitting on a t
monitor.start()
image = _create_test_image()
response = omni_runner_handler.send_diffusion_request(
response = offline_client.send_diffusion_request(
{
"prompt": prompt,
"images": image,
@@ -112,7 +112,7 @@ def _generate_i2v_video(omni_runner_handler, prompt: str = "A cat sitting on a t
peak = monitor.peak_used_mb
monitor.stop()
assert response.success, f"Request failed: {response.error_message}"
assert response.success, "Request failed"
assert response.images is not None and len(response.images) > 0, "Expected image output"
frames = response.images[0]
@@ -122,7 +122,7 @@ def _generate_i2v_video(omni_runner_handler, prompt: str = "A cat sitting on a t
return frames, peak
def _generate_t2v_video(omni_runner_handler, prompt: str = "A cat sitting on a table") -> tuple:
def _generate_t2v_video(offline_client, prompt: str = "A cat sitting on a table") -> tuple:
"""Generate one T2V video, return (frames, peak_memory_mb)."""
gc.collect()
current_omni_platform.empty_cache()
@@ -131,7 +131,7 @@ def _generate_t2v_video(omni_runner_handler, prompt: str = "A cat sitting on a t
monitor = DeviceMemoryMonitor(device_index=device_index, interval=0.02)
monitor.start()
response = omni_runner_handler.send_diffusion_request(
response = offline_client.send_diffusion_request(
{
"prompt": prompt,
"sampling_params": _sampling_params_t2v(),
@@ -141,7 +141,7 @@ def _generate_t2v_video(omni_runner_handler, prompt: str = "A cat sitting on a t
peak = monitor.peak_used_mb
monitor.stop()
assert response.success, f"Request failed: {response.error_message}"
assert response.success, "Request failed"
assert response.images is not None and len(response.images) > 0, "Expected image output"
frames = response.images[0]
@@ -158,9 +158,9 @@ def _generate_t2v_video(omni_runner_handler, prompt: str = "A cat sitting on a t
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_runner", quant_i2v_params, indirect=True)
def test_wan22_i2v_autoround_w4a16_generates_video(omni_runner, omni_runner_handler):
def test_wan22_i2v_autoround_w4a16_generates_video(omni_runner, offline_client):
"""Load the W4A16 quantized Wan2.2 I2V model and verify it produces a valid video."""
frames, _ = _generate_i2v_video(omni_runner_handler)
frames, _ = _generate_i2v_video(offline_client)
assert frames is not None, "Expected video frames output"
assert hasattr(frames, "shape"), "Expected frames to have a shape attribute"
@@ -182,9 +182,9 @@ def test_wan22_i2v_autoround_w4a16_generates_video(omni_runner, omni_runner_hand
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_runner", quant_t2v_params, indirect=True)
def test_wan22_t2v_autoround_w4a16_generates_video(omni_runner, omni_runner_handler):
def test_wan22_t2v_autoround_w4a16_generates_video(omni_runner, offline_client):
"""Load the W4A16 quantized Wan2.2 T2V model and verify it produces a valid video."""
frames, _ = _generate_t2v_video(omni_runner_handler)
frames, _ = _generate_t2v_video(offline_client)
assert frames is not None, "Expected video frames output"
assert hasattr(frames, "shape"), "Expected frames to have a shape attribute"
@@ -204,9 +204,9 @@ def test_wan22_t2v_autoround_w4a16_generates_video(omni_runner, omni_runner_hand
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_runner", quant_i2v_params, indirect=True)
def test_wan22_i2v_autoround_w4a16_quant_peak(omni_runner, omni_runner_handler):
def test_wan22_i2v_autoround_w4a16_quant_peak(omni_runner, offline_client):
"""Measure peak GPU memory of W4A16 quantized I2V model."""
frames, peak = _generate_i2v_video(omni_runner_handler)
frames, peak = _generate_i2v_video(offline_client)
assert frames is not None, "Expected video frames output"
_memory_results["quant_i2v"] = peak
@@ -220,9 +220,9 @@ def test_wan22_i2v_autoround_w4a16_quant_peak(omni_runner, omni_runner_handler):
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_runner", baseline_i2v_params, indirect=True)
def test_wan22_i2v_autoround_w4a16_baseline_peak(omni_runner, omni_runner_handler):
def test_wan22_i2v_autoround_w4a16_baseline_peak(omni_runner, offline_client):
"""Measure peak GPU memory of BF16 baseline I2V model."""
frames, peak = _generate_i2v_video(omni_runner_handler)
frames, peak = _generate_i2v_video(offline_client)
assert frames is not None, "Expected video frames output"
_memory_results["baseline_i2v"] = peak
@@ -261,9 +261,9 @@ def test_wan22_i2v_autoround_w4a16_memory_savings():
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_runner", quant_t2v_params, indirect=True)
def test_wan22_t2v_autoround_w4a16_quant_peak(omni_runner, omni_runner_handler):
def test_wan22_t2v_autoround_w4a16_quant_peak(omni_runner, offline_client):
"""Measure peak GPU memory of W4A16 quantized T2V model."""
frames, peak = _generate_t2v_video(omni_runner_handler)
frames, peak = _generate_t2v_video(offline_client)
assert frames is not None, "Expected video frames output"
_memory_results["quant_t2v"] = peak
@@ -277,9 +277,9 @@ def test_wan22_t2v_autoround_w4a16_quant_peak(omni_runner, omni_runner_handler):
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_runner", baseline_t2v_params, indirect=True)
def test_wan22_t2v_autoround_w4a16_baseline_peak(omni_runner, omni_runner_handler):
def test_wan22_t2v_autoround_w4a16_baseline_peak(omni_runner, offline_client):
"""Measure peak GPU memory of BF16 baseline T2V model."""
frames, peak = _generate_t2v_video(omni_runner_handler)
frames, peak = _generate_t2v_video(offline_client)
assert frames is not None, "Expected video frames output"
_memory_results["baseline_t2v"] = peak
tests/e2e/offline_inference/test_wan22_t2v.py
+6 -3
View File
@@ -1,7 +1,10 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniRunnerHandler
from tests.helpers.runtime import OfflineOmniClient
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
MODEL = "Wan-AI/Wan2.2-T2V-A14B-Diffusers"
@@ -13,7 +16,7 @@ NEGATIVE_PROMPT = "low quality, blurry, watermark, text"
@pytest.mark.diffusion
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards={"cuda": 1, "rocm": 1})
@pytest.mark.parametrize("omni_runner", [(MODEL, None)], indirect=True)
def test_text_to_video_001(omni_runner_handler: OmniRunnerHandler):
def test_text_to_video_001(offline_client: OfflineOmniClient):
sampling = OmniDiffusionSamplingParams(
height=512,
width=512,
@@ -29,4 +32,4 @@ def test_text_to_video_001(omni_runner_handler: OmniRunnerHandler):
"negative_prompt": NEGATIVE_PROMPT,
"sampling_params": sampling,
}
omni_runner_handler.send_diffusion_request(request_config)
offline_client.send_diffusion_request(request_config)
tests/e2e/offline_inference/test_z_image.py
+4 -4
View File
@@ -1,10 +1,10 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.runtime import OmniRunnerHandler
from tests.helpers.runtime import OfflineOmniClient
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
MODEL = "Tongyi-MAI/Z-Image-Turbo"
@@ -15,7 +15,7 @@ PROMPT = "A high-detail studio photo of an orange tabby cat sitting on a laptop
@pytest.mark.diffusion
@hardware_test(res={"cuda": "L4", "rocm": "MI325", "xpu": "B60"}, num_cards={"cuda": 1, "rocm": 1, "xpu": 2})
@pytest.mark.parametrize("omni_runner", [(MODEL, None)], indirect=True)
def test_zimage(omni_runner_handler: OmniRunnerHandler):
def test_zimage(offline_client: OfflineOmniClient):
# high resolution may cause OOM on L4
sampling = OmniDiffusionSamplingParams(
height=256,
@@ -30,4 +30,4 @@ def test_zimage(omni_runner_handler: OmniRunnerHandler):
"prompt": PROMPT,
"sampling_params": sampling,
}
omni_runner_handler.send_diffusion_request(request_config)
offline_client.send_diffusion_request(request_config)
tests/e2e/online_serving/helpers/minicpmo_4_5_duplex.py
+6 -2
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Shared fixtures and argument builders for MiniCPM-o 4.5 duplex E2E tests."""
from __future__ import annotations
@@ -11,7 +14,7 @@ from types import SimpleNamespace
import pytest
from huggingface_hub import snapshot_download
from tests.helpers.runtime import OmniServerParams
from tests.helpers.runtime import OmniServerParams, get_model_prefix
from tests.helpers.stage_config import get_deploy_config_path, modify_stage_config
MODEL = "openbmb/MiniCPM-o-4_5"
@@ -73,10 +76,11 @@ def validated_soft_interrupt_wav() -> Path:
return validated_wav(SOFT_INTERRUPT_WAV, SOFT_INTERRUPT_SHA256)
def resolve_ref_audio(model_prefix: str) -> Path:
def resolve_ref_audio() -> Path:
# ``MODEL`` may be a local checkpoint directory instead of a Hub repo id
# (``Path("/prefix") / "/abs/path"`` collapses to the absolute path), so only
# fall back to the Hub cache when no such directory exists on disk.
model_prefix = get_model_prefix()
model_root = Path(model_prefix) / MODEL if model_prefix else Path(MODEL)
if not model_root.is_dir():
model_root = Path(snapshot_download(MODEL, local_files_only=True))
tests/e2e/online_serving/test_bagel.py
+5 -5
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
End-to-end online serving test for Bagel text2img and img2img generation.
@@ -79,7 +79,7 @@ def _build_img2img_messages(prompt: str, image_b64: str) -> list[dict]:
@pytest.mark.diffusion
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_bagel_text2img_online(omni_server, openai_client) -> None:
def test_bagel_text2img_online(omni_server, online_client) -> None:
"""Test Bagel text2img via OpenAI-compatible chat completions API."""
request_config = {
"model": omni_server.model,
@@ -94,14 +94,14 @@ def test_bagel_text2img_online(omni_server, openai_client) -> None:
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.slow
@pytest.mark.diffusion
@hardware_test(res={"cuda": "H100", "rocm": "MI325"})
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_bagel_img2img_online(omni_server, openai_client) -> None:
def test_bagel_img2img_online(omni_server, online_client) -> None:
"""Test Bagel img2img via OpenAI-compatible chat completions API."""
input_image = ImageAsset("2560px-Gfp-wisconsin-madison-the-nature-boardwalk").pil_image.convert("RGB")
buffer = BytesIO()
@@ -119,4 +119,4 @@ def test_bagel_img2img_online(omni_server, openai_client) -> None:
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_bagel_expansion.py
+6 -3
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Comprehensive tests of diffusion features that are available in online serving mode
and are supported by the following models:
@@ -19,7 +22,7 @@ assert_diffusion_response validates successful generation and the expected
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
from tests.helpers.stage_config import get_deploy_config_path, modify_stage_config
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -183,7 +186,7 @@ def _get_diffusion_feature_cases(model: str):
)
def test_bagel(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
):
"""L4 diffusion feature coverage for Bagel on H100.
@@ -218,4 +221,4 @@ def test_bagel(
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_bagel_multi_replicas.py
+5 -2
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Weekly e2e coverage for BAGEL diffusion multi-replica serving.
This test needs 4 H100 GPUs; keep it out of generic test_bagel_* jobs.
@@ -33,7 +36,7 @@ test_params = [
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_multi_stage_diffusion_uses_multi_replica_dit_stage(omni_server, openai_client) -> None:
def test_multi_stage_diffusion_uses_multi_replica_dit_stage(omni_server, online_client) -> None:
request_config = {
"model": omni_server.model,
"messages": dummy_messages_from_mix_data(
@@ -49,7 +52,7 @@ def test_multi_stage_diffusion_uses_multi_replica_dit_stage(omni_server, openai_
},
}
responses = openai_client.send_diffusion_request(request_config, request_num=ROUTE_STRESS_REQUESTS)
responses = online_client.send_diffusion_request(request_config, request_num=ROUTE_STRESS_REQUESTS)
assert len(responses) == ROUTE_STRESS_REQUESTS
assert all(response.success for response in responses)
tests/e2e/online_serving/test_cosmos3.py
+9 -8
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Online serving L2 smoke tests for ``nvidia/Cosmos3-Nano``.
@@ -8,6 +8,7 @@ broader image/video similarity checks stay in ``tests/e2e/accuracy``.
"""
import os
from typing import Any
import pytest
@@ -16,7 +17,7 @@ from tests.helpers.media import generate_synthetic_image, generate_synthetic_vid
from tests.helpers.runtime import (
OmniServer,
OmniServerParams,
OpenAIClientHandler,
OnlineOmniClient,
)
os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
@@ -56,9 +57,9 @@ def _get_diffusion_feature_cases(model: str):
@pytest.mark.core_model
@pytest.mark.diffusion
@pytest.mark.parametrize("omni_server", _get_diffusion_feature_cases(MODEL), indirect=True)
def test_text_to_image_001(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
def test_text_to_image_001(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
"""Default Cosmos3 T2I smoke through ``/v1/images/generations``."""
responses = openai_client.send_images_generations_http_request(
responses = online_client.send_images_generations_http_request(
{
"json": {
"model": omni_server.model,
@@ -85,12 +86,12 @@ def test_text_to_image_001(omni_server: OmniServer, openai_client: OpenAIClientH
@pytest.mark.core_model
@pytest.mark.diffusion
@pytest.mark.parametrize("omni_server", _get_diffusion_feature_cases(MODEL), indirect=True)
def test_video_generation_modes_001(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
def test_video_generation_modes_001(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
"""Default Cosmos3 T2V, I2V, and V2V smoke through the async ``/v1/videos`` endpoint."""
image_reference = f"data:image/jpeg;base64,{generate_synthetic_image(256, 256, seed=42)['base64']}"
video_reference = f"data:video/mp4;base64,{generate_synthetic_video(256, 256, 5)['base64']}"
video_cases = [
video_cases: list[dict[str, Any]] = [
{
"form_data": {
**VIDEO_SHAPE,
@@ -118,8 +119,8 @@ def test_video_generation_modes_001(omni_server: OmniServer, openai_client: Open
]
for case in video_cases:
request_config = {
request_config: dict[str, Any] = {
"model": omni_server.model,
**case,
}
openai_client.send_video_diffusion_request(request_config)
online_client.send_video_diffusion_request(request_config)
tests/e2e/online_serving/test_cosyvoice3_tts_expansion.py
+8 -8
View File
@@ -14,7 +14,7 @@ os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServerParams
from tests.helpers.stage_config import get_deploy_config_path
from vllm_omni.platforms import current_omni_platform
@@ -33,7 +33,7 @@ MODEL = "FunAudioLLM/Fun-CosyVoice3-0.5B-2512"
# Official CosyVoice zero-shot prompt audio and its transcript. Vendored under
# tests/assets/ so the server does not depend on raw.githubusercontent.com being
# reachable at request time (same rationale as issue #3263 for Qwen3-TTS).
REF_AUDIO_URL = load_test_audio_data_url("cosyvoice3/zero_shot_prompt.wav")
REF_AUDIO_URL = get_asset_path("cosyvoice3/zero_shot_prompt.wav", as_data_url=True)
REF_TEXT = "希望你以后能够做的比我还好呦。"
@@ -82,7 +82,7 @@ tts_async_chunk_server_params = [
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_voice_clone_zh_001(omni_server, openai_client) -> None:
def test_voice_clone_zh_001(omni_server, online_client) -> None:
"""
Test voice cloning TTS with Chinese text via OpenAI API.
Deploy Setting: default yaml
@@ -100,12 +100,12 @@ def test_voice_clone_zh_001(omni_server, openai_client) -> None:
"ref_text": REF_TEXT,
}
_set_rocm_request_timeout(request_config)
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_async_chunk_server_params, indirect=True)
def test_voice_clone_zh_002(omni_server, openai_client) -> None:
def test_voice_clone_zh_002(omni_server, online_client) -> None:
"""
Test voice cloning TTS with Chinese text via async_chunk streaming.
Deploy Setting: cosyvoice3.yaml with default ``async_chunk: true``
@@ -124,12 +124,12 @@ def test_voice_clone_zh_002(omni_server, openai_client) -> None:
"ref_text": REF_TEXT,
}
_set_rocm_request_timeout(request_config)
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_voice_clone_en_001(omni_server, openai_client) -> None:
def test_voice_clone_en_001(omni_server, online_client) -> None:
"""
Test voice cloning TTS with English text via OpenAI API.
Deploy Setting: default yaml
@@ -147,4 +147,4 @@ def test_voice_clone_en_001(omni_server, openai_client) -> None:
"ref_text": REF_TEXT,
}
_set_rocm_request_timeout(request_config)
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
tests/e2e/online_serving/test_covo_audio_expansion.py
+3 -3
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for Covo-Audio-Chat model.
"""
@@ -37,7 +37,7 @@ test_params = [
@pytest.mark.omni
@hardware_test(res={"cuda": "H100"}, num_cards=1)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_audio_to_audio_001(omni_server, openai_client) -> None:
def test_audio_to_audio_001(omni_server, online_client) -> None:
"""
Input Modal: text + audio
Output Modal: audio
@@ -58,4 +58,4 @@ def test_audio_to_audio_001(omni_server, openai_client) -> None:
"stream": False,
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
tests/e2e/online_serving/test_dynin_omni_expansion.py
+7 -7
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Example online tests for Dynin-Omni model.
"""
@@ -64,7 +64,7 @@ def _build_i2i_messages(prompt: str) -> list[dict]:
@hardware_test(res={"cuda": "H100", "rocm": "MI325"})
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
def test_send_i2i_request_001(omni_server, openai_client) -> None:
def test_send_i2i_request_001(omni_server, online_client) -> None:
request_config = {
"model": omni_server.model,
"messages": _build_i2i_messages(I2I_PROMPT),
@@ -73,27 +73,27 @@ def test_send_i2i_request_001(omni_server, openai_client) -> None:
"sampling_params_list": [dict(_I2I_STAGE_SAMPLING) for _ in range(_STAGE_COUNT)],
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"})
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
def test_send_t2i_request_001(omni_server, openai_client) -> None:
def test_send_t2i_request_001(omni_server, online_client) -> None:
request_config = {
"model": omni_server.model,
"messages": _build_t2i_messages(T2I_PROMPT),
"modalities": ["image"],
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"})
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
def test_send_t2s_request_001(omni_server, openai_client) -> None:
def test_send_t2s_request_001(omni_server, online_client) -> None:
request_config = {
"model": omni_server.model,
"messages": _build_t2s_messages(T2S_PROMPT),
"modalities": ["audio"],
"audio_ref_text": T2S_PROMPT,
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
tests/e2e/online_serving/test_ernie_image_expansion.py
+4 -4
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
End-to-end test for ErnieImage text-to-image generation via online serving.
@@ -11,7 +11,7 @@ from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import (
OmniServer,
OmniServerParams,
OpenAIClientHandler,
OnlineOmniClient,
dummy_messages_from_mix_data,
)
@@ -45,7 +45,7 @@ def _get_diffusion_feature_cases(model: str):
_get_diffusion_feature_cases(MODEL),
indirect=True,
)
def test_ernie_image(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_ernie_image(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test each diffusion feature with ERNIE-Image (text-to-image), one feature per case."""
messages = dummy_messages_from_mix_data(content_text=T2I_PROMPT)
request_config = {
@@ -60,4 +60,4 @@ def test_ernie_image(omni_server: OmniServer, openai_client: OpenAIClientHandler
"seed": 42,
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_flux2_expansion.py
+6 -3
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Tests for Flux2 Klein; currently Dev is implemented separately,
but ideally these models will fold together in the future.
@@ -15,7 +18,7 @@ from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import (
OmniServer,
OmniServerParams,
OpenAIClientHandler,
OnlineOmniClient,
dummy_messages_from_mix_data,
)
@@ -113,7 +116,7 @@ def _get_diffusion_feature_cases(model: str):
),
indirect=True,
)
def test_flux2_klein(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_flux2_klein(omni_server: OmniServer, online_client: OnlineOmniClient):
messages = dummy_messages_from_mix_data(content_text=POSITIVE_PROMPT)
request_config = {
"model": omni_server.model,
@@ -128,4 +131,4 @@ def test_flux2_klein(omni_server: OmniServer, openai_client: OpenAIClientHandler
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_flux_2_dev_expansion.py
+11 -6
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
End-to-end diffusion coverage for FLUX.2-dev in online serving mode.
@@ -19,7 +22,7 @@ image resolution.
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -116,7 +119,7 @@ def _get_flux_2_dev_feature_cases(model: str):
)
def test_flux_2_dev(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
):
"""Validate FLUX.2-dev online serving with CPU offload."""
@@ -135,7 +138,7 @@ def test_flux_2_dev(
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.parametrize(
@@ -145,14 +148,14 @@ def test_flux_2_dev(
)
def test_flux_2_dev_batched_chat_completions(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
):
"""Validate batched chat completions for diffusion models."""
messages = [
[{"role": "user", "content": PROMPT}],
[{"role": "user", "content": "A sunset over the ocean."}],
]
responses = openai_client.send_batched_chat_completions_http_request(
responses = online_client.send_batched_chat_completions_http_request(
{
"json": {
"model": omni_server.model,
@@ -167,5 +170,7 @@ def test_flux_2_dev_batched_chat_completions(
assert responses and len(responses) == 1
resp = responses[0]
assert resp.success
choices = resp.json_body["choices"]
body = resp.json_body
assert isinstance(body, dict)
choices = body["choices"]
assert len(choices) == len(messages)
tests/e2e/online_serving/test_flux_kontext_expansion.py
+14 -11
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Comprehensive tests of diffusion features that are available in online serving mode
and are supported by the FluxKontext model.
@@ -7,7 +10,7 @@ import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.media import generate_synthetic_image
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -32,7 +35,7 @@ def _get_diffusion_feature_cases(model: str):
_get_diffusion_feature_cases(MODEL),
indirect=True,
)
def test_flux_kontext_text_to_image(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_flux_kontext_text_to_image(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test text-to-image generation with FluxKontext in regular end-user scenarios."""
messages = dummy_messages_from_mix_data(content_text="A photo of a cat sitting on a laptop keyboard")
@@ -47,7 +50,7 @@ def test_flux_kontext_text_to_image(omni_server: OmniServer, openai_client: Open
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.parametrize(
@@ -55,7 +58,7 @@ def test_flux_kontext_text_to_image(omni_server: OmniServer, openai_client: Open
_get_diffusion_feature_cases(MODEL),
indirect=True,
)
def test_flux_kontext_image_edit(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_flux_kontext_image_edit(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test image editing with FluxKontext in regular end-user scenarios."""
image_data_url = f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}"
@@ -74,7 +77,7 @@ def test_flux_kontext_image_edit(omni_server: OmniServer, openai_client: OpenAIC
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.parametrize(
@@ -82,7 +85,7 @@ def test_flux_kontext_image_edit(omni_server: OmniServer, openai_client: OpenAIC
_get_diffusion_feature_cases(MODEL),
indirect=True,
)
def test_flux_kontext_image_edit_no_negative(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_flux_kontext_image_edit_no_negative(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test image editing with FluxKontext without negative prompt."""
image_data_url = f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}"
@@ -99,7 +102,7 @@ def test_flux_kontext_image_edit_no_negative(omni_server: OmniServer, openai_cli
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.parametrize(
@@ -107,7 +110,7 @@ def test_flux_kontext_image_edit_no_negative(omni_server: OmniServer, openai_cli
_get_diffusion_feature_cases(MODEL),
indirect=True,
)
def test_flux_kontext_high_resolution(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_flux_kontext_high_resolution(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test high-resolution generation with FluxKontext."""
messages = dummy_messages_from_mix_data(content_text="A beautiful landscape with mountains and a lake")
@@ -122,7 +125,7 @@ def test_flux_kontext_high_resolution(omni_server: OmniServer, openai_client: Op
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.parametrize(
@@ -130,7 +133,7 @@ def test_flux_kontext_high_resolution(omni_server: OmniServer, openai_client: Op
_get_diffusion_feature_cases(MODEL),
indirect=True,
)
def test_flux_kontext_multiple_outputs(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_flux_kontext_multiple_outputs(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test generating multiple outputs with FluxKontext."""
messages = dummy_messages_from_mix_data(content_text="A photo of a cat sitting on a laptop keyboard")
@@ -146,4 +149,4 @@ def test_flux_kontext_multiple_outputs(omni_server: OmniServer, openai_client: O
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_glm_image_expansion.py
+6 -3
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Comprehensive tests of diffusion features that are available in online serving mode
and are supported by the following model:
@@ -25,7 +28,7 @@ from tests.helpers.media import generate_synthetic_image
from tests.helpers.runtime import (
OmniServer,
OmniServerParams,
OpenAIClientHandler,
OnlineOmniClient,
dummy_messages_from_mix_data,
)
from tests.helpers.stage_config import get_deploy_config_path, modify_stage_config
@@ -152,7 +155,7 @@ MODES = ["t2i", "i2i"]
def test_glm_image(
omni_server: OmniServer,
mode: str,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
):
"""GLM-Image diffusion feature coverage for t2i and i2i on two H100s."""
@@ -180,4 +183,4 @@ def test_glm_image(
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_glm_tts_expansion.py
+9 -9
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E expansion tests for GLM-TTS online serving (nightly CI).
@@ -14,7 +14,7 @@ os.environ["VLLM_TEST_CLEAN_GPU_MEMORY"] = "0"
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServerParams
from tests.helpers.stage_config import get_deploy_config_path
@@ -24,7 +24,7 @@ MODEL = os.environ.get("GLM_TTS_MODEL_PATH", "zai-org/GLM-TTS")
REF_TEXT = "他当时还跟线下其他的站姐吵架，然后，打架进局子了。"
DEPLOY_CONFIG = get_deploy_config_path("glm_tts.yaml")
REF_AUDIO_URL = load_test_audio_data_url("glm_tts/jiayan_zh.wav")
REF_AUDIO_URL = get_asset_path("glm_tts/jiayan_zh.wav", as_data_url=True)
SYNC_EXTRA_ARGS = [
"--trust-remote-code",
@@ -64,7 +64,7 @@ tts_async_chunk_server_params = [
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_sync_server_params, indirect=True)
def test_voice_clone_zh_sync(omni_server, openai_client) -> None:
def test_voice_clone_zh_sync(omni_server, online_client) -> None:
"""
Test voice cloning TTS with Chinese text.
Deploy Setting: glm_tts.yaml with ``--no-async-chunk`` (sync two-stage)
@@ -80,12 +80,12 @@ def test_voice_clone_zh_sync(omni_server, openai_client) -> None:
"ref_audio": REF_AUDIO_URL,
"ref_text": REF_TEXT,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_async_chunk_server_params, indirect=True)
def test_voice_clone_zh_async_chunk(omni_server, openai_client) -> None:
def test_voice_clone_zh_async_chunk(omni_server, online_client) -> None:
"""
Test voice cloning TTS with Chinese text via async_chunk streaming.
Deploy Setting: glm_tts.yaml default ``async_chunk: true``
@@ -102,12 +102,12 @@ def test_voice_clone_zh_async_chunk(omni_server, openai_client) -> None:
"ref_audio": REF_AUDIO_URL,
"ref_text": REF_TEXT,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_sync_server_params, indirect=True)
def test_models_endpoint(omni_server, openai_client) -> None:
def test_models_endpoint(omni_server, online_client) -> None:
"""Test the /v1/models endpoint returns loaded model."""
models = openai_client.client.models.list()
models = online_client.client.models.list()
assert len(models.data) > 0
tests/e2e/online_serving/test_hidream_i1_full_expansion.py
+6 -3
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
End-to-end diffusion coverage for HiDream-I1-Full in online serving mode.
@@ -12,7 +15,7 @@ image resolution.
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -49,7 +52,7 @@ def _get_hidream_i1_image_feature_cases(model: str):
@pytest.mark.skip(reason="https://github.com/vllm-project/vllm-omni/issues/4662")
def test_hidream_i1_image(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
):
"""Validate HiDream-I1-Full online serving with CPU offload."""
@@ -67,4 +70,4 @@ def test_hidream_i1_image(
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_higgs_audio_v2_expansion.py
+18 -120
View File
@@ -1,17 +1,13 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online expansion tests for higgs-audio v2 against /v1/audio/speech.
v1 scope is plain text -> 24 kHz speech plus shallow voice clone via
ref_audio + ref_text (inline) or voice=<name> (after POST /v1/audio/voices).
The model-aware request validator
(vllm_omni/entrypoints/openai/tts_adapters/higgs_audio_v2::validate)
rejects multi-speaker tags, language overrides, task_type, and bare
voice=<name> for names that do not match an uploaded speaker — this suite
exercises both the happy path (plain text in, audio bytes out) and the
validator rejections, so a regression that loosens the schema will fail
loudly.
Model-aware validator rejections live in L1
``tests/entrypoints/openai_api/test_tts_adapter.py``
(``test_higgs_audio_v2_validate_*``).
"""
from __future__ import annotations
@@ -27,7 +23,7 @@ os.environ.setdefault("VLLM_MOE_USE_DEEP_GEMM", "0")
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServerParams
from tests.helpers.stage_config import get_deploy_config_path
@@ -63,7 +59,7 @@ _MIN_AUDIO_BYTES = 20_000
# speech) + its transcript. See tests/e2e/online_serving/test_qwen3_tts_base.py
# for the asset rationale — keeping a single shared reference clip across TTS
# tests avoids duplicating WAVs in the repo.
_REF_AUDIO_URL = load_test_audio_data_url("qwen3_tts/clone_2.wav")
_REF_AUDIO_URL = get_asset_path("qwen3_tts/clone_2.wav", as_data_url=True)
_REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
@@ -72,9 +68,9 @@ class TestHiggsAudioV2OnlineHappyPath:
"""Plain-text -> audio happy paths against the live HTTP server."""
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_plain_text_wav(self, omni_server, openai_client) -> None:
def test_plain_text_wav(self, omni_server, online_client) -> None:
"""Single non-streaming WAV request — covers the canonical TTS happy path."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hello world.",
@@ -86,7 +82,7 @@ class TestHiggsAudioV2OnlineHappyPath:
)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_plain_text_pcm_streaming(self, omni_server, openai_client) -> None:
def test_plain_text_pcm_streaming(self, omni_server, online_client) -> None:
"""Streaming PCM via the shared-memory connector's codec_streaming path.
higgs_audio_v2.yaml pins ``codec_streaming: true`` + ``async_chunk: false``,
@@ -102,7 +98,7 @@ class TestHiggsAudioV2OnlineHappyPath:
content correctness is separately guarded by the full_model transcript
check.
"""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Streaming the quick brown fox over the lazy dog.",
@@ -116,9 +112,9 @@ class TestHiggsAudioV2OnlineHappyPath:
)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_plain_text_with_max_new_tokens(self, omni_server, openai_client) -> None:
def test_plain_text_with_max_new_tokens(self, omni_server, online_client) -> None:
"""max_new_tokens is one of the few extra fields the higgs validator accepts."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Innovation distinguishes between a leader and a follower.",
@@ -131,9 +127,9 @@ class TestHiggsAudioV2OnlineHappyPath:
)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_concurrent_plain_text(self, omni_server, openai_client) -> None:
def test_concurrent_plain_text(self, omni_server, online_client) -> None:
"""Three concurrent non-streaming requests — guards the per-slot audio state."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "It was the night before my birthday.",
@@ -146,104 +142,6 @@ class TestHiggsAudioV2OnlineHappyPath:
)
# ---------------------------------------------------------------------------
# Validator rejections served over HTTP. Each case targets one out-of-scope
# field; the validator returns 4xx via the OpenAI error path. We keep the
# error-message substring loose so a phrasing tweak in the validator does not
# break CI, but tight enough to catch a regression that silently accepts the
# field.
# ---------------------------------------------------------------------------
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
class TestHiggsAudioV2OnlineValidatorRejections:
"""Out-of-scope fields must come back as 4xx with a higgs-named message."""
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_rejects_ref_audio_without_ref_text(self, omni_server, openai_client) -> None:
"""Voice clone needs the transcript too — half-supplied is a 4xx."""
openai_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hello world.",
"ref_audio": _REF_AUDIO_URL,
"response_format": "wav",
"timeout": DEFAULT_SPEECH_TIMEOUT_S,
"status_code": (400, 422),
"err_message": "ref_text",
}
)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_rejects_ref_text_without_ref_audio(self, omni_server, openai_client) -> None:
"""Symmetric guard: ref_text alone is not enough — must come with ref_audio."""
openai_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hello world.",
"ref_text": "some transcript",
"response_format": "wav",
"timeout": DEFAULT_SPEECH_TIMEOUT_S,
"status_code": (400, 422),
"err_message": "ref_audio",
}
)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_rejects_task_type(self, omni_server, openai_client) -> None:
openai_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hello world.",
"task_type": "Base",
"response_format": "wav",
"timeout": DEFAULT_SPEECH_TIMEOUT_S,
"status_code": (400, 422),
"err_message": "task_type",
}
)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_rejects_language_override(self, omni_server, openai_client) -> None:
openai_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hello world.",
"language": "Chinese",
"response_format": "wav",
"timeout": DEFAULT_SPEECH_TIMEOUT_S,
"status_code": (400, 422),
"err_message": "language",
}
)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_rejects_multi_speaker_tag_in_text(self, omni_server, openai_client) -> None:
openai_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "[SPEAKER0] hi",
"response_format": "wav",
"timeout": DEFAULT_SPEECH_TIMEOUT_S,
"status_code": (400, 422),
"err_message": "multi-speaker",
}
)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_rejects_empty_input(self, omni_server, openai_client) -> None:
openai_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": " ",
"response_format": "wav",
"timeout": DEFAULT_SPEECH_TIMEOUT_S,
"status_code": (400, 422),
"err_message": "empty",
}
)
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
class TestHiggsAudioV2OnlineVoiceClone:
"""Shallow voice clone: ref_audio + ref_text -> speech in the cloned voice.
@@ -256,9 +154,9 @@ class TestHiggsAudioV2OnlineVoiceClone:
"""
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_voice_clone_basic(self, omni_server, openai_client) -> None:
def test_voice_clone_basic(self, omni_server, online_client) -> None:
"""Single non-streaming WAV request driven by the qwen3_tts ref clip."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hello world.",
@@ -272,9 +170,9 @@ class TestHiggsAudioV2OnlineVoiceClone:
)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_voice_clone_pcm_streaming(self, omni_server, openai_client) -> None:
def test_voice_clone_pcm_streaming(self, omni_server, online_client) -> None:
"""Voice clone over the streaming PCM path."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Innovation distinguishes a leader from a follower.",
tests/e2e/online_serving/test_higgs_audio_v3.py
+27 -27
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""End-to-end online tests for higgs-audio v3 against /v1/audio/speech.
Mirrors the higgs_audio_v2 test layout. Covers the plain-text-in / audio-out
@@ -21,7 +21,7 @@ os.environ.setdefault("VLLM_MOE_USE_DEEP_GEMM", "0")
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServerParams
from tests.helpers.stage_config import get_deploy_config_path
@@ -55,7 +55,7 @@ _SKIP_ISSUE_6418 = pytest.mark.skip(
# Reuse the shared TTS reference clip (clean ~5 s 24 kHz mono human speech)
# vendored under tests/assets/qwen3_tts/. Keeps a single WAV across TTS
# tests rather than duplicating asset bytes.
_REF_AUDIO_URL = load_test_audio_data_url("qwen3_tts/clone_2.wav")
_REF_AUDIO_URL = get_asset_path("qwen3_tts/clone_2.wav", as_data_url=True)
_REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
@@ -66,9 +66,9 @@ class TestHiggsAudioV3OnlineHappyPath:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_plain_text_wav(self, omni_server, openai_client) -> None:
def test_plain_text_wav(self, omni_server, online_client) -> None:
"""Single non-streaming WAV request - canonical TTS happy path."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hello world.",
@@ -86,9 +86,9 @@ class TestHiggsAudioV3OnlineHappyPath:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_plain_text_with_max_new_tokens(self, omni_server, openai_client) -> None:
def test_plain_text_with_max_new_tokens(self, omni_server, online_client) -> None:
"""``max_new_tokens`` is one of the few extra fields the v3 validator accepts."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Innovation distinguishes between a leader and a follower.",
@@ -104,9 +104,9 @@ class TestHiggsAudioV3OnlineHappyPath:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_concurrent_plain_text(self, omni_server, openai_client) -> None:
def test_concurrent_plain_text(self, omni_server, online_client) -> None:
"""Three concurrent non-streaming requests - guards per-slot audio state and Stage-0 PC under batching."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "it was the night before my birthday",
@@ -120,7 +120,7 @@ class TestHiggsAudioV3OnlineHappyPath:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_plain_text_pcm_streaming(self, omni_server, openai_client) -> None:
def test_plain_text_pcm_streaming(self, omni_server, online_client) -> None:
"""Streaming PCM via talker2code2wav_async_chunk + the connector's codec_streaming path.
The deploy YAML pins ``async_chunk: true`` and ``codec_streaming: true`` so chunks
@@ -136,7 +136,7 @@ class TestHiggsAudioV3OnlineHappyPath:
~0.16 dB margin over measured single-request output (1.16 dB on L4), so
flake risk is too high there.
"""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "The quick brown fox jumps over the lazy dog.",
@@ -151,7 +151,7 @@ class TestHiggsAudioV3OnlineHappyPath:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_concurrent_pcm_streaming(self, omni_server, openai_client) -> None:
def test_concurrent_pcm_streaming(self, omni_server, online_client) -> None:
"""Three concurrent streaming requests - guards per-request frame cursors
in ``talker2code2wav_async_chunk`` and per-slot delay-pattern state under batched AR.
@@ -161,7 +161,7 @@ class TestHiggsAudioV3OnlineHappyPath:
over the measured worst-of-3 on L4 (-0.42 dB) while still well above the
catastrophic-failure region.
"""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "She sells seashells by the seashore.",
@@ -194,9 +194,9 @@ class TestHiggsAudioV3OnlineInlineControlTokens:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_inline_emotion_and_expressive(self, omni_server, openai_client) -> None:
def test_inline_emotion_and_expressive(self, omni_server, online_client) -> None:
"""Delivery tokens (emotion + expressive_high) at the start of input."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": ("<|emotion:amusement|><|prosody:expressive_high|>Wait, that was actually hilarious."),
@@ -211,9 +211,9 @@ class TestHiggsAudioV3OnlineInlineControlTokens:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_inline_style_whispering(self, omni_server, openai_client) -> None:
def test_inline_style_whispering(self, omni_server, online_client) -> None:
"""Style token at the start - ``whispering``."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "<|style:whispering|>It is just between you and me, alright?",
@@ -228,9 +228,9 @@ class TestHiggsAudioV3OnlineInlineControlTokens:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_inline_prosody_speed_and_pitch(self, omni_server, openai_client) -> None:
def test_inline_prosody_speed_and_pitch(self, omni_server, online_client) -> None:
"""Two prosody tokens at the start - slow speed plus low pitch."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": (
@@ -247,9 +247,9 @@ class TestHiggsAudioV3OnlineInlineControlTokens:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_inline_pause_mid_text(self, omni_server, openai_client) -> None:
def test_inline_pause_mid_text(self, omni_server, online_client) -> None:
"""Positional pause token placed inline between two clauses."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hold on a moment <|prosody:pause|> let me think about it.",
@@ -264,9 +264,9 @@ class TestHiggsAudioV3OnlineInlineControlTokens:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_inline_sfx_with_onomatopoeia(self, omni_server, openai_client) -> None:
def test_inline_sfx_with_onomatopoeia(self, omni_server, online_client) -> None:
"""SFX token paired with its written onomatopoeia (``<|sfx:laughter|>Hehe``)."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": (
@@ -288,9 +288,9 @@ class TestHiggsAudioV3OnlineVoiceClone:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_voice_clone_ref_audio_ref_text(self, omni_server, openai_client) -> None:
def test_voice_clone_ref_audio_ref_text(self, omni_server, online_client) -> None:
"""Canonical vllm-omni voice clone via ``ref_audio`` + ``ref_text``."""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hello world.",
@@ -305,14 +305,14 @@ class TestHiggsAudioV3OnlineVoiceClone:
@pytest.mark.slow
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_voice_clone_references_alias(self, omni_server, openai_client) -> None:
def test_voice_clone_references_alias(self, omni_server, online_client) -> None:
"""BosonAI cookbook payload: ``references=[{audio_path, text}]``.
``normalize_references_alias`` (``protocol/audio.py``) is supposed to
translate the cookbook field into ``ref_audio`` / ``ref_text``; without
it the request would silently fall through to zero-shot synthesis.
"""
openai_client.send_audio_speech_request(
online_client.send_audio_speech_request(
{
"model": omni_server.model,
"input": "Hello world.",
tests/e2e/online_serving/test_hunyuan_video_15_expansion.py
+4 -4
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Tests of common diffusion feature combinations in online serving mode
for HunyuanVideo-1.5-T2V (480p).
@@ -12,7 +12,7 @@ Coverage (H100, since model cannot fit L4):
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -82,7 +82,7 @@ def _get_diffusion_feature_cases(model: str):
)
def test_hunyuan_video_15_t2v(
omni_server: OmniServer,
openai_client: OpenAIClientHandler,
online_client: OnlineOmniClient,
):
"""L4 diffusion feature coverage for HunyuanVideo-1.5-T2V on H100."""
form_data = {
@@ -101,4 +101,4 @@ def test_hunyuan_video_15_t2v(
"form_data": form_data,
}
openai_client.send_video_diffusion_request(request_config)
online_client.send_video_diffusion_request(request_config)
tests/e2e/online_serving/test_indextts2_expansion.py
+7 -7
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""E2E online serving tests for IndexTTS2 via /v1/audio/speech endpoint.
Two-stage pipeline: GPT AR → S2Mel + BigVGAN. Output is 22050 Hz mono WAV.
@@ -15,7 +15,7 @@ os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServerParams
from tests.helpers.stage_config import get_deploy_config_path
@@ -23,7 +23,7 @@ pytestmark = [pytest.mark.slow, pytest.mark.tts]
MODEL = "IndexTeam/IndexTTS-2"
REF_AUDIO_URL = load_test_audio_data_url("indextts2/ref_audio.wav")
REF_AUDIO_URL = get_asset_path("indextts2/ref_audio.wav", as_data_url=True)
tts_server_params = [
pytest.param(
@@ -39,7 +39,7 @@ tts_server_params = [
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_basic_english(omni_server, openai_client) -> None:
def test_basic_english(omni_server, online_client) -> None:
"""
Basic English TTS: text + ref_audio, no emotion.
Deploy Setting: default yaml
@@ -55,12 +55,12 @@ def test_basic_english(omni_server, openai_client) -> None:
"ref_audio": REF_AUDIO_URL,
"min_audio_bytes": 1024,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_streaming(omni_server, openai_client) -> None:
def test_streaming(omni_server, online_client) -> None:
"""
stream=True compatibility: IndexTTS2 itself is non-async-chunking, but the
OpenAI speech endpoint should still accept the streaming request path and
@@ -80,4 +80,4 @@ def test_streaming(omni_server, openai_client) -> None:
"ref_audio": REF_AUDIO_URL,
"min_audio_bytes": 1024,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
tests/e2e/online_serving/test_krea2_expansion.py
+4 -4
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""L4 online-serving tests for the Krea 2 text-to-image diffusion pipeline.
@@ -29,7 +29,7 @@ import requests
from PIL import Image
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -76,7 +76,7 @@ def _get_diffusion_feature_cases(model: str):
_get_diffusion_feature_cases(MODEL),
indirect=True,
)
def test_krea2(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_krea2(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Serve Krea 2 and validate the OpenAI-compatible image-generation route.
guidance_scale resolution is checkpoint-aware inside the pipeline (distilled -> no-CFG,
@@ -96,7 +96,7 @@ def test_krea2(omni_server: OmniServer, openai_client: OpenAIClientHandler):
"seed": 42,
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
def _basic_lora_payload(lora: dict | None = None) -> dict:
tests/e2e/online_serving/test_lance.py
+5 -5
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""End-to-end online serving test for Lance (single-stage).
Verifies that the Lance pipeline can serve text-to-image and image-edit
@@ -100,7 +100,7 @@ def _build_img2img_messages(prompt: str, image_b64: str) -> list[dict]:
@pytest.mark.diffusion
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_lance_text2img_online(omni_server, openai_client) -> None:
def test_lance_text2img_online(omni_server, online_client) -> None:
"""Lance text2img via the OpenAI-compatible chat completions API."""
request_config = {
"model": omni_server.model,
@@ -115,7 +115,7 @@ def test_lance_text2img_online(omni_server, openai_client) -> None:
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.core_model
@@ -123,7 +123,7 @@ def test_lance_text2img_online(omni_server, openai_client) -> None:
@pytest.mark.diffusion
@hardware_test(res={"cuda": "H100"})
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_lance_img2img_online(omni_server, openai_client) -> None:
def test_lance_img2img_online(omni_server, online_client) -> None:
"""Lance image_edit via the OpenAI-compatible chat completions API."""
input_image = ImageAsset("2560px-Gfp-wisconsin-madison-the-nature-boardwalk").pil_image.convert("RGB")
buffer = BytesIO()
@@ -141,4 +141,4 @@ def test_lance_img2img_online(omni_server, openai_client) -> None:
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_longcat_image_edit_expansion.py
+6 -3
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Recommended tests of diffusion features that are available in online serving mode
and are supported by the following model:
@@ -16,7 +19,7 @@ import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.media import generate_synthetic_image
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -66,7 +69,7 @@ def _get_diffusion_feature_cases(model: str):
_get_diffusion_feature_cases(MODEL),
indirect=True,
)
def test_longcat_image_edit(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_longcat_image_edit(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test the recommended feature combinations for LongCat-Image-Edit."""
image_data_url = f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}"
messages = dummy_messages_from_mix_data(image_data_url=image_data_url, content_text=EDIT_PROMPT)
@@ -84,4 +87,4 @@ def test_longcat_image_edit(omni_server: OmniServer, openai_client: OpenAIClient
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_longcat_image_expansion.py
+6 -3
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Recommended tests of diffusion features that are available in online serving mode
and are supported by the following model:
@@ -15,7 +18,7 @@ This validates:
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -66,7 +69,7 @@ def _get_diffusion_feature_cases(model: str):
_get_diffusion_feature_cases(MODEL),
indirect=True,
)
def test_longcat_image(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_longcat_image(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test the recommended feature combinations for LongCat-Image."""
messages = dummy_messages_from_mix_data(content_text=TEXT_TO_IMAGE_PROMPT)
@@ -83,4 +86,4 @@ def test_longcat_image(omni_server: OmniServer, openai_client: OpenAIClientHandl
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_mimo_audio_expansion.py
+5 -5
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E expansion tests for MiMo-Audio online serving (nightly Omni · Function Test with H100).
"""
@@ -77,7 +77,7 @@ def get_max_batch_size(size_type="few"):
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=1)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_to_text_001(omni_server, openai_client) -> None:
def test_text_to_text_001(omni_server, online_client) -> None:
"""
Test text input processing and text-only output generation via OpenAI API.
Deploy Setting: default yaml
@@ -95,13 +95,13 @@ def test_text_to_text_001(omni_server, openai_client) -> None:
"key_words": {"text": ["beijing"]},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=1)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
@pytest.mark.skip(reason="CI failed 8571")
def test_audio_to_text_audio_001(omni_server, openai_client) -> None:
def test_audio_to_text_audio_001(omni_server, online_client) -> None:
"""
Test audio and text input processing and text/audio output generation via OpenAI API.
Deploy Setting: default yaml
@@ -126,4 +126,4 @@ def test_audio_to_text_audio_001(omni_server, openai_client) -> None:
},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
tests/e2e/online_serving/test_ming_flash_omni_expansion.py
+13 -13
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E online serving tests for Ming-flash-omni-2.0 model (Thinker stage).
Tests multimodal understanding via OpenAI-compatible API.
@@ -70,7 +70,7 @@ def get_max_batch_size(size_type="few"):
@_SKIP_NEED_4_H100_NOT_CI
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_to_text_001(omni_server, openai_client) -> None:
def test_text_to_text_001(omni_server, online_client) -> None:
"""
Input Modal: text
Output Modal: text
@@ -90,13 +90,13 @@ def test_text_to_text_001(omni_server, openai_client) -> None:
"key_words": {"text": ["beijing"]},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@_SKIP_NEED_4_H100_NOT_CI
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_to_text_stream_001(omni_server, openai_client) -> None:
def test_text_to_text_stream_001(omni_server, online_client) -> None:
"""
Input Modal: text
Output Modal: text
@@ -116,13 +116,13 @@ def test_text_to_text_stream_001(omni_server, openai_client) -> None:
"key_words": {"text": ["beijing"]},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
@_SKIP_NEED_4_H100_NOT_CI
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_image_to_text_001(omni_server, openai_client) -> None:
def test_image_to_text_001(omni_server, online_client) -> None:
"""
Input Modal: image + text
Output Modal: text
@@ -143,13 +143,13 @@ def test_image_to_text_001(omni_server, openai_client) -> None:
"modalities": ["text"],
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@_SKIP_NEED_4_H100_NOT_CI
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_audio_to_text_001(omni_server, openai_client) -> None:
def test_audio_to_text_001(omni_server, online_client) -> None:
"""
Input Modal: audio + text
Output Modal: text
@@ -170,13 +170,13 @@ def test_audio_to_text_001(omni_server, openai_client) -> None:
"modalities": ["text"],
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@_SKIP_NEED_4_H100_NOT_CI
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_video_to_text_001(omni_server, openai_client) -> None:
def test_video_to_text_001(omni_server, online_client) -> None:
"""
Input Modal: video + text
Output Modal: text
@@ -197,13 +197,13 @@ def test_video_to_text_001(omni_server, openai_client) -> None:
"modalities": ["text"],
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@_SKIP_NEED_4_H100_NOT_CI
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_mix_to_text_001(omni_server, openai_client) -> None:
def test_mix_to_text_001(omni_server, online_client) -> None:
"""
Input Modal: text + audio + image + video
Output Modal: text
@@ -228,4 +228,4 @@ def test_mix_to_text_001(omni_server, openai_client) -> None:
"modalities": ["text"],
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
tests/e2e/online_serving/test_ming_tts_expansion.py
+5 -5
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E expansion tests for Ming-omni-tts online serving (nightly CI).
@@ -56,7 +56,7 @@ def get_prompt(prompt_type="zh"):
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", no_async_chunk_params, indirect=True)
def test_text_to_audio_non_streaming_001(omni_server, openai_client) -> None:
def test_text_to_audio_non_streaming_001(omni_server, online_client) -> None:
"""
Deploy Setting: ming_tts.yaml with --no-async-chunk
Input Modal: text
@@ -71,12 +71,12 @@ def test_text_to_audio_non_streaming_001(omni_server, openai_client) -> None:
"response_format": "wav",
"timeout": 300.0,
}
openai_client.send_audio_speech_request(request_config, request_num=2)
online_client.send_audio_speech_request(request_config, request_num=2)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", async_chunk_params, indirect=True)
def test_text_to_audio_streaming_001(omni_server, openai_client) -> None:
def test_text_to_audio_streaming_001(omni_server, online_client) -> None:
"""
Deploy Setting: ming_tts.yaml (async_chunk=true)
Input Modal: text + voice
@@ -93,4 +93,4 @@ def test_text_to_audio_streaming_001(omni_server, openai_client) -> None:
"response_format": "wav",
"timeout": 300.0,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
tests/e2e/online_serving/test_minicpmo_4_5_duplex.py
+9 -6
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""CI coverage for the MiniCPM-o 4.5 native-duplex Realtime API."""
from __future__ import annotations
@@ -107,8 +110,8 @@ async def _run_protocol_smoke(*, url: str, model: str, ref_audio: Path) -> list[
@pytest.mark.core_model
@hardware_test(res={"cuda": "H100", "npu": "A3"}, num_cards=1)
@pytest.mark.parametrize("omni_server", SERVER_PARAMS, indirect=True)
def test_duplex_websocket_protocol_smoke(omni_server, model_prefix: str) -> None:
ref_audio = resolve_ref_audio(model_prefix)
def test_duplex_websocket_protocol_smoke(omni_server) -> None:
ref_audio = resolve_ref_audio()
events = asyncio.run(
_run_protocol_smoke(
url=realtime_url(omni_server),
@@ -125,11 +128,11 @@ def test_duplex_websocket_protocol_smoke(omni_server, model_prefix: str) -> None
@pytest.mark.advanced_model
@hardware_test(res={"cuda": "H100", "npu": "A3"}, num_cards=1)
@pytest.mark.parametrize("omni_server", SERVER_PARAMS, indirect=True)
def test_duplex_single_session_response_required(omni_server, model_prefix: str, tmp_path: Path) -> None:
def test_duplex_single_session_response_required(omni_server, tmp_path: Path) -> None:
args = demo_args(
omni_server=omni_server,
input_wav=validated_input_wav(),
ref_audio=resolve_ref_audio(model_prefix),
ref_audio=resolve_ref_audio(),
output_dir=tmp_path / "single_session",
)
args.turns = 2
@@ -151,13 +154,13 @@ def test_duplex_single_session_response_required(omni_server, model_prefix: str,
@pytest.mark.advanced_model
@hardware_test(res={"cuda": "H100", "npu": "A3"}, num_cards=1)
@pytest.mark.parametrize("omni_server", SERVER_PARAMS, indirect=True)
def test_duplex_two_sessions_resume_and_takeover(omni_server, model_prefix: str, tmp_path: Path) -> None:
def test_duplex_two_sessions_resume_and_takeover(omni_server, tmp_path: Path) -> None:
result = asyncio.run(
run_multi_session(
multi_session_args(
omni_server=omni_server,
input_wav=validated_input_wav(),
ref_audio=resolve_ref_audio(model_prefix),
ref_audio=resolve_ref_audio(),
output_dir=tmp_path / "multi_session",
response_required=True,
)
tests/e2e/online_serving/test_minicpmo_4_5_duplex_expansion.py
+7 -4
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Nightly lifecycle coverage for the MiniCPM-o 4.5 native-duplex API."""
from __future__ import annotations
@@ -31,11 +34,11 @@ pytestmark = [pytest.mark.full_model, pytest.mark.omni]
@hardware_test(res={"cuda": "H100", "npu": "A3"}, num_cards=1)
@pytest.mark.parametrize("omni_server", SERVER_PARAMS, indirect=True)
def test_duplex_admission_and_expiry_reaper(omni_server, model_prefix: str, tmp_path: Path) -> None:
def test_duplex_admission_and_expiry_reaper(omni_server, tmp_path: Path) -> None:
args = multi_session_args(
omni_server=omni_server,
input_wav=validated_input_wav(),
ref_audio=resolve_ref_audio(model_prefix),
ref_audio=resolve_ref_audio(),
output_dir=tmp_path / "admission_expiry",
response_required=False,
)
@@ -59,7 +62,7 @@ def test_duplex_admission_and_expiry_reaper(omni_server, model_prefix: str, tmp_
# so it never reaches a mid-stream decision point.
@hardware_test(res={"cuda": "H100"}, num_cards=1)
@pytest.mark.parametrize("omni_server", SERVER_PARAMS, indirect=True)
def test_duplex_soft_interrupt(omni_server, model_prefix: str, tmp_path: Path) -> None:
def test_duplex_soft_interrupt(omni_server, tmp_path: Path) -> None:
input_wav = validated_soft_interrupt_wav()
result = asyncio.run(
run_soft_interrupt(
@@ -67,7 +70,7 @@ def test_duplex_soft_interrupt(omni_server, model_prefix: str, tmp_path: Path) -
url=realtime_url(omni_server),
model=omni_server.model,
input_wav=str(input_wav),
ref_audio=str(resolve_ref_audio(model_prefix)),
ref_audio=str(resolve_ref_audio()),
output_dir=str(tmp_path / "soft_interrupt"),
summary_output=None,
chunk_ms=200,
tests/e2e/online_serving/test_moss_tts_expansion.py
+7 -7
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E online tests for the full MOSS-TTS family via /v1/audio/speech.
@@ -96,7 +96,7 @@ tts_server_params = [
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_001(omni_server, openai_client, ref_audio_data_url) -> None:
def test_text_to_audio_001(omni_server, online_client, ref_audio_data_url) -> None:
"""
Realtime voice_clone via /v1/audio/speech, non-streaming.
Deploy Setting: moss_tts_realtime.yaml
@@ -118,12 +118,12 @@ def test_text_to_audio_001(omni_server, openai_client, ref_audio_data_url) -> No
"min_audio_bytes": _MIN_AUDIO_BYTES,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_002_streaming(omni_server, openai_client, ref_audio_data_url) -> None:
def test_text_to_audio_002_streaming(omni_server, online_client, ref_audio_data_url) -> None:
"""
Realtime voice_clone via /v1/audio/speech, streaming PCM.
Deploy Setting: moss_tts_realtime.yaml
@@ -147,12 +147,12 @@ def test_text_to_audio_002_streaming(omni_server, openai_client, ref_audio_data_
"min_hnr_db": -5.0,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_003_chinese(omni_server, openai_client, ref_audio_data_url) -> None:
def test_text_to_audio_003_chinese(omni_server, online_client, ref_audio_data_url) -> None:
"""
Realtime voice_clone via /v1/audio/speech, Chinese input.
Deploy Setting: moss_tts_realtime.yaml
@@ -172,4 +172,4 @@ def test_text_to_audio_003_chinese(omni_server, openai_client, ref_audio_data_ur
"min_audio_bytes": _MIN_AUDIO_BYTES,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
tests/e2e/online_serving/test_moss_tts_nano_expansion.py
+9 -9
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for MOSS-TTS-Nano model with text input and audio output.
@@ -86,7 +86,7 @@ tts_server_params = [
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_001(omni_server, openai_client, ref_audio_data_url) -> None:
def test_text_to_audio_001(omni_server, online_client, ref_audio_data_url) -> None:
"""
Test voice_clone mode (no ref_text) via /v1/audio/speech.
Deploy Setting: default yaml
@@ -103,12 +103,12 @@ def test_text_to_audio_001(omni_server, openai_client, ref_audio_data_url) -> No
"ref_audio": ref_audio_data_url,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_002(omni_server, openai_client, ref_audio_data_url) -> None:
def test_text_to_audio_002(omni_server, online_client, ref_audio_data_url) -> None:
"""
Test streaming voice_clone via /v1/audio/speech.
Deploy Setting: default yaml
@@ -135,12 +135,12 @@ def test_text_to_audio_002(omni_server, openai_client, ref_audio_data_url) -> No
"min_hnr_db": -5.0,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_003(omni_server, openai_client, ref_audio_data_url) -> None:
def test_text_to_audio_003(omni_server, online_client, ref_audio_data_url) -> None:
"""
Test Chinese voice_clone via /v1/audio/speech.
Deploy Setting: default yaml
@@ -157,12 +157,12 @@ def test_text_to_audio_003(omni_server, openai_client, ref_audio_data_url) -> No
"ref_audio": ref_audio_data_url,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_004_ref_text_ignored(omni_server, openai_client, ref_audio_data_url) -> None:
def test_text_to_audio_004_ref_text_ignored(omni_server, online_client, ref_audio_data_url) -> None:
"""
Sending ``ref_text`` alongside ``ref_audio`` is accepted but ignored.
@@ -180,4 +180,4 @@ def test_text_to_audio_004_ref_text_ignored(omni_server, openai_client, ref_audi
"ref_text": REF_AUDIO_TRANSCRIPT,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
tests/e2e/online_serving/test_nextstep_expansion.py
+6 -3
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Online serving E2E for NextStep-1.1 text-to-image (tensor parallel).
"""
@@ -10,7 +13,7 @@ from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import (
OmniServer,
OmniServerParams,
OpenAIClientHandler,
OnlineOmniClient,
dummy_messages_from_mix_data,
)
@@ -66,7 +69,7 @@ def _get_diffusion_feature_cases(model: str):
_get_diffusion_feature_cases(model=os.environ.get("VLLM_TEST_NEXTSTEP_MODEL", _DEFAULT_MODEL)),
indirect=True,
)
def test_nextstep_11(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_nextstep_11(omni_server: OmniServer, online_client: OnlineOmniClient):
messages = dummy_messages_from_mix_data(content_text=POSITIVE_PROMPT)
request_config = {
"model": omni_server.model,
@@ -82,4 +85,4 @@ def test_nextstep_11(omni_server: OmniServer, openai_client: OpenAIClientHandler
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_omnivoice_expansion.py
+31 -45
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for OmniVoice TTS model via /v1/audio/speech endpoint.
@@ -14,7 +14,7 @@ os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import get_asset_path, load_test_audio_data_url
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServerParams
from tests.helpers.stage_config import get_deploy_config_path
from vllm_omni.entrypoints.openai.serving_speech import _DEFAULT_VOICE_NAME
@@ -47,7 +47,7 @@ TEST_PARAMS = [
_DEFAULT_MIN_AUDIO_BYTES = 5000
REF_AUDIO_URL = load_test_audio_data_url("qwen3_tts/clone_2.wav")
REF_AUDIO_URL = get_asset_path("qwen3_tts/clone_2.wav", as_data_url=True)
REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
@@ -63,7 +63,7 @@ class TestOmniVoiceTTS:
"""E2E tests for OmniVoice TTS model."""
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_speech_auto_voice(self, omni_server, openai_client) -> None:
def test_speech_auto_voice(self, omni_server, online_client) -> None:
"""Test auto voice TTS generation (text only, no reference audio)."""
request_config = {
"model": omni_server.model,
@@ -72,7 +72,7 @@ class TestOmniVoiceTTS:
"timeout": 180.0,
"min_audio_bytes": _DEFAULT_MIN_AUDIO_BYTES,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
@@ -80,7 +80,7 @@ class TestOmniVoiceSeed:
"""E2E tests for OmniVoice Seed Params."""
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_speech_auto_voice_seed_deterministic(self, omni_server, openai_client) -> None:
def test_speech_auto_voice_seed_deterministic(self, omni_server, online_client) -> None:
cfg = {
"model": omni_server.model,
"input": get_prompt("text"),
@@ -89,12 +89,12 @@ class TestOmniVoiceSeed:
"min_audio_bytes": _DEFAULT_MIN_AUDIO_BYTES,
}
r1 = openai_client.send_audio_speech_request(cfg)[0]
r2 = openai_client.send_audio_speech_request(cfg)[0]
r1 = online_client.send_audio_speech_request(cfg)[0]
r2 = online_client.send_audio_speech_request(cfg)[0]
assert r1.audio_bytes == r2.audio_bytes
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_speech_auto_voice_seed_non_deterministic(self, omni_server, openai_client) -> None:
def test_speech_auto_voice_seed_non_deterministic(self, omni_server, online_client) -> None:
cfg = {
"model": omni_server.model,
"input": get_prompt("text"),
@@ -105,8 +105,8 @@ class TestOmniVoiceSeed:
cfg1 = {**cfg, "seed": 42}
cfg2 = {**cfg, "seed": 43}
r1 = openai_client.send_audio_speech_request(cfg1)[0]
r2 = openai_client.send_audio_speech_request(cfg2)[0]
r1 = online_client.send_audio_speech_request(cfg1)[0]
r2 = online_client.send_audio_speech_request(cfg2)[0]
assert r1.audio_bytes != r2.audio_bytes
@@ -116,7 +116,7 @@ class TestOmniVoiceVoiceCloning:
"""E2E tests for OmniVoice voice cloning functionality."""
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_voice_clone_ref_audio_only(self, omni_server, openai_client) -> None:
def test_voice_clone_ref_audio_only(self, omni_server, online_client) -> None:
"""Test voice cloning with ref_audio only (x_vector mode)."""
request_config = {
"model": omni_server.model,
@@ -126,10 +126,10 @@ class TestOmniVoiceVoiceCloning:
"timeout": 180.0,
"min_audio_bytes": _DEFAULT_MIN_AUDIO_BYTES,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
def test_voice_clone_ref_audio_and_text(self, omni_server, openai_client) -> None:
def test_voice_clone_ref_audio_and_text(self, omni_server, online_client) -> None:
"""Test voice cloning with ref_audio and ref_text (in-context mode)."""
request_config = {
"model": omni_server.model,
@@ -140,13 +140,13 @@ class TestOmniVoiceVoiceCloning:
"timeout": 180.0,
"min_audio_bytes": _DEFAULT_MIN_AUDIO_BYTES,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
@pytest.mark.parametrize("voice", [_DEFAULT_VOICE_NAME, {"id": _DEFAULT_VOICE_NAME}])
def test_speech_with_voice_param_accepted(omni_server, openai_client, voice) -> None:
def test_speech_with_voice_param_accepted(omni_server, online_client, voice) -> None:
"""The default voice should be accepted as both a plain string and a
VoiceID dict, even when the model has no uploaded speakers."""
request_config = {
@@ -157,26 +157,12 @@ def test_speech_with_voice_param_accepted(omni_server, openai_client, voice) ->
"timeout": 180.0,
"min_audio_bytes": _DEFAULT_MIN_AUDIO_BYTES,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
def test_unknown_voice_rejected(omni_server, openai_client) -> None:
"""An unrecognized voice name should be rejected with a 400."""
request_config = {
"model": omni_server.model,
"input": get_prompt("text"),
"voice": "nonexistent_voice",
"status_code": 400,
"err_message": "Invalid voice",
}
openai_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
def test_voice_request_before_and_after_clone_registration(omni_server, openai_client) -> None:
def test_voice_request_before_and_after_clone_registration(omni_server, online_client) -> None:
"""The default voice should always be listed and usable, even after
a different cloned voice is registered."""
speech_config = {
@@ -186,11 +172,11 @@ def test_voice_request_before_and_after_clone_registration(omni_server, openai_c
}
# 1) "default" should appear in the voices list before any uploads
voices = openai_client.send_audio_voices_list_http_request()[0]
voices = online_client.send_audio_voices_list_http_request()[0]
assert _DEFAULT_VOICE_NAME in voices.json_body["voices"]
# 2) Speech with the default voice should succeed
openai_client.send_audio_speech_request(speech_config)
online_client.send_audio_speech_request(speech_config)
# 3) Register a cloned voice under a different name
register_config = {
@@ -203,20 +189,20 @@ def test_voice_request_before_and_after_clone_registration(omni_server, openai_c
),
},
}
openai_client.send_audio_voices_create_http_request(register_config)
online_client.send_audio_voices_create_http_request(register_config)
# 4) "default" should still be listed alongside the new voice
voices = openai_client.send_audio_voices_list_http_request()[0]
voices = online_client.send_audio_voices_list_http_request()[0]
assert _DEFAULT_VOICE_NAME in voices.json_body["voices"]
assert "bar" in voices.json_body["voices"]
# 5) Speech with the default voice should still succeed
openai_client.send_audio_speech_request(speech_config)
online_client.send_audio_speech_request(speech_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", TEST_PARAMS, indirect=True)
def test_registered_default_voice_overrides_placeholder(omni_server, openai_client) -> None:
def test_registered_default_voice_overrides_placeholder(omni_server, online_client) -> None:
"""When a real voice is uploaded under the default name, it should be
used as a cloned voice; after deletion, the placeholder behavior resumes."""
speech_config = {
@@ -226,7 +212,7 @@ def test_registered_default_voice_overrides_placeholder(omni_server, openai_clie
}
# 1) Placeholder default works before any registration
openai_client.send_audio_speech_request(speech_config)
online_client.send_audio_speech_request(speech_config)
# 2) Register a real voice under the default name
register_config = {
@@ -239,23 +225,23 @@ def test_registered_default_voice_overrides_placeholder(omni_server, openai_clie
),
},
}
openai_client.send_audio_voices_create_http_request(register_config)
online_client.send_audio_voices_create_http_request(register_config)
# 3) After creating the voice with the default name, it's visible as an uploaded_voice
voices = openai_client.send_audio_voices_list_http_request()[0]
voices = online_client.send_audio_voices_list_http_request()[0]
assert _DEFAULT_VOICE_NAME in voices.json_body["voices"]
uploaded_names = [v["name"] for v in voices.json_body["uploaded_voices"]]
assert _DEFAULT_VOICE_NAME in uploaded_names
openai_client.send_audio_speech_request(speech_config)
online_client.send_audio_speech_request(speech_config)
# 4) Delete the uploaded default voice
openai_client.send_audio_voices_delete_http_request(
online_client.send_audio_voices_delete_http_request(
{"name": _DEFAULT_VOICE_NAME},
)
# 5) Ensure the placeholder behavior is consistent, but it's not an uploaded voice now
voices = openai_client.send_audio_voices_list_http_request()[0]
voices = online_client.send_audio_voices_list_http_request()[0]
assert _DEFAULT_VOICE_NAME in voices.json_body["voices"]
uploaded_names = [v["name"] for v in voices.json_body["uploaded_voices"]]
assert _DEFAULT_VOICE_NAME not in uploaded_names
openai_client.send_audio_speech_request(speech_config)
online_client.send_audio_speech_request(speech_config)
tests/e2e/online_serving/test_qwen2_5_omni_expansion.py
+7 -4
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for Qwen2.5-Omni model with video input and audio output.
"""
@@ -59,7 +62,7 @@ def get_max_batch_size(size_type="few"):
@pytest.mark.omni
@hardware_test(res={"cuda": "L4", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_mix_to_text_audio_001(omni_server, openai_client) -> None:
def test_mix_to_text_audio_001(omni_server, online_client) -> None:
"""
Test multi-modal input processing and text/audio output generation via OpenAI API.
Deploy Setting: default yaml
@@ -90,13 +93,13 @@ def test_mix_to_text_audio_001(omni_server, openai_client) -> None:
}
# Test single completion
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@pytest.mark.slow
@pytest.mark.omni
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_to_text_001(omni_server, openai_client) -> None:
def test_text_to_text_001(omni_server, online_client) -> None:
"""
Test text input processing and text/audio output generation via OpenAI API.
Deploy Setting: default yaml
@@ -114,4 +117,4 @@ def test_text_to_text_001(omni_server, openai_client) -> None:
"key_words": {"text": ["beijing"]},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
tests/e2e/online_serving/test_qwen3_omni.py
+16 -13
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for Qwen3-Omni model with video input and audio output.
"""
@@ -82,7 +85,7 @@ def get_max_batch_size(size_type="few"):
@pytest.mark.skipif(_USE_PD, reason="Temporarily skip PD mode in this test module.")
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=3 if _USE_PD else 2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_mix_to_text_audio_001(omni_server, openai_client) -> None:
def test_mix_to_text_audio_001(omni_server, online_client) -> None:
"""
Test multi-modal input processing and text/audio output generation via OpenAI API.
Deploy Setting: default yaml
@@ -113,7 +116,7 @@ def test_mix_to_text_audio_001(omni_server, openai_client) -> None:
}
# Test single completion
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
@pytest.mark.advanced_model
@@ -122,7 +125,7 @@ def test_mix_to_text_audio_001(omni_server, openai_client) -> None:
@pytest.mark.skipif(_USE_PD, reason="Temporarily skip PD mode in this test module.")
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=3 if _USE_PD else 2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_to_text_001(omni_server, openai_client) -> None:
def test_text_to_text_001(omni_server, online_client) -> None:
"""
Test text input processing and text/audio output generation via OpenAI API.
Deploy Setting: default yaml
@@ -140,10 +143,10 @@ def test_text_to_text_001(omni_server, openai_client) -> None:
"key_words": {"text": ["beijing"]},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
def _run_prefix_cache_check(openai_client, request_config: dict):
def _run_prefix_cache_check(online_client, request_config: dict):
"""Make two requests given a request config, and validate that:
1. The second request actually had cached tokens
2. The number of cached tokens is divisible by the block size used in
@@ -157,8 +160,8 @@ def _run_prefix_cache_check(openai_client, request_config: dict):
Returns the (first, second) responses for additional per-test assertions.
"""
first_response = openai_client.send_omni_request(request_config, request_num=1)[0]
cached_response = openai_client.send_omni_request(request_config, request_num=1)[0]
first_response = online_client.send_omni_request(request_config, request_num=1)[0]
cached_response = online_client.send_omni_request(request_config, request_num=1)[0]
assert first_response.cached_tokens is not None
@@ -180,7 +183,7 @@ def _run_prefix_cache_check(openai_client, request_config: dict):
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_thinker_prefix_caching_text_output(omni_server, openai_client) -> None:
def test_thinker_prefix_caching_text_output(omni_server, online_client) -> None:
"""
Test thinker prefix caching by sending identical requests with an image (i.e.,
a large shared prefix) and verifying that the second request uses cached tokens
@@ -206,7 +209,7 @@ def test_thinker_prefix_caching_text_output(omni_server, openai_client) -> None:
"stream": False,
"modalities": ["text"],
}
first_response, _ = _run_prefix_cache_check(openai_client, request_config)
first_response, _ = _run_prefix_cache_check(online_client, request_config)
assert first_response.multimodal_tokens is not None
assert first_response.multimodal_tokens.get("image", 0) > 0
@@ -217,7 +220,7 @@ def test_thinker_prefix_caching_text_output(omni_server, openai_client) -> None:
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_thinker_prefix_caching_audio_output(omni_server, openai_client) -> None:
def test_thinker_prefix_caching_audio_output(omni_server, online_client) -> None:
"""
Verify that thinker prefix caching does not hang when the request
produces audio output (text + audio modalities). Sends two identical
@@ -239,7 +242,7 @@ def test_thinker_prefix_caching_audio_output(omni_server, openai_client) -> None
},
}
_run_prefix_cache_check(openai_client, request_config)
_run_prefix_cache_check(online_client, request_config)
@pytest.mark.advanced_model
@@ -247,13 +250,13 @@ def test_thinker_prefix_caching_audio_output(omni_server, openai_client) -> None
@pytest.mark.omni
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_completions_rejected_for_thinker_talker(omni_server, openai_client) -> None:
def test_completions_rejected_for_thinker_talker(omni_server, online_client) -> None:
"""Ensure Thinker-talker models reject /v1/completions; we do this because the
thinker-talker handoff implementations currently use ChatML <|im_start|> and
<|im_end|> markers to segment the input sequence; when we don't have them,
the talker does not get any embeddings, which breaks the server.
"""
responses = openai_client.send_completions_http_request(
responses = online_client.send_completions_http_request(
{
"json": {
"model": omni_server.model,
tests/e2e/online_serving/test_qwen3_omni_expansion.py
+38 -38
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for Qwen3-Omni model.
"""
@@ -139,7 +139,7 @@ def get_max_batch_size(size_type="few"):
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_to_audio_001(omni_server, openai_client) -> None:
def test_text_to_audio_001(omni_server, online_client) -> None:
"""
Input Modal: text
Output Modal: audio
@@ -156,12 +156,12 @@ def test_text_to_audio_001(omni_server, openai_client) -> None:
"key_words": {"text": ["beijing"]},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params + test_token_params, indirect=True)
def test_text_to_text_audio_001(omni_server, openai_client) -> None:
def test_text_to_text_audio_001(omni_server, online_client) -> None:
"""
Input Modal: text
Output Modal: text, audio
@@ -176,12 +176,12 @@ def test_text_to_text_audio_001(omni_server, openai_client) -> None:
"key_words": {"text": ["beijing"]},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_video_to_text_001(omni_server, openai_client) -> None:
def test_text_video_to_text_001(omni_server, online_client) -> None:
"""
Input Modal: long synthetic video (120s @ 30fps, LONG_VIDEO_FRAMES frames)
Output Modal: text
@@ -200,12 +200,12 @@ def test_text_video_to_text_001(omni_server, openai_client) -> None:
"key_words": {"video": VIDEO_KEY},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params + test_token_params, indirect=True)
def test_text_audio_to_text_audio_001(omni_server, openai_client) -> None:
def test_text_audio_to_text_audio_001(omni_server, online_client) -> None:
"""
Input Modal: text, audio
Output Modal: text, audio
@@ -223,12 +223,12 @@ def test_text_audio_to_text_audio_001(omni_server, openai_client) -> None:
"key_words": {"audio": AUDIO_KEY},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params + test_token_params, indirect=True)
def test_text_audio_to_text_audio_002(omni_server, openai_client) -> None:
def test_text_audio_to_text_audio_002(omni_server, online_client) -> None:
"""
Input Modal: text, long-duration audio (~LONG_AUDIO_DURATION_SEC s WAV)
Output Modal: text, audio
@@ -248,12 +248,12 @@ def test_text_audio_to_text_audio_002(omni_server, openai_client) -> None:
"key_words": {"audio": AUDIO_KEY},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params + test_token_params, indirect=True)
def test_text_image_to_text_audio_001(omni_server, openai_client) -> None:
def test_text_image_to_text_audio_001(omni_server, online_client) -> None:
"""
Input Modal: text, image
Output Modal: text, audio
@@ -272,12 +272,12 @@ def test_text_image_to_text_audio_001(omni_server, openai_client) -> None:
"key_words": {"image": IMAGE_KEY},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params + test_token_params, indirect=True)
def test_large_image_to_text_audio_001(omni_server, openai_client) -> None:
def test_large_image_to_text_audio_001(omni_server, online_client) -> None:
"""
Input Modal: text, high-resolution image (1080p-class JPEG)
Output Modal: text, audio
@@ -300,12 +300,12 @@ def test_large_image_to_text_audio_001(omni_server, openai_client) -> None:
"key_words": {"image": IMAGE_KEY},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params + test_token_params, indirect=True)
def test_text_video_to_text_audio_001(omni_server, openai_client) -> None:
def test_text_video_to_text_audio_001(omni_server, online_client) -> None:
"""
Input Modal: text, video
Output Modal: text, audio
@@ -325,13 +325,13 @@ def test_text_video_to_text_audio_001(omni_server, openai_client) -> None:
"key_words": {"video": VIDEO_KEY},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@pytest.mark.skip(reason="There is a known issue with shape mismatch error.")
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params + test_token_params, indirect=True)
def test_mix_to_text_audio_001(omni_server, openai_client) -> None:
def test_mix_to_text_audio_001(omni_server, online_client) -> None:
"""
Input Modal: text, audio, image, video
Output Modal: text, audio
@@ -355,12 +355,12 @@ def test_mix_to_text_audio_001(omni_server, openai_client) -> None:
"stream": True,
"key_words": {"audio": AUDIO_KEY, "image": IMAGE_KEY, "video": VIDEO_KEY},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_audio_in_video_001(omni_server, openai_client) -> None:
def test_audio_in_video_001(omni_server, online_client) -> None:
"""
Input Modal: text + video (synthetic MP4 with embedded audio; ``use_audio_in_video`` uses audio from the video).
Output Modal: text, audio
@@ -381,12 +381,12 @@ def test_audio_in_video_001(omni_server, openai_client) -> None:
"use_audio_in_video": True,
"key_words": {"video": VIDEO_KEY},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_audio_in_video_002(omni_server, openai_client) -> None:
def test_audio_in_video_002(omni_server, online_client) -> None:
"""
Input Modal: text + video (synthetic MP4 with embedded audio; ``use_audio_in_video`` uses audio from the video).
Output Modal: text, audio
@@ -408,12 +408,12 @@ def test_audio_in_video_002(omni_server, openai_client) -> None:
"key_words": {"video": VIDEO_KEY},
}
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_audio_in_video_default_loader_sampling_regression(omni_server, openai_client) -> None:
def test_audio_in_video_default_loader_sampling_regression(omni_server, online_client) -> None:
"""
Regression: ``use_audio_in_video`` with default video loader sampling (no
``media_io_kwargs.video.fps``) after a prior request that pinned fps.
@@ -438,7 +438,7 @@ def test_audio_in_video_default_loader_sampling_regression(omni_server, openai_c
}
# Prime mm-cache with explicit loader fps (historical failure mode).
openai_client.send_omni_request(
online_client.send_omni_request(
{
**base_config,
"extra_body": {"media_io_kwargs": {"video": {"fps": 1, "num_frames": 128}}},
@@ -446,12 +446,12 @@ def test_audio_in_video_default_loader_sampling_regression(omni_server, openai_c
)
# Default loader sampling: must succeed without explicit media_io fps.
openai_client.send_omni_request(dict(base_config))
online_client.send_omni_request(dict(base_config))
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_one_word_prompt_001(omni_server, openai_client) -> None:
def test_one_word_prompt_001(omni_server, online_client) -> None:
"""Catastrophic-regression gate on one-word pronunciation.
Whether a one-word answer is pronounced intelligibly is sampled, not guaranteed: the
@@ -498,14 +498,14 @@ def test_one_word_prompt_001(omni_server, openai_client) -> None:
# concurrently where the previous request_num=5 path judged serially -- worth watching
# on tighter cards, since each judge loads whisper on the same device as the talker and
# code2wav stages.
openai_client.send_omni_request(
online_client.send_omni_request(
request_config, request_num=40, min_successes=29, max_concurrency=get_max_batch_size()
)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_speaker_001(omni_server, openai_client) -> None:
def test_speaker_001(omni_server, online_client) -> None:
"""
Input Modal: text only (one-word answer constraint).
Output Modal: text, audio (default ``modalities``); ``key_words`` only assert on text.
@@ -525,12 +525,12 @@ def test_speaker_001(omni_server, openai_client) -> None:
"key_words": {"text": ["beijing"]},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_speaker_002(omni_server, openai_client) -> None:
def test_speaker_002(omni_server, online_client) -> None:
"""
Input Modal: text only (one-word answer constraint).
Output Modal: text, audio (default ``modalities``); ``key_words`` only assert on text.
@@ -555,7 +555,7 @@ def test_speaker_002(omni_server, openai_client) -> None:
_max_retries = 10
for attempt in range(_max_retries):
try:
openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
online_client.send_omni_request(request_config, request_num=get_max_batch_size())
break
except AssertionError as e:
if _gender_assert_substr not in str(e) or attempt == _max_retries - 1:
@@ -565,7 +565,7 @@ def test_speaker_002(omni_server, openai_client) -> None:
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_speaker_003(omni_server, openai_client) -> None:
def test_speaker_003(omni_server, online_client) -> None:
"""
Input Modal: text only (one-word answer constraint).
Output Modal: text, audio (default ``modalities``); ``key_words`` only assert on text.
@@ -585,12 +585,12 @@ def test_speaker_003(omni_server, openai_client) -> None:
"key_words": {"text": ["beijing"]},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_language_001(omni_server, openai_client) -> None:
def test_language_001(omni_server, online_client) -> None:
"""
Input Modal: text only (one-word answer constraint).
Output Modal: text, audio (default ``modalities``); ``key_words`` only assert on text.
@@ -609,12 +609,12 @@ def test_language_001(omni_server, openai_client) -> None:
"key_words": {"text": ["北京"]},
}
openai_client.send_omni_request(request_config)
online_client.send_omni_request(request_config)
@hardware_test(res={"cuda": "H100", "rocm": "MI325"}, num_cards=2)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_to_audio_long_output_001(omni_server, openai_client) -> None:
def test_text_to_audio_long_output_001(omni_server, online_client) -> None:
"""
Input Modal: text only (long-form generation prompt).
Output Modal: text, audio (default ``modalities``);
@@ -627,7 +627,7 @@ def test_text_to_audio_long_output_001(omni_server, openai_client) -> None:
)
request_config = {"model": omni_server.model, "messages": messages, "stream": True}
responses = openai_client.send_omni_request(request_config, request_num=get_max_batch_size())
responses = online_client.send_omni_request(request_config, request_num=get_max_batch_size())
text = responses[0].text_content if responses else ""
word_count = len(text.split())
assert word_count >= 200, f"Expected at least 200 words in long output, got {word_count}"
tests/e2e/online_serving/test_qwen3_omni_multi_replicas.py
+9 -6
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Core-model CI guard for Qwen3-Omni multi-replica stage-pool routing on 4 GPUs.
"""
@@ -90,7 +93,7 @@ def _assert_audio_outputs(responses: list[OmniResponse], *, expect_text: bool) -
@pytest.mark.omni
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_only_batch_uses_multi_replica_talker(omni_server, openai_client) -> None:
def test_text_only_batch_uses_multi_replica_talker(omni_server, online_client) -> None:
request_config = {
"model": omni_server.model,
"messages": _text_messages(),
@@ -98,7 +101,7 @@ def test_text_only_batch_uses_multi_replica_talker(omni_server, openai_client) -
"modalities": ["text"],
}
responses = openai_client.send_omni_request(request_config, request_num=ROUTE_STRESS_REQUESTS)
responses = online_client.send_omni_request(request_config, request_num=ROUTE_STRESS_REQUESTS)
_assert_batch_size(responses, ROUTE_STRESS_REQUESTS)
_assert_text_outputs(responses)
@@ -107,7 +110,7 @@ def test_text_only_batch_uses_multi_replica_talker(omni_server, openai_client) -
@pytest.mark.omni
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_text_to_audio_stream_batch_uses_multi_replica_vocoder(omni_server, openai_client) -> None:
def test_text_to_audio_stream_batch_uses_multi_replica_vocoder(omni_server, online_client) -> None:
request_config = {
"model": omni_server.model,
"messages": _text_messages(),
@@ -115,7 +118,7 @@ def test_text_to_audio_stream_batch_uses_multi_replica_vocoder(omni_server, open
"modalities": ["audio"],
}
responses = openai_client.send_omni_request(request_config, request_num=ROUTE_STRESS_REQUESTS)
responses = online_client.send_omni_request(request_config, request_num=ROUTE_STRESS_REQUESTS)
_assert_batch_size(responses, ROUTE_STRESS_REQUESTS)
_assert_audio_outputs(responses, expect_text=False)
@@ -124,13 +127,13 @@ def test_text_to_audio_stream_batch_uses_multi_replica_vocoder(omni_server, open
@pytest.mark.omni
@hardware_test(res={"cuda": "H100"}, num_cards=4)
@pytest.mark.parametrize("omni_server", test_params, indirect=True)
def test_mixed_modal_stream_batch_generates_text_and_audio(omni_server, openai_client) -> None:
def test_mixed_modal_stream_batch_generates_text_and_audio(omni_server, online_client) -> None:
request_config = {
"model": omni_server.model,
"messages": _mixed_messages(),
"stream": True,
}
responses = openai_client.send_omni_request(request_config, request_num=MIXED_MODAL_REQUESTS)
responses = online_client.send_omni_request(request_config, request_num=MIXED_MODAL_REQUESTS)
_assert_batch_size(responses, MIXED_MODAL_REQUESTS)
_assert_audio_outputs(responses, expect_text=True)
tests/e2e/online_serving/test_qwen3_tts_base.py
+7 -7
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for Qwen3-TTS model with text input and audio output.
@@ -14,7 +14,7 @@ os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServerParams
from tests.helpers.stage_config import get_deploy_config_path
@@ -24,7 +24,7 @@ DEFAULT_AUDIO_SPEECH_TIMEOUT_S = 180.0
# Vendored under tests/assets/qwen3_tts/clone_2.wav so the server does not need
# to fetch the reference audio over HTTPS at request time (CI runners in some
# regions cannot reach the original Aliyun OSS host; see issue #3263).
REF_AUDIO_URL = load_test_audio_data_url("qwen3_tts/clone_2.wav")
REF_AUDIO_URL = get_asset_path("qwen3_tts/clone_2.wav", as_data_url=True)
REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
@@ -59,7 +59,7 @@ tts_server_params = [
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_001(omni_server, openai_client) -> None:
def test_text_to_audio_001(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -80,14 +80,14 @@ def test_text_to_audio_001(omni_server, openai_client) -> None:
"ref_audio": REF_AUDIO_URL,
"ref_text": REF_TEXT,
}
openai_client.send_audio_speech_request(request_config, request_num=get_max_batch_size("few"))
online_client.send_audio_speech_request(request_config, request_num=get_max_batch_size("few"))
@pytest.mark.advanced_model
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_002(omni_server, openai_client) -> None:
def test_text_to_audio_002(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -108,4 +108,4 @@ def test_text_to_audio_002(omni_server, openai_client) -> None:
"ref_audio": REF_AUDIO_URL,
"ref_text": REF_TEXT,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
tests/e2e/online_serving/test_qwen3_tts_base_expansion.py
+53 -13
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for Qwen3-TTS model with text input and audio output.
@@ -12,7 +12,7 @@ import os
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.media import get_asset_path
from tests.helpers.runtime import OmniServerParams
from tests.helpers.stage_config import get_deploy_config_path
@@ -23,7 +23,7 @@ os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
MODEL = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
# See tests/e2e/online_serving/test_qwen3_tts_base.py for the vendored-asset rationale.
REF_AUDIO_URL = load_test_audio_data_url("qwen3_tts/clone_2.wav")
REF_AUDIO_URL = get_asset_path("qwen3_tts/clone_2.wav", as_data_url=True)
REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
@@ -41,7 +41,7 @@ def get_max_batch_size(size_type="few"):
return batch_sizes.get(size_type, 5)
tts_server_params = [
tts_async_chunk_server_params = [
pytest.param(
OmniServerParams(
model=MODEL,
@@ -50,9 +50,12 @@ tts_server_params = [
),
id="async_chunk",
),
# Synchronous (no async-chunk) variant — ``--no-async-chunk`` alone
# flips the deploy yaml's bool and the pipeline dispatches to the
# end-to-end codec processor. No variant yaml / pipeline needed.
]
# Synchronous (no async-chunk) variant — ``--no-async-chunk`` alone
# flips the deploy yaml's bool and the pipeline dispatches to the
# end-to-end codec processor. No variant yaml / pipeline needed.
tts_no_async_chunk_server_params = [
pytest.param(
OmniServerParams(
model=MODEL,
@@ -63,10 +66,16 @@ tts_server_params = [
),
]
DEFAULT_AUDIO_SPEECH_TIMEOUT_S = 180.0
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_voice_clone_streaming_001(omni_server, openai_client) -> None:
@pytest.mark.parametrize(
"omni_server",
tts_async_chunk_server_params + tts_no_async_chunk_server_params,
indirect=True,
)
def test_voice_clone_streaming_001(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -87,12 +96,16 @@ def test_voice_clone_streaming_001(omni_server, openai_client) -> None:
"ref_audio": REF_AUDIO_URL,
"ref_text": REF_TEXT,
}
openai_client.send_audio_speech_request(request_config, request_num=get_max_batch_size("few"))
online_client.send_audio_speech_request(request_config, request_num=get_max_batch_size("few"))
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_response_format_001(omni_server, openai_client) -> None:
@pytest.mark.parametrize(
"omni_server",
tts_async_chunk_server_params + tts_no_async_chunk_server_params,
indirect=True,
)
def test_response_format_001(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -111,4 +124,31 @@ def test_response_format_001(omni_server, openai_client) -> None:
"ref_audio": REF_AUDIO_URL,
"ref_text": REF_TEXT,
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_async_chunk_server_params, indirect=True)
def test_xvector_then_icl_same_ref_audio_keeps_engine_alive(omni_server, online_client) -> None:
"""Regression for #5049: x-vector-only then ICL on the same ``ref_audio``.
Pre-fix, readiness was mode-agnostic: an x-vector request marked the
artifact ready (embedding only, no ``ref_code``), so the following ICL
request took the artifact-only path and killed EngineCore. After the fix
(readiness keyed by ``(artifact_key, x_vector_only)``), both modes succeed
and the server stays up. Only runs on the ``async_chunk`` server param.
"""
base_request = {
"model": omni_server.model,
"input": get_prompt(),
"stream": False,
"timeout": DEFAULT_AUDIO_SPEECH_TIMEOUT_S,
"response_format": "wav",
"task_type": "Base",
"ref_audio": REF_AUDIO_URL,
"min_audio_bytes": 1,
}
online_client.send_audio_speech_request({**base_request, "x_vector_only_mode": True})
online_client.send_audio_speech_request({**base_request, "x_vector_only_mode": False, "ref_text": REF_TEXT})
online_client.send_audio_speech_request({**base_request, "x_vector_only_mode": True})
tests/e2e/online_serving/test_qwen3_tts_customvoice.py
+5 -5
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for Qwen3-TTS model with text input and audio output.
@@ -93,7 +93,7 @@ def test_default_cuda_graph_startup(omni_server) -> None:
@pytest.mark.tts
@hardware_test(res={"cuda": "L4", "npu": "A3"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_001(omni_server, openai_client) -> None:
def test_text_to_audio_001(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -111,7 +111,7 @@ def test_text_to_audio_001(omni_server, openai_client) -> None:
"voice": "vivian",
}
openai_client.send_audio_speech_request(request_config, request_num=get_max_batch_size())
online_client.send_audio_speech_request(request_config, request_num=get_max_batch_size())
@pytest.mark.core_model
@@ -119,7 +119,7 @@ def test_text_to_audio_001(omni_server, openai_client) -> None:
@pytest.mark.tts
@hardware_test(res={"cuda": "L4", "npu": "A3"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_text_to_audio_002(omni_server, openai_client) -> None:
def test_text_to_audio_002(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -138,4 +138,4 @@ def test_text_to_audio_002(omni_server, openai_client) -> None:
"voice": "vivian",
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
tests/e2e/online_serving/test_qwen3_tts_customvoice_expansion.py
+9 -9
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
E2E Online tests for Qwen3-TTS model with text input and audio output.
@@ -62,7 +62,7 @@ tts_server_params = [
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_voice_001(omni_server, openai_client) -> None:
def test_voice_001(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -84,7 +84,7 @@ def test_voice_001(omni_server, openai_client) -> None:
# Retry once on assertion failures from transcript similarity / gender checks (flaky ASR or estimators).
for attempt in range(2):
try:
openai_client.send_audio_speech_request(request_config, request_num=get_max_batch_size())
online_client.send_audio_speech_request(request_config, request_num=get_max_batch_size())
break
except AssertionError:
if attempt == 0:
@@ -94,7 +94,7 @@ def test_voice_001(omni_server, openai_client) -> None:
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_voice_002(omni_server, openai_client) -> None:
def test_voice_002(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -112,12 +112,12 @@ def test_voice_002(omni_server, openai_client) -> None:
"voice": "Serena",
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_voice_003(omni_server, openai_client) -> None:
def test_voice_003(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -135,12 +135,12 @@ def test_voice_003(omni_server, openai_client) -> None:
"voice": "SERENA",
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_language_001(omni_server, openai_client) -> None:
def test_language_001(omni_server, online_client) -> None:
"""
Test text input processing and audio output via OpenAI API.
Deploy Setting: default yaml
@@ -158,4 +158,4 @@ def test_language_001(omni_server, openai_client) -> None:
"voice": "vivian",
}
openai_client.send_audio_speech_request(request_config)
online_client.send_audio_speech_request(request_config)
tests/e2e/online_serving/test_qwen3_tts_ref_audio_artifact_mode_e2e.py
-88
View File
@@ -1,88 +0,0 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
"""E2E regression for Qwen3-TTS ref_audio artifact mode collision (#5049).
Reproduces the issue's exact sequence on a live server: an
``x_vector_only_mode=true`` request followed by an ICL
(``x_vector_only_mode=false`` + ``ref_text``) request using the **same**
``ref_audio``.
Before the fix, the ref_audio artifact readiness was mode-agnostic: the
x-vector-only request (which caches a speaker embedding but no ``ref_code``)
marked the artifact "ready", so the ICL request took the artifact-only serving
path (``ref_audio`` stripped before dispatch), found no ``ref_code`` and no
audio to recompute from, and raised in ``build_prompt_embeds`` ->
``EngineDeadError`` — taking the whole server down rather than failing the one
request.
After the fix (readiness keyed by ``(artifact_key, x_vector_only)``), the ICL
request no longer matches the x-vector artifact, so it sends ``ref_audio`` and
the worker computes ``ref_code``. Both requests succeed and the server stays up.
"""
import os
os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
import pytest
from tests.helpers.mark import hardware_test
from tests.helpers.media import load_test_audio_data_url
from tests.helpers.runtime import OmniServerParams
from tests.helpers.stage_config import get_deploy_config_path
MODEL = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
DEFAULT_AUDIO_SPEECH_TIMEOUT_S = 180.0
# Vendored under tests/assets/qwen3_tts/clone_2.wav (see test_qwen3_tts_base.py).
REF_AUDIO_URL = load_test_audio_data_url("qwen3_tts/clone_2.wav")
REF_TEXT = "Okay. Yeah. I resent you. I love you. I respect you. But you know what? You blew it! And thanks to you."
INPUT_TEXT = "The weather is nice today, perfect for a walk in the park."
tts_server_params = [
pytest.param(
OmniServerParams(
model=MODEL,
stage_config_path=get_deploy_config_path("qwen3_tts.yaml"),
server_args=["--trust-remote-code"],
),
id="async_chunk",
)
]
@pytest.mark.advanced_model
@pytest.mark.core_model
@pytest.mark.tts
@hardware_test(res={"cuda": "L4"}, num_cards=1)
@pytest.mark.parametrize("omni_server", tts_server_params, indirect=True)
def test_xvector_then_icl_same_ref_audio_keeps_engine_alive(omni_server, openai_client) -> None:
"""Regression for #5049: x-vector-only then ICL on the same ref_audio.
Both requests must return audio and the engine must stay alive (pre-fix the
second request killed EngineCore).
"""
base_request = {
"model": omni_server.model,
"input": INPUT_TEXT,
"stream": False,
"timeout": DEFAULT_AUDIO_SPEECH_TIMEOUT_S,
"response_format": "wav",
"task_type": "Base",
"ref_audio": REF_AUDIO_URL,
# Assert non-empty audio came back (not just HTTP 200).
"min_audio_bytes": 1,
}
# 1) x-vector-only request: caches a speaker embedding (no ref_code) and, on
# completion, marks the ref_audio artifact "ready".
openai_client.send_audio_speech_request({**base_request, "x_vector_only_mode": True})
# 2) ICL request with the SAME ref_audio. Pre-fix this reused the x-vector
# artifact via the artifact-only path, hit the missing ref_code, and
# killed EngineCore (#5049). It must now succeed with audio.
openai_client.send_audio_speech_request({**base_request, "x_vector_only_mode": False, "ref_text": REF_TEXT})
# 3) The server must still be serving after the ICL request (pre-fix this
# would fail with EngineDeadError / connection refused).
openai_client.send_audio_speech_request({**base_request, "x_vector_only_mode": True})
tests/e2e/online_serving/test_qwen_image.py
+6 -6
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Online serving tests for ``Qwen/Qwen-Image`` (text-to-image).
@@ -24,7 +24,7 @@ import os
import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
@@ -61,7 +61,7 @@ def _get_default_case(model: str):
@pytest.mark.advanced_model
@pytest.mark.diffusion
@pytest.mark.parametrize("omni_server", _get_default_case(MODEL), indirect=True)
def test_text_to_image_001(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
def test_text_to_image_001(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
"""Default Qwen-Image T2I smoke (single ``default`` server config)."""
messages = dummy_messages_from_mix_data(content_text=T2I_PROMPT)
request_config = {
@@ -76,14 +76,14 @@ def test_text_to_image_001(omni_server: OmniServer, openai_client: OpenAIClientH
"seed": 42,
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.core_model
@pytest.mark.advanced_model
@pytest.mark.diffusion
@pytest.mark.parametrize("omni_server", _get_default_case(MODEL), indirect=True)
def test_batch_001(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
def test_batch_001(omni_server: OmniServer, online_client: OnlineOmniClient) -> None:
"""Concurrent T2I: one ``request_config`` dict per prompt (``send_diffusion_request`` list mode)."""
request_config = [
{
@@ -100,4 +100,4 @@ def test_batch_001(omni_server: OmniServer, openai_client: OpenAIClientHandler)
}
for prompt in TEST_PROMPTS
]
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_qwen_image_edit.py
+9 -9
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Online serving tests for ``Qwen/Qwen-Image-Edit-2511`` (image-to-image via chat completions).
@@ -8,7 +8,7 @@ Online serving tests for ``Qwen/Qwen-Image-Edit-2511`` (image-to-image via chat
- ``test_multi_images_to_image_001``: two reference images, fixed 512×512.
- ``test_different_sizes_001``: ``slow`` — mixed input resolutions; ``extra_body`` uses
per-output ``width``/``height`` lists; the test client sends one scalar-size request
per list index in parallel and merges images (see ``OpenAIClientHandler.send_diffusion_request``).
per list index in parallel and merges images (see ``OnlineOmniClient.send_diffusion_request``).
``_get_diffusion_feature_cases`` registers a single ``default`` ``OmniServerParams`` row.
@@ -24,7 +24,7 @@ from tests.helpers.media import generate_synthetic_image
from tests.helpers.runtime import (
OmniServer,
OmniServerParams,
OpenAIClientHandler,
OnlineOmniClient,
dummy_messages_from_mix_data,
)
@@ -59,7 +59,7 @@ def _get_diffusion_feature_cases(model: str = MODEL):
_get_diffusion_feature_cases(),
indirect=True,
)
def test_single_image_to_image_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_single_image_to_image_001(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Single-reference edit smoke for ``Qwen/Qwen-Image-Edit-2511`` (CFG when negative prompt + ``true_cfg_scale`` > 1)."""
image_data_url = f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}"
@@ -79,7 +79,7 @@ def test_single_image_to_image_001(omni_server: OmniServer, openai_client: OpenA
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.slow
@@ -89,7 +89,7 @@ def test_single_image_to_image_001(omni_server: OmniServer, openai_client: OpenA
_get_diffusion_feature_cases(),
indirect=True,
)
def test_multi_images_to_image_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_multi_images_to_image_001(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Two-reference edit smoke for ``Qwen/Qwen-Image-Edit-2511``."""
image_data_url_list = [f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}" for _ in range(2)]
@@ -110,7 +110,7 @@ def test_multi_images_to_image_001(omni_server: OmniServer, openai_client: OpenA
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.slow
@@ -120,7 +120,7 @@ def test_multi_images_to_image_001(omni_server: OmniServer, openai_client: OpenA
_get_diffusion_feature_cases(),
indirect=True,
)
def test_different_sizes_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_different_sizes_001(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Multi-reference edit with distinct input resolutions; per-output ``width``/``height`` as lists (client splits into concurrent scalar-size calls)."""
image_data_url_list = [
f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}",
@@ -143,4 +143,4 @@ def test_different_sizes_001(omni_server: OmniServer, openai_client: OpenAIClien
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
tests/e2e/online_serving/test_qwen_image_edit_expansion.py
+8 -5
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Comprehensive tests of diffusion features that are available in online serving mode
and are supported by Qwen-Image-Edit-2511 (single image input and two image inputs).
@@ -7,7 +10,7 @@ import pytest
from tests.helpers.mark import hardware_marks
from tests.helpers.media import generate_synthetic_image
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
from tests.helpers.runtime import OmniServer, OmniServerParams, OnlineOmniClient, dummy_messages_from_mix_data
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
@@ -122,7 +125,7 @@ def _get_diffusion_feature_cases(model: str = MODEL):
_get_diffusion_feature_cases(),
indirect=True,
)
def test_qwen_image_edit_2511_single_image(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_qwen_image_edit_2511_single_image(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test Qwen-Image-Edit-2511 with a single image input.
Regression: with tea_cache enabled and zero_cond_t=True, the TeaCache
@@ -149,7 +152,7 @@ def test_qwen_image_edit_2511_single_image(omni_server: OmniServer, openai_clien
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
@pytest.mark.parametrize(
@@ -157,7 +160,7 @@ def test_qwen_image_edit_2511_single_image(omni_server: OmniServer, openai_clien
_get_diffusion_feature_cases(),
indirect=True,
)
def test_qwen_image_edit_2511_two_images(omni_server: OmniServer, openai_client: OpenAIClientHandler):
def test_qwen_image_edit_2511_two_images(omni_server: OmniServer, online_client: OnlineOmniClient):
"""Test Qwen-Image-Edit-2511 with two image inputs."""
image_data_url_1 = f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}"
image_data_url_2 = f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}"
@@ -180,4 +183,4 @@ def test_qwen_image_edit_2511_two_images(omni_server: OmniServer, openai_client:
},
}
openai_client.send_diffusion_request(request_config)
online_client.send_diffusion_request(request_config)
Loaded 100 of 140 files, more files were not shown because too many files have changed in this diff. Show more