anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Docs] Clarify pipeline orchestration defaults and override timing

Browse Source 
Signed-off-by: Sy03 <1370724210@qq.com>
This commit is contained in:
Sy03 committed 2026-09-21 19:07:20 +08:00
1 parent ea7de7e3de
commit b3129048e6
2 files changed
+10 -8
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/serving/speech_api.md
+9 -7
View File
@@ -957,8 +957,8 @@ Use `/v1/audio/voices` to list available voices for the loaded model.
## Orchestration Loop (experimental)
Multi-stage omni deployments route stage outputs through a single orchestrator
loop. By default that loop polls every stage replica on a 1 ms cadence. An
the event-driven mode replaces the poll with one reader task per live stage
loop. The legacy loop polls every stage replica on a 1 ms cadence. The
event-driven mode replaces the poll with one reader task per live stage
replica awaiting its client directly, and switches the serving-side
final-output drain to a condition-variable wakeup at the same time. Qwen3-TTS
uses this mode by default; other pipelines keep the legacy poll unless enabled.
@@ -967,13 +967,14 @@ uses this mode by default; other pipelines keep the legacy poll unless enabled.
| Variable | Default | Description |
| --- | --- | --- |
| `VLLM_OMNI_EVENT_DRIVEN_ORCH` | `0` (off), except Qwen3-TTS defaults to on | Switches the orchestration loop and the final-output drain from the legacy 1 ms poll to event-driven wakeups. An explicit value wins; otherwise the pipeline default computed at engine initialization is used. Values are matched case-insensitively after surrounding whitespace is stripped; unrecognized values select the legacy poll loop. |
| `VLLM_OMNI_EVENT_DRIVEN_ORCH` | On for Qwen3-TTS; off for other pipelines | Switches the orchestration loop and the final-output drain from the legacy 1 ms poll to event-driven wakeups. An explicit value wins; otherwise the pipeline default computed at engine initialization is used. The override is resolved at orchestrator construction and separately when the final-output drain starts. `1`, `true`, `yes`, or `on` enables it, ignoring case and surrounding whitespace; other values select the legacy poll loop. |
Set it on the process that runs the orchestrator (stage 0 of an omni deployment)
before starting the server when overriding the pipeline default:
before starting the server when overriding the pipeline default. For example,
explicitly disable event-driven orchestration for Qwen3-TTS:
```bash
VLLM_OMNI_EVENT_DRIVEN_ORCH=1 \
VLLM_OMNI_EVENT_DRIVEN_ORCH=0 \
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-Base \
--omni \
--port 8091
@@ -983,8 +984,9 @@ The server logs the selected loop mode and its reader/poller counts once at
startup, so you can confirm which loop is live.
Routing, output ordering, and terminal-state behavior are identical on both
loops; only the poll cadence changes. Leaving the variable unset keeps the
legacy poll loop, which is the supported default.
loops; only the poll cadence changes. Leaving the variable unset selects the
pipeline default: event-driven for Qwen3-TTS, and legacy polling for other
pipelines, including Qwen3-Omni.
**Known limitations:**