Rebase multi-API serving on latest main

Signed-off-by: Sy03 <1370724210@qq.com>
This commit is contained in:
Sy03 committed 2026-09-16 12:13:04 +08:00
1 parent 78934753ff
commit dc3c7dbe16
25 files changed
+1832 -202

No files matched your search

@@ -357,6 +357,22 @@ vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --omni --port 8091
./qwen3_tts/run_server.sh Base
```
For a local deployment with multiple API frontend processes sharing one set
of TTS stage engines, add `--api-server-count`:
```bash
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
--omni --api-server-count 2 --port 8091
```
This mode supports local EngineCore stages with one local process group per
replica, including stages configured with multiple `num_replicas`. Headless,
remote, intra-replica data-parallel, and Ray stage deployments are not
supported. Runtime voice upload and deletion are disabled with multiple
API frontends because their registries are process-local. Built-in voices,
inline `ref_audio`, and voices restored from `custom_voice_dir` at startup are
supported.
### Executor backend
Single-GPU serves now default to the uniproc executor (lower IPC overhead, the Base cloning use case from [#2603](https://github.com/vllm-project/vllm-omni/issues/2603) / [#2604](https://github.com/vllm-project/vllm-omni/pull/2604)). `vllm_omni/deploy/qwen3_tts.yaml` is the only Qwen3-TTS deploy config; pass `--deploy-config <path>` to override.
@@ -422,6 +438,10 @@ checkpoint. When a request names an uploaded voice, the server infers
`task_type="Base"`. Built-in presets such as `vivian` and `ryan` remain
CustomVoice speakers and require a CustomVoice checkpoint.
The runtime upload and delete routes require a single API frontend; with
`--api-server-count > 1`, use inline `ref_audio` or restore precomputed voices
from `custom_voice_dir` at startup.
### Precomputed custom voices
For reused Base voice-cloning speakers, precompute the reference artifacts once