Rebase multi-API serving on latest main
Signed-off-by: Sy03 <1370724210@qq.com>
This commit is contained in:
1 parent
78934753ff
commit
dc3c7dbe16
25 files changed
+1832
-202
No files matched your search
@@ -357,6 +357,22 @@ vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --omni --port 8091
|
||||
./qwen3_tts/run_server.sh Base
|
||||
```
|
||||
|
||||
For a local deployment with multiple API frontend processes sharing one set
|
||||
of TTS stage engines, add `--api-server-count`:
|
||||
|
||||
```bash
|
||||
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
|
||||
--omni --api-server-count 2 --port 8091
|
||||
```
|
||||
|
||||
This mode supports local EngineCore stages with one local process group per
|
||||
replica, including stages configured with multiple `num_replicas`. Headless,
|
||||
remote, intra-replica data-parallel, and Ray stage deployments are not
|
||||
supported. Runtime voice upload and deletion are disabled with multiple
|
||||
API frontends because their registries are process-local. Built-in voices,
|
||||
inline `ref_audio`, and voices restored from `custom_voice_dir` at startup are
|
||||
supported.
|
||||
|
||||
### Executor backend
|
||||
|
||||
Single-GPU serves now default to the uniproc executor (lower IPC overhead, the Base cloning use case from [#2603](https://github.com/vllm-project/vllm-omni/issues/2603) / [#2604](https://github.com/vllm-project/vllm-omni/pull/2604)). `vllm_omni/deploy/qwen3_tts.yaml` is the only Qwen3-TTS deploy config; pass `--deploy-config <path>` to override.
|
||||
@@ -422,6 +438,10 @@ checkpoint. When a request names an uploaded voice, the server infers
|
||||
`task_type="Base"`. Built-in presets such as `vivian` and `ryan` remain
|
||||
CustomVoice speakers and require a CustomVoice checkpoint.
|
||||
|
||||
The runtime upload and delete routes require a single API frontend; with
|
||||
`--api-server-count > 1`, use inline `ref_audio` or restore precomputed voices
|
||||
from `custom_voice_dir` at startup.
|
||||
|
||||
### Precomputed custom voices
|
||||
|
||||
For reused Base voice-cloning speakers, precompute the reference artifacts once
|
||||
|
||||
Reference in new issue
Block a user