[Config Refactor] Migrate 5 TTS models (VoxCPM2 / CosyVoice3 / MiMo Audio / Voxtral TTS / Fish Speech S2 Pro) to Pipeline + Deploy schema (#2958)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com> Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
This commit is contained in:
1 parent
c2578f7131
commit
3f504b45d2
47 files changed
+702
-844
No files matched your search
@@ -29,15 +29,12 @@ Features:
|
||||
## Launch the Server
|
||||
|
||||
```bash
|
||||
vllm-omni serve fishaudio/s2-pro \
|
||||
--stage-configs-path vllm_omni/model_executor/stage_configs/fish_speech_s2_pro.yaml \
|
||||
--omni \
|
||||
--port 8091 \
|
||||
--trust-remote-code \
|
||||
--enforce-eager \
|
||||
--gpu-memory-utilization 0.9
|
||||
vllm serve fishaudio/s2-pro --omni --port 8091
|
||||
```
|
||||
|
||||
The deploy config is auto-loaded from `vllm_omni/deploy/fish_qwen3_omni.yaml`
|
||||
(the HF `model_type` on the fishaudio checkpoints is `fish_qwen3_omni`).
|
||||
|
||||
Or use the convenience script:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -11,18 +11,13 @@ MODEL="${MODEL:-fishaudio/s2-pro}"
|
||||
PORT="${PORT:-8091}"
|
||||
GRADIO_PORT="${GRADIO_PORT:-7860}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
|
||||
|
||||
echo "Starting Fish Speech S2 Pro server (port $PORT)..."
|
||||
FLASHINFER_DISABLE_VERSION_CHECK=1 \
|
||||
vllm-omni serve "$MODEL" \
|
||||
--stage-configs-path "$REPO_ROOT/vllm_omni/model_executor/stage_configs/fish_speech_s2_pro.yaml" \
|
||||
vllm serve "$MODEL" \
|
||||
--omni \
|
||||
--host 0.0.0.0 \
|
||||
--port "$PORT" \
|
||||
--gpu-memory-utilization 0.9 \
|
||||
--trust-remote-code \
|
||||
--enforce-eager \
|
||||
--omni &
|
||||
--port "$PORT" &
|
||||
SERVER_PID=$!
|
||||
|
||||
cleanup() {
|
||||
|
||||
@@ -13,11 +13,7 @@ PORT="${PORT:-8091}"
|
||||
echo "Starting Fish Speech S2 Pro server with model: $MODEL"
|
||||
|
||||
FLASHINFER_DISABLE_VERSION_CHECK=1 \
|
||||
vllm-omni serve "$MODEL" \
|
||||
--stage-configs-path vllm_omni/model_executor/stage_configs/fish_speech_s2_pro.yaml \
|
||||
vllm serve "$MODEL" \
|
||||
--omni \
|
||||
--host 0.0.0.0 \
|
||||
--port "$PORT" \
|
||||
--gpu-memory-utilization 0.9 \
|
||||
--trust-remote-code \
|
||||
--enforce-eager \
|
||||
--omni
|
||||
--port "$PORT"
|
||||
@@ -13,10 +13,10 @@ Please refer to [README.md](../../../README.md)
|
||||
```bash
|
||||
export MIMO_AUDIO_TOKENIZER_PATH="XiaomiMiMo/MiMo-Audio-Tokenizer"
|
||||
|
||||
vllm-omni serve XiaomiMiMo/MiMo-Audio-7B-Instruct --omni \
|
||||
--served-model-name "MiMo-Audio-7B-Instruct" \
|
||||
--port 18091 --stage-configs-path ./vllm_omni/model_executor/stage_configs/mimo_audio.yaml \
|
||||
--chat-template ./examples/online_serving/mimo_audio/chat_template.jinja
|
||||
vllm serve XiaomiMiMo/MiMo-Audio-7B-Instruct --omni \
|
||||
--served-model-name "MiMo-Audio-7B-Instruct" \
|
||||
--port 18091 \
|
||||
--chat-template ./examples/online_serving/mimo_audio/chat_template.jinja
|
||||
```
|
||||
> ⚠️ **Important**
|
||||
> **MiMo-Audio is not compatible with the default chat template.**
|
||||
|
||||
@@ -5,12 +5,13 @@ Serve VoxCPM2 TTS via the OpenAI-compatible `/v1/audio/speech` endpoint.
|
||||
## Start the Server
|
||||
|
||||
```bash
|
||||
python -m vllm_omni.entrypoints.openai.api_server \
|
||||
--model openbmb/VoxCPM2 \
|
||||
--stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm2.yaml \
|
||||
--host 0.0.0.0 --port 8000
|
||||
vllm serve openbmb/VoxCPM2 --omni --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
The deploy config is auto-loaded from `vllm_omni/deploy/voxcpm2.yaml`. Pass
|
||||
`--deploy-config <path>` to override, or `--stage-N-<field> <value>` (e.g.
|
||||
`--stage-0-max-num-seqs 8`) for per-stage runtime overrides.
|
||||
|
||||
## Zero-shot Synthesis
|
||||
|
||||
```bash
|
||||
|
||||
@@ -8,10 +8,7 @@ the audio clock rate.
|
||||
|
||||
Usage:
|
||||
# Start the vLLM server first:
|
||||
python -m vllm_omni.entrypoints.openai.api_server \
|
||||
--model openbmb/VoxCPM2 \
|
||||
--stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm2.yaml \
|
||||
--host 0.0.0.0 --port 8000
|
||||
vllm serve openbmb/VoxCPM2 --omni --host 0.0.0.0 --port 8000
|
||||
|
||||
# Then launch the demo:
|
||||
python gradio_demo.py --api-base http://localhost:8000
|
||||
|
||||
@@ -13,10 +13,7 @@ Examples:
|
||||
--ref-audio "https://example.com/reference.wav"
|
||||
|
||||
Server setup:
|
||||
python -m vllm_omni.entrypoints.openai.api_server \
|
||||
--model openbmb/VoxCPM2 \
|
||||
--stage-configs-path vllm_omni/model_executor/stage_configs/voxcpm2.yaml \
|
||||
--host 0.0.0.0 --port 8000
|
||||
vllm serve openbmb/VoxCPM2 --omni --host 0.0.0.0 --port 8000
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
Reference in new issue
Block a user