anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[CI]Remove SoulX-Singer support (#6362)

Browse Source 
Signed-off-by: wangyu <410167048@qq.com>
This commit is contained in:
wangyu authored and GitHub committed 2026-08-23 07:16:32 +08:00
1 parent b2c9376539
commit 2e096788a6
51 files changed
+1 -9575
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

examples/online_serving/text_to_speech/README.md
-63
View File
@@ -25,7 +25,6 @@ For the full list of supported architectures across all modalities, see
| Qwen3-TTS | `Qwen/Qwen3-TTS-12Hz-1.7B-{CustomVoice,VoiceDesign,Base}` | ✓ (Base) | ✓ (PCM + WebSocket) | ✓ (presets + `/v1/audio/voices` upload) | ✓ (standard + FastRTC) |
| VoxCPM2 | `openbmb/VoxCPM2` | ✓ | ✓ (AudioWorklet via gradio) | — | ✓ |
| Voxtral TTS | `mistralai/Voxtral-4B-TTS-2603` | ✓ (gated upstream) | ✓ | ✓ (presets) | ✓ |
| SoulX-Singer | `Soul-AILab/SoulX-Singer` | ✓ (prompt audio) | — (batch only) | — (prompt + target audio) | — (chat client) |
CosyVoice3 is intentionally absent: no online example exists for it yet. See its [offline section](../../offline_inference/text_to_speech/README.md#cosyvoice3) instead.
@@ -715,65 +714,3 @@ The demo handles voice-preset selection and reference-audio upload. `voxtral_tts
- Voice presets are listed on the HF model card (`mistralai/Voxtral-4B-TTS-2603`).
- Voice cloning is gated upstream and may require a recent `mistral_common`.
- A standalone CLI client is not yet shipped; the gradio demo is the canonical reference for now.
---
## SoulX-Singer
Singing voice synthesis (SVS) and conversion (SVC) at 24 kHz. Single-stage DiT with inline preprocess. Uses the `/v1/chat/completions` endpoint with multimodal input (`prompt_audio` + `target_audio`).
### Prerequisites
Download DiT and preprocess weights, then set up separate SVS / SVC view directories and install dependencies as described in the [offline README](../../offline_inference/text_to_speech/README.md#soulx-singer). `config.json` `architectures` field is the single source of truth for SVS vs SVC — point `MODEL` at the matching directory.
### Launch
```bash
# SVS (default)
export MODEL=/path/to/SoulX-Singer
export PREPROCESS=/path/to/SoulX-Singer-Preprocess
bash examples/online_serving/text_to_speech/soulxsinger/run_server.sh
# SVC
export MODE=svc
export MODEL=/path/to/SoulX-Singer-svc
bash examples/online_serving/text_to_speech/soulxsinger/run_server.sh
```
Or equivalently, set `SOULX_PREPROCESS_WEIGHTS_DIR` and launch directly:
```bash
export SOULX_PREPROCESS_WEIGHTS_DIR=$PREPROCESS
vllm serve $MODEL --omni \
--deploy-config vllm_omni/deploy/soulxsinger_${MODE}.yaml \
--port 8192 --trust-remote-code --enforce-eager
```
### Sending requests
Audio paths must be reachable from the server host (local filesystem or data URL). The client sends prompt vocal via `input_audio` and target accompaniment via `extra_args['target_audio']`.
```bash
# Default demo audio: tests/assets/soulxsinger/zh_prompt.mp3 + music.mp3
python examples/online_serving/text_to_speech/soulxsinger/openai_chat_client.py \
--prompt-audio /path/on/server/zh_prompt.mp3 \
--target-audio /path/on/server/music.mp3 \
--preprocess-weights-dir /path/on/server/SoulX-Singer-Preprocess \
-o output.wav
```
Use precomputed metadata to skip online preprocess with following command:
```bash
python examples/online_serving/text_to_speech/soulxsinger/openai_chat_client.py \
--prompt-metadata-path /path/on/server/zh_prompt.json \
--target-metadata-path /path/on/server/music.json \
--audio-path /path/on/server/zh_prompt.mp3 \
-o output.wav
```
`SOULX_PREPROCESS_WEIGHTS_DIR` makes `--preprocess-weights-dir` optional. See `openai_chat_client.py --help` for `--vocal-sep`, `--language`, `--num-inference-steps`, `--guidance-scale`, and `--seed`.
### Notes
- Output: 24 kHz mono WAV; batch only.
- Defaults match upstream: `--guidance-scale 3.0`, `--seed 42`, `--auto-shift` on.
- SVS `--control`: `score` or `melody`. MIDI / lyric QC: upstream `midi_editor` only.
examples/online_serving/text_to_speech/soulxsinger/openai_chat_client.py
-186
View File
@@ -1,186 +0,0 @@
#!/usr/bin/env python3
"""SoulX-Singer OpenAI-compatible chat client (SVS / SVC).
Sends prompt audio via ``input_audio`` and target accompaniment via
``extra_args['target_audio']`` (server-local path). For integrated preprocess,
also pass ``preprocess_weights_dir`` in ``extra_args``.
Usage:
python openai_chat_client.py \\
--prompt-audio /path/on/server/zh_prompt.mp3 \\
--target-audio /path/on/server/music.mp3 \\
--preprocess-weights-dir /path/on/server/SoulX-Singer-Preprocess \\
-o output.wav
"""
from __future__ import annotations
import argparse
import base64
import io
import sys
from pathlib import Path
import requests
import soundfile
import torch
def _audio_to_data_url(path: Path) -> str:
with path.open("rb") as handle:
data = base64.b64encode(handle.read()).decode("ascii")
return f"data:audio/mpeg;base64,{data}"
def _save_wav(audio: torch.Tensor, path: Path, sample_rate: int) -> None:
audio = audio.to(torch.float32)
peak = audio.abs().max().clamp(min=1e-8)
audio = audio / peak
path.parent.mkdir(parents=True, exist_ok=True)
soundfile.write(str(path), audio.clamp(-1.0, 1.0).cpu().T.numpy(), sample_rate, subtype="PCM_16")
def _decode_audio_from_response(body: dict) -> tuple[torch.Tensor, int]:
for choice in body.get("choices", []):
audio_obj = choice.get("message", {}).get("audio")
if isinstance(audio_obj, dict) and audio_obj.get("data"):
data, sr = soundfile.read(
io.BytesIO(base64.b64decode(audio_obj["data"])),
dtype="float32",
always_2d=True,
)
return torch.from_numpy(data).transpose(0, 1), sr
brief = {k: v for k, v in body.items() if k != "choices"}
raise RuntimeError(f"no audio in response message.audio: {brief}")
def main() -> int:
repo_root = Path(__file__).resolve().parents[4]
default_assets = repo_root / "tests" / "assets" / "soulxsinger"
parser = argparse.ArgumentParser(description="SoulX-Singer online chat client")
parser.add_argument("--port", type=int, default=8192)
parser.add_argument("--model", default="Soul-AILab/SoulX-Singer")
parser.add_argument(
"--prompt-audio",
default=str(default_assets / "zh_prompt.mp3"),
help="Prompt vocal audio (path on server if using extra_args, or local for input_audio)",
)
parser.add_argument(
"--target-audio",
default=str(default_assets / "music.mp3"),
help="Target accompaniment path on the server (extra_args['target_audio'])",
)
parser.add_argument(
"--prompt-metadata-path",
default=None,
help="SVS precomputed prompt metadata.json",
)
parser.add_argument(
"--target-metadata-path",
default=None,
help="SVS precomputed target metadata.json",
)
parser.add_argument(
"--audio-path",
default=None,
help="SVS prompt vocal wav for precomputed metadata",
)
parser.add_argument("--preprocess-weights-dir", default=None)
parser.add_argument("--output", "-o", default="soulxsinger_out.wav")
parser.add_argument("--svc", action="store_true", help="Use SVC mode knobs")
parser.add_argument("--language", default="Mandarin")
parser.add_argument("--num-inference-steps", type=int, default=32)
parser.add_argument("--guidance-scale", type=float, default=3.0)
parser.add_argument(
"--seed",
type=int,
default=42,
help="Optional CFM seed. Omit for non-deterministic sampling.",
)
parser.add_argument(
"--auto-shift",
action=argparse.BooleanOptionalAction,
default=True,
help="Auto pitch shift (default: on, original upstream infer.sh)",
)
parser.add_argument(
"--control",
default="melody",
choices=["melody", "score"],
help="SVS control mode",
)
parser.add_argument("--vocal-sep", action="store_true")
args = parser.parse_args()
meta_paths = (args.prompt_metadata_path, args.target_metadata_path, args.audio_path)
if any(meta_paths) and not all(meta_paths):
print(
"ERROR: precomputed metadata requires --prompt-metadata-path, "
"--target-metadata-path, and --audio-path together.",
file=sys.stderr,
)
return 2
extra_args: dict = {
"vocal_sep": args.vocal_sep,
"auto_shift": args.auto_shift,
"pitch_shift": 0,
}
if all(meta_paths):
extra_args.update(
{
"prompt_metadata_path": str(Path(args.prompt_metadata_path).expanduser().resolve()),
"target_metadata_path": str(Path(args.target_metadata_path).expanduser().resolve()),
"audio_path": str(Path(args.audio_path).expanduser().resolve()),
}
)
content = [{"type": "text", "text": "soulx-singer"}]
else:
prompt_path = Path(args.prompt_audio).expanduser().resolve()
if not prompt_path.is_file():
print(f"ERROR: prompt audio not found: {prompt_path}", file=sys.stderr)
return 2
extra_args["prompt_audio"] = str(prompt_path)
extra_args["target_audio"] = str(Path(args.target_audio).expanduser().resolve())
if args.preprocess_weights_dir:
extra_args["preprocess_weights_dir"] = str(Path(args.preprocess_weights_dir).expanduser().resolve())
content = [
{"type": "text", "text": "soulx-singer"},
{
"type": "input_audio",
"input_audio": {"data": _audio_to_data_url(prompt_path), "format": "mp3"},
},
]
if not args.svc:
extra_args["language"] = args.language
extra_args["control"] = args.control
payload = {
"model": args.model,
"modalities": ["audio"],
"messages": [{"role": "user", "content": content}],
"num_inference_steps": args.num_inference_steps,
"guidance_scale": args.guidance_scale,
"extra_args": extra_args,
}
if args.seed is not None:
payload["seed"] = args.seed
print(f"POST http://localhost:{args.port}/v1/chat/completions")
response = requests.post(
f"http://localhost:{args.port}/v1/chat/completions",
headers={"Content-Type": "application/json"},
json=payload,
timeout=1800,
)
response.raise_for_status()
audio, sample_rate = _decode_audio_from_response(response.json())
_save_wav(audio, Path(args.output), sample_rate)
duration = audio.shape[-1] / sample_rate
print(f"saved {args.output} sr={sample_rate}Hz duration={duration:.2f}s")
return 0
if __name__ == "__main__":
raise SystemExit(main())
examples/online_serving/text_to_speech/soulxsinger/run_server.sh
-39
View File
@@ -1,39 +0,0 @@
#!/bin/bash
# Launch vLLM-Omni server for SoulX-Singer (single-stage DiT, preprocess inline).
#
# Usage:
# MODEL=/path/to/SoulX-Singer PREPROCESS=/path/to/Preprocess \
#
# Audio paths in client extra_args must be readable on the server host.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../../../.." && pwd)"
MODEL="${MODEL:-Soul-AILab/SoulX-Singer}"
MODE="${MODE:-svs}"
PORT="${PORT:-8192}"
GPUS="${GPUS:-0}"
if [[ "$MODE" == "svc" ]]; then
DEPLOY_CONFIG="${DEPLOY_CONFIG:-$REPO_ROOT/vllm_omni/deploy/soulxsinger_svc.yaml}"
else
DEPLOY_CONFIG="${DEPLOY_CONFIG:-$REPO_ROOT/vllm_omni/deploy/soulxsinger_svs.yaml}"
fi
echo "Starting SoulX-Singer server"
echo " MODEL=$MODEL"
echo " MODE=$MODE"
echo " PORT=$PORT"
echo " DEPLOY_CONFIG=$DEPLOY_CONFIG"
echo " CUDA_VISIBLE_DEVICES=$GPUS"
CUDA_VISIBLE_DEVICES="$GPUS" \
vllm serve "$MODEL" \
--omni \
--deploy-config "$DEPLOY_CONFIG" \
--host 0.0.0.0 \
--port "$PORT" \
--trust-remote-code \
--enforce-eager