Sy03andYueqian Lin
d37912b071
Signed-off-by: Sy03 <1370724210@qq.com> Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
152 lines
4.3 KiB
Markdown
152 lines
4.3 KiB
Markdown
# Fish Speech S2 Pro
|
|
|
|
> Online serving for TTS
|
|
|
|
## Summary
|
|
|
|
- Vendor: FishAudio
|
|
- Model: `fishaudio/s2-pro`
|
|
- Task: Text-to-speech synthesis with optional voice cloning
|
|
- Mode: Online serving with the OpenAI-compatible `/v1/audio/speech` API
|
|
- Maintainer: Community
|
|
|
|
## When to use this recipe
|
|
|
|
Use this recipe as a practical baseline for running `fishaudio/s2-pro` for
|
|
high-quality text-to-speech synthesis. Fish Speech S2 Pro outputs 44.1 kHz
|
|
audio and supports voice cloning from reference audio.
|
|
|
|
## References
|
|
|
|
- User guide: [`docs/user_guide/examples/online_serving/fish_speech.md`](../../docs/user_guide/examples/online_serving/fish_speech.md)
|
|
- Example guide: [`examples/online_serving/fish_speech/README.md`](../../examples/online_serving/fish_speech/README.md)
|
|
- Related issue or discussion:
|
|
[RFC: add recipes folder](https://github.com/vllm-project/vllm-omni/issues/2645)
|
|
|
|
## Hardware Support
|
|
|
|
This recipe documents reference GPU configuration for Fish Speech S2 Pro.
|
|
Other hardware and configurations are welcome as community validation lands.
|
|
|
|
## GPU
|
|
|
|
### 1x A800 80GB
|
|
|
|
#### Environment
|
|
|
|
- OS: Linux
|
|
- Python: 3.10+
|
|
- Required package: `fish-speech` (for DAC codec)
|
|
- CUDA 12.8
|
|
- vLLM version: 0.19.0
|
|
- vLLM-Omni version or commit: c93359bb354a6aa5c14d062430cb85b2c4db251e
|
|
|
|
```bash
|
|
# Install PortAudio dependency if missing (required by pyaudio which is a dependency of fish-speech)
|
|
# For Ubuntu/Debian:
|
|
if ! dpkg -l | grep -q libportaudio2; then
|
|
sudo apt-get update && sudo apt-get install -y libportaudio2 portaudio19-dev
|
|
fi
|
|
pip install fish-speech
|
|
```
|
|
|
|
#### Kvcache attention fast path
|
|
|
|
Fish Speech S2 Pro uses a Triton decode-only kvcache attention fast path by
|
|
default on CUDA builds. Set `VLLM_OMNI_FISH_KVCACHE_ATTN=0` to disable it, or
|
|
`VLLM_OMNI_FISH_KVCACHE_ATTN=required` to fail fast if the fast path cannot be
|
|
installed.
|
|
|
|
```bash
|
|
# Verify fast path availability.
|
|
python - <<'PY'
|
|
from vllm_omni.attention import fish_kvcache_attn
|
|
|
|
print(fish_kvcache_attn.is_available())
|
|
print(fish_kvcache_attn.load_error())
|
|
PY
|
|
|
|
# Optional: disable the runtime fast path.
|
|
export VLLM_OMNI_FISH_KVCACHE_ATTN=0
|
|
```
|
|
|
|
#### Command
|
|
|
|
```bash
|
|
vllm serve fishaudio/s2-pro --omni --port 8091
|
|
```
|
|
|
|
Notes:
|
|
|
|
- `--omni` is required.
|
|
- The default deploy config `vllm_omni/deploy/fish_qwen3_omni.yaml` is loaded
|
|
automatically by model registry (HF `model_type=fish_qwen3_omni`).
|
|
|
|
|
|
#### Verification
|
|
|
|
Basic TTS:
|
|
|
|
```bash
|
|
curl -X POST http://localhost:8091/v1/audio/speech \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"input": "Hello, how are you?",
|
|
"voice": "default",
|
|
"response_format": "wav"
|
|
}' --output output.wav
|
|
```
|
|
[output.wav](https://github.com/user-attachments/files/27134970/output.wav)
|
|
|
|
Voice cloning:
|
|
|
|
```bash
|
|
curl -X POST http://localhost:8091/v1/audio/speech \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"input": "Hello, this is a cloned voice.",
|
|
"voice": "default",
|
|
"ref_audio": "https://example.com/reference.wav",
|
|
"ref_text": "Transcript of the reference audio."
|
|
}' --output cloned.wav
|
|
```
|
|
[reference.wav](https://github.com/user-attachments/files/27134971/reference.wav) <br>
|
|
[cloned.wav](https://github.com/user-attachments/files/27134969/cloned.wav)
|
|
|
|
|
|
|
|
#### Notes
|
|
|
|
- Key flags: `--omni` is required.
|
|
- Known limitations: Output audio is 44.1 kHz mono WAV. Voice cloning requires
|
|
both `ref_audio` and `ref_text` parameters.
|
|
- Memory usage: Model loads at ~48.3 GiB, peaks at ~48.9 GiB during inference
|
|
headroom for video frames and audio caches.
|
|
|
|
### 2x H100 80GB
|
|
|
|
#### Environment
|
|
|
|
- OS: Linux
|
|
- Python: 3.10+
|
|
- Required package: `fish-speech` (for DAC codec)
|
|
- CUDA 12.8
|
|
- vLLM version: 0.19.0+
|
|
|
|
#### Command
|
|
|
|
```bash
|
|
CUDA_VISIBLE_DEVICES=0,1 vllm serve fishaudio/s2-pro --omni --port 8091 \
|
|
--deploy-config vllm_omni/deploy/fish_qwen3_omni_2gpu.yaml
|
|
```
|
|
|
|
This profile pins Stage0 (Slow/Fast AR) to GPU 0 and Stage1 (DAC decoder)
|
|
to GPU 1 to remove AR/DAC contention, sets `max_num_seqs=64` on both
|
|
stages so concurrencies above 8 don't queue, and reduces
|
|
`connector_get_sleep_s` from 10 ms to 1 ms so DAC chunks aren't held in
|
|
the connector once they're ready.
|
|
|
|
For benchmark numbers see the discussion in
|
|
vllm-project/vllm-omni#2515 (initial H100x2 sweep by @Sy0307) and
|
|
vllm-project/vllm-omni#3323 (H20x2 sweep with the shipped config).
|