vllm-omni/recipes/fishaudio/Fish-Speech-S2-Pro.md

Fish Speech S2 Pro

Online serving for TTS

Summary

• Vendor: FishAudio
• Model: fishaudio/s2-pro
• Task: Text-to-speech synthesis with optional voice cloning
• Mode: Online serving with the OpenAI-compatible /v1/audio/speech API
• Maintainer: Community

When to use this recipe

Use this recipe as a practical baseline for running fishaudio/s2-pro for high-quality text-to-speech synthesis. Fish Speech S2 Pro outputs 44.1 kHz audio and supports voice cloning from reference audio.

References

• User guide: docs/user_guide/examples/online_serving/fish_speech.md
• Example guide: examples/online_serving/fish_speech/README.md
• Related issue or discussion: RFC: add recipes folder

Hardware Support

This recipe documents reference GPU configuration for Fish Speech S2 Pro. Other hardware and configurations are welcome as community validation lands.

GPU

1x A800 80GB

Environment

• OS: Linux
• Python: 3.10+
• CUDA 12.8
• vLLM version: 0.19.0
• vLLM-Omni version or commit: c93359bb35

No extra packages are required: the DAC codec architecture is vendored in vLLM-Omni (vllm_omni/model_executor/models/fish_speech/dac_modules/), so the model runs out of the box, including in the official Docker image.

Kvcache attention fast path

Fish Speech S2 Pro uses a Triton decode-only kvcache attention fast path by default on CUDA builds. Set VLLM_OMNI_FISH_KVCACHE_ATTN=0 to disable it, or VLLM_OMNI_FISH_KVCACHE_ATTN=required to fail fast if the fast path cannot be installed.

# Verify fast path availability.
python - <<'PY'
from vllm_omni.attention import fish_kvcache_attn

print(fish_kvcache_attn.is_available())
print(fish_kvcache_attn.load_error())
PY

# Optional: disable the runtime fast path.
export VLLM_OMNI_FISH_KVCACHE_ATTN=0

Command

vllm serve fishaudio/s2-pro --omni --port 8091

Notes:

• --omni is required.
• The default deploy config vllm_omni/deploy/fish_qwen3_omni.yaml is loaded automatically by model registry (HF model_type=fish_qwen3_omni).

Verification

Basic TTS:

curl -X POST http://localhost:8091/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{
        "input": "Hello, how are you?",
        "voice": "default",
        "response_format": "wav"
    }' --output output.wav

output.wav

Voice cloning:

curl -X POST http://localhost:8091/v1/audio/speech \
    -H "Content-Type: application/json" \
    -d '{
        "input": "Hello, this is a cloned voice.",
        "voice": "default",
        "ref_audio": "https://example.com/reference.wav",
        "ref_text": "Transcript of the reference audio."
    }' --output cloned.wav

reference.wav
cloned.wav

Notes

• Key flags: --omni is required.
• Known limitations: Output audio is 44.1 kHz mono WAV. Voice cloning requires both ref_audio and ref_text parameters.
• Memory usage: Model loads at ~48.3 GiB, peaks at ~48.9 GiB during inference headroom for video frames and audio caches.

2x H100 80GB

Environment

• OS: Linux
• Python: 3.10+
• CUDA 12.8
• vLLM version: 0.19.0+

Command

CUDA_VISIBLE_DEVICES=0,1 vllm serve fishaudio/s2-pro --omni --port 8091 \
    --deploy-config vllm_omni/deploy/fish_qwen3_omni_2gpu.yaml

This profile pins Stage0 (Slow/Fast AR) to GPU 0 and Stage1 (DAC decoder) to GPU 1 to remove AR/DAC contention, sets max_num_seqs=64 on both stages so concurrencies above 8 don't queue, and reduces connector_get_sleep_s from 10 ms to 1 ms so DAC chunks aren't held in the connector once they're ready.

For benchmark numbers see the discussion in vllm-project/vllm-omni#2515 (initial H100x2 sweep by @Sy0307) and vllm-project/vllm-omni#3323 (H20x2 sweep with the shipped config).