[Doc] Stability AI/Stable-Audio-Open (#3664)
Signed-off-by: CrazyCoder <18235787078@163.com>
This commit is contained in:
CrazyCoder authored and GitHub committed
1 parent
9c260b3774
commit
378359fc11
2 files changed
+171
No files matched your search
@@ -58,6 +58,7 @@ recipes/
|
||||
| [`Qwen/Qwen3-Omni.md`](./Qwen/Qwen3-Omni.md) | Online serving for multimodal chat | 1x A100 80GB |
|
||||
| [`Qwen/Qwen3-TTS.md`](./Qwen/Qwen3-TTS.md) | Text-to-speech serving (CustomVoice / VoiceDesign / Base) | 1x H100/A100 80GB |
|
||||
| [`SenseNova/SenseNova-U1.md`](./SenseNova/SenseNova-U1.md) | Unified image generation and understanding | 1x H200 (144GB) |
|
||||
| [`StabilityAI/Stable-Audio-Open.md`](./StabilityAI/Stable-Audio-Open.md) | Offline + online text-to-audio generation (Stable Audio Open) | 1x RTX 4090 24GB |
|
||||
| [`Tencent/Covo-Audio-Chat.md`](./Tencent/Covo-Audio-Chat.md) | Online serving for audio chat | 1x A100 80GB |
|
||||
| [`Tencent/HunyuanImage-3.0-Instruct.md`](./Tencent/HunyuanImage-3.0-Instruct.md) | DiT-only text-to-image serving and benchmark, including ModelOpt mixed FP8/NVFP4 | 4x H100/H800 80GB / 2x B200 |
|
||||
| [`Wan-AI/Wan2.2-I2V.md`](./Wan-AI/Wan2.2-I2V.md) | Image-to-video serving (Wan2.2 14B) | 8x Ascend NPU (A2/A3) |
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
# Stable-Audio-Open Text-To-Audio Generation on 1x GPU
|
||||
|
||||
> Text-to-audio recipe for Stable Audio Open with offline inference and
|
||||
> OpenAI-compatible online serving on 1x RTX 4090 24GB.
|
||||
|
||||
## Summary
|
||||
|
||||
- Vendor: Stability AI
|
||||
- Model: `stabilityai/stable-audio-open-1.0`
|
||||
- Task: Text-to-audio generation
|
||||
- Mode: Offline inference and online serving
|
||||
- Maintainer: Community
|
||||
|
||||
## When to use this recipe
|
||||
|
||||
Use this recipe when you want to run Stable Audio Open on a single RTX 4090
|
||||
24GB GPU for music or sound-effect generation. The recipe covers a 10-second
|
||||
offline validation sample with TeaCache and online serving through the
|
||||
`/v1/audio/generate` endpoint.
|
||||
|
||||
## References
|
||||
|
||||
- Model: <https://huggingface.co/stabilityai/stable-audio-open-1.0>
|
||||
- Offline example:
|
||||
[`examples/offline_inference/text_to_audio`](../../examples/offline_inference/text_to_audio)
|
||||
- Online example:
|
||||
[`examples/online_serving/stable_audio`](../../examples/online_serving/stable_audio)
|
||||
- Audio generation API:
|
||||
[`docs/serving/audio_generate_api.md`](../../docs/serving/audio_generate_api.md)
|
||||
- Related issue:
|
||||
[#2645](https://github.com/vllm-project/vllm-omni/issues/2645)
|
||||
|
||||
## Hardware Support
|
||||
|
||||
### GPU
|
||||
|
||||
#### 1x RTX 4090 24GB
|
||||
|
||||
##### Environment
|
||||
|
||||
- OS: Ubuntu 22.04.5
|
||||
- Python: 3.12
|
||||
- GPU: NVIDIA GeForce RTX 4090, 24564 MiB VRAM
|
||||
- Driver / runtime: NVIDIA driver 595.80, CUDA-capable runtime matching the
|
||||
repository build
|
||||
- vLLM version: 0.22.0
|
||||
- vLLM-Omni version: source checkout
|
||||
- PyTorch: 2.11.0+cu130
|
||||
- Model path used in the commands below: `/path/to/stable-audio-open-1.0`
|
||||
|
||||
Stable Audio Open is a gated Hugging Face model. Accept the model license on
|
||||
the Hugging Face model card before downloading the checkpoint.
|
||||
|
||||
```bash
|
||||
hf auth login
|
||||
|
||||
hf download stabilityai/stable-audio-open-1.0 \
|
||||
--local-dir /path/to/stable-audio-open-1.0
|
||||
```
|
||||
|
||||
##### Commands
|
||||
|
||||
Run a 10-second offline validation sample from the repository root:
|
||||
|
||||
```bash
|
||||
python examples/offline_inference/text_to_audio/text_to_audio.py \
|
||||
--model /path/to/stable-audio-open-1.0 \
|
||||
--prompt "A gentle piano melody with soft room ambience" \
|
||||
--negative-prompt "Low quality, distorted, noisy" \
|
||||
--seed 42 \
|
||||
--guidance-scale 7.0 \
|
||||
--audio-length 10.0 \
|
||||
--num-inference-steps 50 \
|
||||
--cache-backend tea_cache \
|
||||
--output examples/offline_inference/text_to_audio/stable_audio_10s.wav
|
||||
```
|
||||
|
||||
Start the online serving endpoint:
|
||||
|
||||
```bash
|
||||
vllm-omni serve /path/to/stable-audio-open-1.0 \
|
||||
--host 0.0.0.0 \
|
||||
--port 8091 \
|
||||
--gpu-memory-utilization 0.9 \
|
||||
--trust-remote-code \
|
||||
--enforce-eager \
|
||||
--omni
|
||||
```
|
||||
|
||||
Generate a 10-second WAV file from the repository root in another terminal:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8091/health
|
||||
|
||||
curl -X POST http://localhost:8091/v1/audio/generate \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"input": "A gentle piano melody with soft room ambience",
|
||||
"audio_length": 10.0,
|
||||
"num_inference_steps": 50,
|
||||
"guidance_scale": 7.0,
|
||||
"negative_prompt": "Low quality, distorted, noisy",
|
||||
"seed": 42,
|
||||
"response_format": "wav"
|
||||
}' \
|
||||
--output examples/online_serving/stable_audio/piano_10s.wav
|
||||
```
|
||||
|
||||
##### Verification
|
||||
|
||||
Check that:
|
||||
|
||||
- The offline command writes a valid WAV file.
|
||||
- The server responds on `http://localhost:8091/health`.
|
||||
- The online request writes a valid WAV file.
|
||||
- The generated audio sample rate is 44.1 kHz.
|
||||
- The generated duration is approximately 10 seconds.
|
||||
- Peak sampled GPU memory is within the 24GB RTX 4090 budget. In the validated
|
||||
run, offline and online generation each peaked at about 12.6 GiB.
|
||||
|
||||
Validate the offline outputs:
|
||||
|
||||
```bash
|
||||
ls -lh examples/offline_inference/text_to_audio/stable_audio_10s.wav
|
||||
|
||||
python - <<'PY'
|
||||
import soundfile as sf
|
||||
|
||||
path = "examples/offline_inference/text_to_audio/stable_audio_10s.wav"
|
||||
audio, sample_rate = sf.read(path)
|
||||
print("sample_rate:", sample_rate)
|
||||
print("shape:", audio.shape)
|
||||
print("duration:", len(audio) / sample_rate)
|
||||
PY
|
||||
```
|
||||
|
||||
Validate the online output:
|
||||
|
||||
```bash
|
||||
ls -lh examples/online_serving/stable_audio/piano_10s.wav
|
||||
|
||||
python - <<'PY'
|
||||
import soundfile as sf
|
||||
|
||||
path = "examples/online_serving/stable_audio/piano_10s.wav"
|
||||
audio, sample_rate = sf.read(path)
|
||||
print("sample_rate:", sample_rate)
|
||||
print("shape:", audio.shape)
|
||||
print("duration:", len(audio) / sample_rate)
|
||||
PY
|
||||
```
|
||||
|
||||
##### Notes
|
||||
|
||||
- `stable-audio-open-1.0` can generate up to about 47 seconds of 44.1 kHz
|
||||
stereo audio. This recipe validates 10-second WAV outputs.
|
||||
- `--cache-backend tea_cache` is supported and was used for the 10-second
|
||||
offline validation command.
|
||||
- The model is gated on Hugging Face and requires license acceptance before
|
||||
download.
|
||||
- If online serving fails while importing `torchaudio`, make sure the
|
||||
`torchaudio` wheel matches the installed PyTorch and CUDA build. The
|
||||
validated environment used `torch==2.11.0+cu130` and
|
||||
`torchaudio==2.11.0+cu130`.
|
||||
- The `NIXL is not available`, `GLOO_SOCKET_IFNAME`, and `torchsde` boundary
|
||||
warnings observed during validation did not prevent successful generation.
|
||||
- This recipe was validated on one RTX 4090 24GB GPU only. Other GPU counts,
|
||||
ROCm, XPU, and NPU setups are not covered here.
|
||||
- Long generations, higher inference-step counts, and non-WAV response formats
|
||||
were not benchmarked in this recipe.
|
||||
Reference in new issue
Block a user