[Docs][skip ci] Fix omni and tts docs (#2130)

Signed-off-by: gcanlin <canlinguosdu@gmail.com>
This commit is contained in:
Canlin Guo authored and GitHub committed 2026-03-24 22:39:23 +08:00
1 parent 47ba887b93
commit 7bcd43f456
3 files changed
+20 -23

No files matched your search

@@ -73,7 +73,6 @@ cd /vllm-workspace/vllm
git pull origin main
git fetch origin --tags
git checkout v0.18.0
VLLM_TARGET_DEVICE=empty pip install -v -e .
# Because vllm-ascend has not yet entered continuous development and has not been officially released, we need to pin it to a specific commit. Please note that this commit may change over time.
cd /vllm-workspace/vllm-ascend
@@ -83,8 +82,8 @@ git checkout 1e05c4908f31737bc4eef865a9f351d030a77c9d
pip install -v -e .
# Install vLLM-Omni from the latest main branch
cd /vllm-workspace/vllm-omni
git clone https://github.com/vllm-project/vllm-omni.git
cd /vllm-workspace/vllm-omni
pip install -v -e . --no-build-isolation
# or VLLM_OMNI_TARGET_DEVICE=npu pip install -v -e .
export VLLM_WORKER_MULTIPROC_METHOD=spawn
+11 -6
View File
@@ -15,7 +15,7 @@ vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
If you want to open async chunking for qwen3-omni, launch the server with command below
```bash
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 --stage-configs-path /vllm_omni/model_executor/stage_configs/qwen3_omni_moe_async_chunk.yaml
vllm serve Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091 --stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_omni_moe_async_chunk.yaml
```
If you have custom stage configs file, launch the server with command below
@@ -79,7 +79,7 @@ You can control output modalities to specify which types of output the model sho
| Modalities | Output |
|------------|--------|
| `["text"]` | Text only |
| `["audio"]` | Text + Audio |
| `["audio"]` | Audio only |
| `["text", "audio"]` | Text + Audio |
| Not specified | Text + Audio (default) |
@@ -100,13 +100,13 @@ curl http://localhost:8091/v1/chat/completions \
#### Text + Audio
```bash
curl http://localhost:8091/v1/chat/completions \
curl -s http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen3-Omni-30B-A3B-Instruct",
"messages": [{"role": "user", "content": "Describe vLLM in brief."}],
"modalities": ["audio"]
}'
}' | jq -r '.choices[0].message.audio.data' | base64 -d > output.wav
```
### Using Python client
@@ -137,6 +137,7 @@ print(response.choices[0].message.content)
#### Text + Audio
```python
import base64
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8091/v1", api_key="EMPTY")
@@ -144,11 +145,15 @@ client = OpenAI(base_url="http://localhost:8091/v1", api_key="EMPTY")
response = client.chat.completions.create(
model="Qwen/Qwen3-Omni-30B-A3B-Instruct",
messages=[{"role": "user", "content": "Describe vLLM in brief."}],
modalities=["audio"]
modalities=["text", "audio"]
)
# Response contains two choices: one with text, one with audio
print(response.choices[0].message.content) # Text response
print(response.choices[1].message.audio) # Audio response
# Save audio to file
audio_data = base64.b64decode(response.choices[1].message.audio.data)
with open("output.wav", "wb") as f:
f.write(audio_data)
```
## Speaker selection
+8 -15
View File
@@ -43,30 +43,23 @@ Then open http://localhost:7860 in your browser.
### Launch the Server
The default stage config is located at `vllm_omni/model_executor/stage_configs/qwen3_tts.yaml`. For other platforms (e.g., NPU), refer to `vllm_omni/platforms/npu/stage_configs/qwen3_tts.yaml`.
```bash
# CustomVoice model (predefined speakers)
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
--stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_tts.yaml \
--omni \
--port 8091 \
--trust-remote-code \
--enforce-eager
--port 8091
# VoiceDesign model
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign \
--stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_tts.yaml \
--omni \
--port 8091 \
--trust-remote-code \
--enforce-eager
--port 8091
# Base model (voice cloning)
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-Base \
--stage-configs-path vllm_omni/model_executor/stage_configs/qwen3_tts.yaml \
--omni \
--port 8091 \
--trust-remote-code \
--enforce-eager
--port 8091
```
If you have custom stage configs file, launch the server with command below
@@ -74,9 +67,7 @@ If you have custom stage configs file, launch the server with command below
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
--stage-configs-path /path/to/stage_configs_file \
--omni \
--port 8091 \
--trust-remote-code \
--enforce-eager
--port 8091
```
Alternatively, use the convenience script:
@@ -99,12 +90,14 @@ cd examples/online_serving/qwen3_tts
```bash
# CustomVoice: Use predefined speaker
python openai_speech_client.py \
--model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
--text "你好,我是通义千问" \
--voice vivian \
--language Chinese
# CustomVoice with style instruction
python openai_speech_client.py \
--model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \
--text "今天天气真好" \
--voice ryan \
--instructions "用开心的语气说"