[New Model] Add support for tencent/Covo-Audio-Chat (#2293)

Signed-off-by: Dnoob <dxpouo@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
authored and GitHub committed 2026-05-12 14:39:58 +08:00
1 parent e40076e872
commit d7ea5d5979
25 files changed
+2844

No files matched your search

+2
View File
@@ -29,6 +29,8 @@ recipes/
Qwen-Image on `1x A100 80GB`, including optional step-wise continuous batching replay
- [`Qwen/Qwen3-Omni.md`](./Qwen/Qwen3-Omni.md): online serving recipe for
multimodal chat on `1x A100 80GB`
- [`Tencent/Covo-Audio-Chat.md`](./Tencent/Covo-Audio-Chat.md): online
serving recipe for audio chat on `1x A100 80GB`
- [`Qwen/Qwen3-TTS.md`](./Qwen/Qwen3-TTS.md): text-to-speech serving recipe
for Qwen3-TTS (CustomVoice / VoiceDesign / Base) on `1x H100/A100 80GB`
- [`LTX/LTX-2.md`](./LTX/LTX-2.md): text-to-video and image-to-video serving
+92
View File
@@ -0,0 +1,92 @@
# Covo-Audio-Chat for audio chat on 1x A100 80GB
## Summary
- Vendor: Tencent
- Model: `tencent/Covo-Audio-Chat`
- Task: Audio chat — audio (and optional text) input, interleaved text + audio output
- Mode: Online serving with the OpenAI-compatible API (offline inference also supported)
- Maintainer: Community
## When to use this recipe
Use this recipe when you want a known-good starting point for serving
`tencent/Covo-Audio-Chat` with vLLM-Omni on a single 80 GB-class GPU and
validate the deployment with the bundled client example.
## References
- Related example under `examples/`:
[`examples/online_serving/covo_audio/README.md`](../../examples/online_serving/covo_audio/README.md)
- Offline end-to-end script:
[`examples/offline_inference/covo_audio/end2end.py`](../../examples/offline_inference/covo_audio/end2end.py)
- Deploy config:
[`vllm_omni/deploy/covo_audio.yaml`](../../vllm_omni/deploy/covo_audio.yaml)
- Pipeline topology:
[`vllm_omni/model_executor/models/covo_audio/pipeline.py`](../../vllm_omni/model_executor/models/covo_audio/pipeline.py)
## Hardware Support
This recipe currently documents one tested reference configuration for
CUDA GPU serving. The fused thinker+talker stage and the BigVGAN code2wav
stage both run on the same physical device.
## GPU
### 1x A100 80GB
#### Environment
- OS: Linux
- Python: 3.12
- Driver / runtime: NVIDIA CUDA 12.x with an A100 80 GB GPU
- vLLM version: Match the repository requirements for your checkout
- vLLM-Omni version or commit: Use the commit you are deploying from
- Extra Python deps: `torchdiffeq`, `librosa`
#### Command
Install the two runtime-only deps that are not pulled in by vLLM-Omni itself:
```bash
pip install torchdiffeq librosa
```
Start the server from the repository root:
```bash
CUDA_VISIBLE_DEVICES=0 vllm serve tencent/Covo-Audio-Chat --omni \
--trust-remote-code --port 18091
```
The deploy config at `vllm_omni/deploy/covo_audio.yaml` is loaded
automatically by the model registry.
#### Verification
With the server ready, send a test request via the bundled client:
```bash
cd examples/online_serving/covo_audio
python openai_chat_completion_client.py
```
Expected: the client prints the generated text and writes an output WAV
file to the current directory.
#### Notes
- Memory usage: Stage 0 (7B LLM) ~16 GiB + stage 1 (BigVGAN vocoder)
~2 GiB, both resident on the same device.
- Key flags: `--omni` and `--trust-remote-code` are both required.
- System prompt: the bundled `openai_chat_completion_client.py` and
`end2end.py` already include `COVO_AUDIO_SYSTEM_PROMPT` (defined in
`vllm_omni/model_executor/models/covo_audio/prompt_utils.py`) in the
system message by default. When integrating your own client, make sure
to include it — without it, stage 0 emits text-only tokens and the
output audio is silent.
- Stage 1 runs with `enforce_eager: true` and `dtype: float32` — see the
header of `deploy/covo_audio.yaml` for the specific reasons.
- Known limitations: this starter recipe covers the single-GPU online-serving
path only; multi-GPU layouts and platform-specific overrides are not
documented here.