2025-11-29 04:42:39 -08:00
# Offline Inference Example of vLLM-Omni for Qwen2.5-Omni
2025-11-17 14:44:10 +08:00
2025-11-28 11:03:52 +08:00
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/qwen2_5_omni>.
2025-11-17 14:44:10 +08:00
## 🛠️ Installation
2025-11-29 04:42:39 -08:00
Please refer to [installation ](../../../getting_started/installation/README.md ).
## Run Offline inference with Qwen2.5-Omni.
First, navigate to the example folder
```bash
cd examples/offline_inference/qwen2_5_omni
```
Inside the directory, `end2end.py` is a comprehensive demo suite for initializing a model instance of `Qwen/Qwen2.5-Omni-7B` and use it for various offline inference tasks.
??? abstract "end2end.py"
``````py
--8<-- "examples/offline_inference/qwen2_5_omni/end2end.py"
` `````
Below we also provide simple bash scripts to execute this file.
### Single Prompt
` ``bash
bash run_single_prompt.sh
` ``
??? abstract "run_single_prompt.sh"
` `````sh
--8<-- "examples/offline_inference/qwen2_5_omni/run_single_prompt.sh"
` `````
2025-11-17 14:44:10 +08:00
### Multiple Prompts
2025-11-29 04:42:39 -08:00
Download dataset from [seed_tts](https://drive.google.com/file/d/1GlSjVfSHkW3-leKKBlfrjuuTGqQ_xaLP/edit) and extract the prompts with ` extract_prompts.py`
??? abstract "extract_prompts.py"
` `````py
--8<-- "examples/offline_inference/qwen2_5_omni/extract_prompts.py"
` `````
2025-11-17 14:44:10 +08:00
` ``bash
tar -xf <Your Download Path>/seedtts_testset.tar
cp seedtts_testset/en/meta.lst examples/offline_inference/qwen2_5_omni/meta.lst
python3 examples/offline_inference/qwen2_5_omni/extract_prompts.py \
--input examples/offline_inference/qwen2_5_omni/meta.lst \
--output examples/offline_inference/qwen2_5_omni/top100.txt \
2025-11-29 04:42:39 -08:00
--topk 10
2025-11-17 14:44:10 +08:00
` ``
Then run the command below.
` ``bash
bash run_multiple_prompts.sh
` ``
2025-11-29 04:42:39 -08:00
??? abstract "run_multiple_prompts.sh"
` `````sh
--8<-- "examples/offline_inference/qwen2_5_omni/run_multiple_prompts.sh"
` `````
2025-11-17 14:44:10 +08:00
2025-11-28 11:03:52 +08:00
### FAQ
If you encounter error about backend of librosa, try to install ffmpeg with command below.
` ``
sudo apt update
sudo apt install ffmpeg
` ``