vllm-omni/examples/offline_inference/qwen2_5_omni

Offline Example of vLLM-omni for Qwen2.5-omni

🛠️ Installation

Please refer to README.md

Run examples (Qwen2.5-omni)

Multiple Prompts

Download dataset from seed_tts. To get the prompt, you can:

tar -xf <Your Download Path>/seedtts_testset.tar
cp seedtts_testset/en/meta.lst examples/offline_inference/qwen2_5_omni/meta.lst
python3 examples/offline_inference/qwen2_5_omni/extract_prompts.py \
  --input examples/offline_inference/qwen2_5_omni/meta.lst \
  --output examples/offline_inference/qwen2_5_omni/top100.txt \
  --topk 100

Get into the example folder

cd examples/offline_inference/qwen2_5_omni

Then run the command below.

bash run_multiple_prompts.sh

Single Prompts

Get into the example folder

cd examples/offline_inference/qwen2_5_omni

Then run the command below.

bash run_single_prompt.sh

FAQ

If you encounter error about backend of librosa, try to install ffmpeg with command below.

sudo apt update
sudo apt install ffmpeg