0 Fork 0

Files

T

Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98 )

Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>

2025-11-29 20:42:39 +08:00

1.9 KiB

Raw Blame History

Offline Inference Example of vLLM-Omni for Qwen2.5-Omni

Source https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/qwen2_5_omni.

🛠️ Installation

Please refer to installation.

Run Offline inference with Qwen2.5-Omni.

First, navigate to the example folder

cd examples/offline_inference/qwen2_5_omni

Inside the directory, end2end.py is a comprehensive demo suite for initializing a model instance of Qwen/Qwen2.5-Omni-7B and use it for various offline inference tasks. ??? abstract "end2end.py" py --8<-- "examples/offline_inference/qwen2_5_omni/end2end.py"

Below we also provide simple bash scripts to execute this file.

Single Prompt

bash run_single_prompt.sh

??? abstract "run_single_prompt.sh" sh --8<-- "examples/offline_inference/qwen2_5_omni/run_single_prompt.sh"

Multiple Prompts

Download dataset from seed_tts and extract the prompts with extract_prompts.py ??? abstract "extract_prompts.py" py --8<-- "examples/offline_inference/qwen2_5_omni/extract_prompts.py"

tar -xf <Your Download Path>/seedtts_testset.tar
cp seedtts_testset/en/meta.lst examples/offline_inference/qwen2_5_omni/meta.lst
python3 examples/offline_inference/qwen2_5_omni/extract_prompts.py \
  --input examples/offline_inference/qwen2_5_omni/meta.lst \
  --output examples/offline_inference/qwen2_5_omni/top100.txt \
  --topk 10

Then run the command below.

bash run_multiple_prompts.sh

??? abstract "run_multiple_prompts.sh" sh --8<-- "examples/offline_inference/qwen2_5_omni/run_multiple_prompts.sh"

FAQ

If you encounter error about backend of librosa, try to install ffmpeg with command below.

sudo apt update
sudo apt install ffmpeg

1.9 KiB Raw Blame History