vllm-omni/docs/user_guide/examples/offline_inference/qwen3_omni.md

Qwen3-Omni

Source https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/qwen3_omni.

Setup

Please refer to the stage configuration documentation to configure memory allocation appropriately for your hardware setup.

Run examples

Multiple Prompts

Get into the example folder

cd examples/offline_inference/qwen3_omni

Then run the command below. Note: for processing large volume data, it uses py_generator mode, which will return a python generator from Omni class.

bash run_multiple_prompts.sh

Single Prompt

Get into the example folder

cd examples/offline_inference/qwen3_omni

Then run the command below.

bash run_single_prompt.sh

If you have not enough memory, you can set thinker with tensor parallel. Just run the command below.

bash run_single_prompt_tp.sh

Modality control

If you want to control output modalities, e.g. only output text, you can run the command below:

python end2end.py --output-wav output_audio \
                  --query-type use_audio \
                  --modalities text

Using Local Media Files

The end2end.py script supports local media files (audio, video, image) via command-line arguments:

# Use local video file
python end2end.py --query-type use_video --video-path /path/to/video.mp4

# Use local image file
python end2end.py --query-type use_image --image-path /path/to/image.jpg

# Use local audio file
python end2end.py --query-type use_audio --audio-path /path/to/audio.wav

# Combine multiple local media files
python end2end.py --query-type mixed_modalities \
    --video-path /path/to/video.mp4 \
    --image-path /path/to/image.jpg \
    --audio-path /path/to/audio.wav

If media file paths are not provided, the script will use default assets. Supported query types:

• use_video: Video input
• use_image: Image input
• use_audio: Audio input
• text: Text-only query
• multi_audios: Multiple audio inputs
• mixed_modalities: Combination of video, image, and audio inputs

FAQ

If you encounter error about backend of librosa, try to install ffmpeg with command below.

sudo apt update
sudo apt install ffmpeg

Example materials

??? abstract "end2end.py" py --8<-- "examples/offline_inference/qwen3_omni/end2end.py" ??? abstract "run_multiple_prompts.sh" sh --8<-- "examples/offline_inference/qwen3_omni/run_multiple_prompts.sh" ??? abstract "run_single_prompt.sh" sh --8<-- "examples/offline_inference/qwen3_omni/run_single_prompt.sh" ??? abstract "run_single_prompt_tp.sh" sh --8<-- "examples/offline_inference/qwen3_omni/run_single_prompt_tp.sh" ??? abstract "text_prompts_10.txt" txt --8<-- "examples/offline_inference/qwen3_omni/text_prompts_10.txt"