vllm-omni/examples/online_serving/README.md

Online serving Example of vLLM-omni for Qwen2.5-omni

🛠️ Installation

Please refer to README.md

Run examples (Qwen2.5-omni)

Launch the server

vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091

If you have custom stage configs file, launch the server with command below

vllm serve Qwen/Qwen2.5-Omni-7B --omni --port 8091 --stage-configs-path /path/to/stage_configs_file

Get into the example folder

cd examples/online_serving

Send request via python

python openai_chat_completion_client_for_multimodal_generation.py

Send request via curl

bash run_curl_multimodal_generation.sh