wzliu
33e03b44c5
ray + mooncake connector work on 2 nodes, but needes several updates\n1.start ray inside vllm-omni \n2.placement group optimize \n3.fix close
977 B
977 B
Offline Example of vLLM-omni for Qwen2.5-omni
🛠️ Installation
Please refer to README.md
Run examples (Qwen2.5-omni)
Multiple Prompts
Download dataset from seed_tts. To get the prompt, you can:
tar -xf <Your Download Path>/seedtts_testset.tar
cp seedtts_testset/en/meta.lst examples/offline_inference/qwen2_5_omni/meta.lst
python3 examples/offline_inference/qwen2_5_omni/extract_prompts.py \
--input examples/offline_inference/qwen2_5_omni/meta.lst \
--output examples/offline_inference/qwen2_5_omni/top100.txt \
--topk 100
Get into the example folder
cd examples/offline_inference/qwen2_5_omni
Then run the command below.
bash run_multiple_prompts.sh
Single Prompts
Get into the example folder
cd examples/offline_inference/qwen2_5_omni
Then run the command below.
bash run_single_prompt.sh