wzliu
33e03b44c5
ray + mooncake connector work on 2 nodes, but needes several updates\n1.start ray inside vllm-omni \n2.placement group optimize \n3.fix close
12 lines
662 B
Bash
12 lines
662 B
Bash
export PYTHONPATH=/workspace/omni/vllm-omni/:$PYTHONPATH
|
|
CUDA_VISIBLE_DEVICES=4 python end2end.py --model /workspace/Qwen2.5-Omni-7B/ \
|
|
--voice-type "m02" \
|
|
--dit-ckpt none \
|
|
--bigvgan-ckpt none \
|
|
--output-wav output_audio \
|
|
--prompt_type text \
|
|
--init-sleep-seconds 0 \
|
|
--worker-backend ray \
|
|
--ray-address auto \
|
|
--prompts "Explain the system architecture for a scalable audio generation pipeline. Answer in 15 words."
|