Yueqian LinandHongsheng Liu
afa2b09684
Signed-off-by: Yueqian Lin <linyueqian@outlook.com> Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
20 lines
386 B
Bash
Executable File
20 lines
386 B
Bash
Executable File
#!/bin/bash
|
|
# Launch vLLM-Omni server for MOSS-TTS-Nano
|
|
#
|
|
# Usage:
|
|
# ./run_server.sh
|
|
# CUDA_VISIBLE_DEVICES=0 PORT=8091 ./run_server.sh
|
|
|
|
set -e
|
|
|
|
MODEL="${MODEL:-OpenMOSS-Team/MOSS-TTS-Nano}"
|
|
PORT="${PORT:-8091}"
|
|
|
|
echo "Starting MOSS-TTS-Nano server with model: $MODEL"
|
|
|
|
FLASHINFER_DISABLE_VERSION_CHECK=1 \
|
|
vllm serve "$MODEL" \
|
|
--host 0.0.0.0 \
|
|
--port "$PORT" \
|
|
--omni
|