Yueqian Lin
fc81b8504a
- Register moss_tts_nano as TTS model stage in serving_speech.py (model detection, validation, prompt building) - Fix registry mod_relname: moss_tts_nano -> modeling_moss_tts_nano - Fix stage config: is_comprehension=true (required for generate task) - Fix default mode: voice_clone -> continuation (built-in presets) - Add compute_logits stub for VllmModelForTextGeneration protocol - Remove unused _sentinel nn.Parameter - Add Gradio demo with AudioWorklet streaming player (48kHz stereo) - Add online/offline serving docs and launch scripts TODO: Single-stage generation models don't support true streaming (progressive audio chunks). Current TTFP = full generation time. Needs multi-step scheduling support in GPUGenerationModelRunner. Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
25 lines
629 B
Bash
Executable File
25 lines
629 B
Bash
Executable File
#!/bin/bash
|
|
# Launch vLLM-Omni server for MOSS-TTS-Nano
|
|
#
|
|
# Usage:
|
|
# ./run_server.sh
|
|
# CUDA_VISIBLE_DEVICES=0 PORT=8091 ./run_server.sh
|
|
|
|
set -e
|
|
|
|
MODEL="${MODEL:-OpenMOSS-Team/MOSS-TTS-Nano}"
|
|
PORT="${PORT:-8091}"
|
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
|
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
|
|
|
|
echo "Starting MOSS-TTS-Nano server with model: $MODEL"
|
|
|
|
FLASHINFER_DISABLE_VERSION_CHECK=1 \
|
|
vllm-omni serve "$MODEL" \
|
|
--stage-configs-path "$REPO_ROOT/vllm_omni/model_executor/stage_configs/moss_tts_nano.yaml" \
|
|
--host 0.0.0.0 \
|
|
--port "$PORT" \
|
|
--trust-remote-code \
|
|
--enforce-eager \
|
|
--omni
|