refactor(moss_tts_nano): migrate to Pipeline + Deploy schema
Follow-up to #2958. MOSS-TTS-Nano was authored before the schema refactor and still shipped as a legacy stage_configs yaml; this aligns it with the same layout the 5 migrated TTS models now use. - vllm_omni/model_executor/models/moss_tts_nano/pipeline.py declares MOSS_TTS_NANO_PIPELINE (single LLM_AR stage, owns_tokenizer, audio output, stop_token_ids=[2] as a hard EOS backstop). - vllm_omni/deploy/moss_tts_nano.yaml holds runtime knobs (max_num_seqs, gpu_memory_utilization, enforce_eager, default_sampling_params, skip_mm_profiling); trust_remote_code stays at deploy top-level. - vllm_omni/config/pipeline_registry.py registers the entry so the lazy registry can resolve it. - moss_tts_nano/__init__.py exports MossTTSNanoForGeneration (VoxCPM2 pattern). - Removed vllm_omni/model_executor/stage_configs/moss_tts_nano.yaml. Examples, shell scripts, READMEs, and buildkite-invoked tests are updated to use `vllm serve <model> --omni` / `--deploy-config` (auto-load kicks in; no --stage-configs-path or --trust-remote-code). Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
This commit is contained in:
1 parent
26bbe7b527
commit
b03a41a7f9
12 files changed
+104
-101
No files matched your search
@@ -31,7 +31,7 @@ Options:
|
||||
--text-temperature F Text layer temperature (default: 1.0)
|
||||
--batch Run a built-in batch of diverse samples (ZH/EN/FR)
|
||||
--output-dir DIR Directory for WAV outputs (default: /tmp/moss_tts_nano_output)
|
||||
--stage-config PATH Path to stage config YAML
|
||||
--deploy-config PATH Override deploy YAML (defaults to vllm_omni/deploy/moss_tts_nano.yaml)
|
||||
--stage-init-timeout INT Timeout in seconds for stage init (default: 120)
|
||||
```
|
||||
|
||||
@@ -77,15 +77,17 @@ python end2end.py --text "Deterministic test." --seed 42
|
||||
| `Hina` | JA |
|
||||
| `Mei` | JA |
|
||||
|
||||
## Stage Config
|
||||
## Deploy Config
|
||||
|
||||
The stage config is at `vllm_omni/model_executor/stage_configs/moss_tts_nano.yaml`. Key settings:
|
||||
Runtime knobs live in `vllm_omni/deploy/moss_tts_nano.yaml` (auto-loaded;
|
||||
override with `--deploy-config PATH`). Key stage-level settings:
|
||||
|
||||
```yaml
|
||||
engine_args:
|
||||
gpu_memory_utilization: 0.3 # ~2 GB VRAM; increase for faster init
|
||||
max_num_seqs: 4 # concurrent requests
|
||||
max_model_len: 4096
|
||||
stages:
|
||||
- stage_id: 0
|
||||
gpu_memory_utilization: 0.3 # ~2 GB VRAM; increase for faster init
|
||||
max_num_seqs: 4 # concurrent requests
|
||||
max_model_len: 4096
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
@@ -39,9 +39,6 @@ from vllm_omni import Omni # noqa: E402
|
||||
|
||||
MODEL = "OpenMOSS-Team/MOSS-TTS-Nano"
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[4]
|
||||
DEFAULT_STAGE_CONFIG = str(_REPO_ROOT / "vllm_omni" / "model_executor" / "stage_configs" / "moss_tts_nano.yaml")
|
||||
|
||||
BATCH_SAMPLES = [
|
||||
{"text": "Hello, this is a test of MOSS-TTS-Nano.", "voice": "Ava", "label": "en_hello"},
|
||||
{"text": "你好,这是 MOSS-TTS-Nano 的语音合成测试。", "voice": "Junhao", "label": "zh_hello"},
|
||||
@@ -98,7 +95,7 @@ def save_audio(waveform: torch.Tensor, path: str, sample_rate: int = 48000) -> N
|
||||
def main(args) -> None:
|
||||
omni = Omni(
|
||||
model=MODEL,
|
||||
stage_configs_path=args.stage_config,
|
||||
deploy_config=args.deploy_config,
|
||||
stage_init_timeout=args.stage_init_timeout,
|
||||
)
|
||||
|
||||
@@ -170,7 +167,11 @@ def parse_args():
|
||||
parser.add_argument("--text-temperature", type=float, default=1.0)
|
||||
parser.add_argument("--batch", action="store_true", help="Run built-in batch of diverse samples.")
|
||||
parser.add_argument("--output-dir", default="/tmp/moss_tts_nano_output", help="Directory for WAV outputs.")
|
||||
parser.add_argument("--stage-config", default=DEFAULT_STAGE_CONFIG)
|
||||
parser.add_argument(
|
||||
"--deploy-config",
|
||||
default=None,
|
||||
help="Path to a deploy YAML; leave unset to auto-load vllm_omni/deploy/moss_tts_nano.yaml.",
|
||||
)
|
||||
parser.add_argument("--stage-init-timeout", type=int, default=120)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
@@ -29,14 +29,13 @@ Features:
|
||||
## Launch the Server
|
||||
|
||||
```bash
|
||||
vllm-omni serve OpenMOSS-Team/MOSS-TTS-Nano \
|
||||
--stage-configs-path vllm_omni/model_executor/stage_configs/moss_tts_nano.yaml \
|
||||
--omni \
|
||||
--port 8091 \
|
||||
--trust-remote-code \
|
||||
--enforce-eager
|
||||
vllm serve OpenMOSS-Team/MOSS-TTS-Nano --omni --port 8091
|
||||
```
|
||||
|
||||
The deploy config at `vllm_omni/deploy/moss_tts_nano.yaml` auto-loads; no
|
||||
`--stage-configs-path`, `--trust-remote-code`, or `--enforce-eager` flags
|
||||
are needed.
|
||||
|
||||
Or use the convenience script:
|
||||
|
||||
```bash
|
||||
|
||||
@@ -11,16 +11,12 @@ MODEL="${MODEL:-OpenMOSS-Team/MOSS-TTS-Nano}"
|
||||
PORT="${PORT:-8091}"
|
||||
GRADIO_PORT="${GRADIO_PORT:-7860}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
|
||||
|
||||
echo "Starting MOSS-TTS-Nano server (port $PORT)..."
|
||||
FLASHINFER_DISABLE_VERSION_CHECK=1 \
|
||||
vllm-omni serve "$MODEL" \
|
||||
--stage-configs-path "$REPO_ROOT/vllm_omni/model_executor/stage_configs/moss_tts_nano.yaml" \
|
||||
vllm serve "$MODEL" \
|
||||
--host 0.0.0.0 \
|
||||
--port "$PORT" \
|
||||
--trust-remote-code \
|
||||
--enforce-eager \
|
||||
--omni &
|
||||
SERVER_PID=$!
|
||||
|
||||
|
||||
@@ -9,16 +9,11 @@ set -e
|
||||
|
||||
MODEL="${MODEL:-OpenMOSS-Team/MOSS-TTS-Nano}"
|
||||
PORT="${PORT:-8091}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
REPO_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
|
||||
|
||||
echo "Starting MOSS-TTS-Nano server with model: $MODEL"
|
||||
|
||||
FLASHINFER_DISABLE_VERSION_CHECK=1 \
|
||||
vllm-omni serve "$MODEL" \
|
||||
--stage-configs-path "$REPO_ROOT/vllm_omni/model_executor/stage_configs/moss_tts_nano.yaml" \
|
||||
vllm serve "$MODEL" \
|
||||
--host 0.0.0.0 \
|
||||
--port "$PORT" \
|
||||
--trust-remote-code \
|
||||
--enforce-eager \
|
||||
--omni
|
||||
Reference in new issue
Block a user