Yueqian Lin
fc81b8504a
- Register moss_tts_nano as TTS model stage in serving_speech.py (model detection, validation, prompt building) - Fix registry mod_relname: moss_tts_nano -> modeling_moss_tts_nano - Fix stage config: is_comprehension=true (required for generate task) - Fix default mode: voice_clone -> continuation (built-in presets) - Add compute_logits stub for VllmModelForTextGeneration protocol - Remove unused _sentinel nn.Parameter - Add Gradio demo with AudioWorklet streaming player (48kHz stereo) - Add online/offline serving docs and launch scripts TODO: Single-stage generation models don't support true streaming (progressive audio chunks). Current TTFP = full generation time. Needs multi-step scheduling support in GPUGenerationModelRunner. Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
100 lines
3.3 KiB
Markdown
100 lines
3.3 KiB
Markdown
# MOSS-TTS-Nano Offline Inference
|
|
|
|
## Overview
|
|
|
|
Single-stage offline TTS pipeline using the 0.1B MOSS-TTS-Nano AR LM and MOSS-Audio-Tokenizer-Nano codec. Outputs 48 kHz stereo WAV.
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
python end2end.py --text "Hello, this is MOSS-TTS-Nano."
|
|
```
|
|
|
|
The first run downloads `OpenMOSS-Team/MOSS-TTS-Nano` and `OpenMOSS-Team/MOSS-Audio-Tokenizer-Nano` from Hugging Face.
|
|
|
|
## Usage
|
|
|
|
```
|
|
python end2end.py [OPTIONS]
|
|
|
|
Options:
|
|
--text TEXT Text to synthesize (default: "Hello, this is MOSS-TTS-Nano speaking.")
|
|
--voice VOICE Built-in voice preset (default: Junhao). See voice table below.
|
|
--mode MODE voice_clone (default) or continuation
|
|
--prompt-audio PATH Reference WAV/MP3 for custom voice cloning
|
|
--prompt-text TEXT Reference transcript (continuation mode)
|
|
--max-new-frames N Max AR frames, default 375 (~14 s audio)
|
|
--seed INT Random seed for reproducibility
|
|
--audio-temperature F Audio sampling temperature (default: 0.8)
|
|
--audio-top-k N Audio top-k sampling (default: 25)
|
|
--audio-top-p F Audio top-p sampling (default: 0.95)
|
|
--text-temperature F Text layer temperature (default: 1.0)
|
|
--batch Run a built-in batch of diverse samples (ZH/EN/FR)
|
|
--output-dir DIR Directory for WAV outputs (default: /tmp/moss_tts_nano_output)
|
|
--stage-config PATH Path to stage config YAML
|
|
--stage-init-timeout INT Timeout in seconds for stage init (default: 120)
|
|
```
|
|
|
|
## Examples
|
|
|
|
```bash
|
|
# Built-in Chinese voice
|
|
python end2end.py --text "你好，这是MOSS-TTS-Nano的语音合成演示。" --voice Junhao
|
|
|
|
# Built-in English voice
|
|
python end2end.py --text "Hello from MOSS-TTS-Nano." --voice Ava
|
|
|
|
# Custom voice clone
|
|
python end2end.py \
|
|
--text "Hello, this is a cloned voice." \
|
|
--prompt-audio /path/to/reference.wav \
|
|
--prompt-text "Exact transcript of the reference audio."
|
|
|
|
# Batch synthesis (ZH/EN/FR)
|
|
python end2end.py --batch --output-dir /tmp/batch_output
|
|
|
|
# Reproducible output
|
|
python end2end.py --text "Deterministic test." --seed 42
|
|
```
|
|
|
|
## Built-in Voice Presets
|
|
|
|
| Voice | Language |
|
|
|-------|----------|
|
|
| `Junhao` | ZH |
|
|
| `Zhiming` | ZH |
|
|
| `Weiguo` | ZH |
|
|
| `Xiaoyu` | ZH |
|
|
| `Yuewen` | ZH |
|
|
| `Lingyu` | ZH |
|
|
| `Ava` | EN |
|
|
| `Bella` | EN |
|
|
| `Adam` | EN |
|
|
| `Nathan` | EN |
|
|
| `Sakura` | JA |
|
|
| `Yui` | JA |
|
|
| `Aoi` | JA |
|
|
| `Hina` | JA |
|
|
| `Mei` | JA |
|
|
|
|
## Stage Config
|
|
|
|
The stage config is at `vllm_omni/model_executor/stage_configs/moss_tts_nano.yaml`. Key settings:
|
|
|
|
```yaml
|
|
engine_args:
|
|
gpu_memory_utilization: 0.3 # ~2 GB VRAM; increase for faster init
|
|
max_num_seqs: 4 # concurrent requests
|
|
max_model_len: 4096
|
|
```
|
|
|
|
## Output Format
|
|
|
|
WAV files, 48 kHz, stereo (2-channel). The codec interleaves stereo as `[L, R, L, R, ...]` in the flat tensor returned by the model.
|
|
|
|
## Troubleshooting
|
|
|
|
- **`libnvrtc.so.13: cannot open shared object file`**: torchaudio 2.10+ torchcodec backend requires NVRTC. The model patches `torchaudio.load/save` automatically at load time to fall back to soundfile.
|
|
- **`flash_attn not installed`**: The model falls back to `sdpa` attention automatically.
|
|
- **Empty audio**: Check that `--text` is non-empty and the model loaded successfully (look for "MOSS-TTS-Nano LM loaded" in logs).
|