feat(moss-tts-nano): add online serving, Gradio demo, and fix /v1/audio/speech integration

- Register moss_tts_nano as TTS model stage in serving_speech.py
  (model detection, validation, prompt building)
- Fix registry mod_relname: moss_tts_nano -> modeling_moss_tts_nano
- Fix stage config: is_comprehension=true (required for generate task)
- Fix default mode: voice_clone -> continuation (built-in presets)
- Add compute_logits stub for VllmModelForTextGeneration protocol
- Remove unused _sentinel nn.Parameter
- Add Gradio demo with AudioWorklet streaming player (48kHz stereo)
- Add online/offline serving docs and launch scripts

TODO: Single-stage generation models don't support true streaming
(progressive audio chunks). Current TTFP = full generation time.
Needs multi-step scheduling support in GPUGenerationModelRunner.

Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
This commit is contained in:
Yueqian Lin authored and Yueqian Lin committed 2026-04-14 13:06:44 -04:00
1 parent 571c8d4235
commit fc81b8504a
9 files changed
+1065 -6

No files matched your search

@@ -93,7 +93,7 @@ _DEFAULT_AUDIO_TOP_K = 25
_DEFAULT_AUDIO_REPETITION_PENALTY = 1.2
_DEFAULT_MAX_NEW_FRAMES = 375
_DEFAULT_VOICE = "Junhao"
_DEFAULT_MODE = "voice_clone"
_DEFAULT_MODE = "continuation"
def _pick(info: dict, key: str, default):
@@ -149,9 +149,6 @@ class MossTTSNanoForGeneration(nn.Module):
self._device: torch.device | None = None
self._lock = threading.Lock()
# Dummy parameter so vLLM can resolve the device via next(parameters()).
self._sentinel = nn.Parameter(torch.zeros(1), requires_grad=False)
# ------------------------------------------------------------------
# Weight loading
# ------------------------------------------------------------------
@@ -405,6 +402,13 @@ class MossTTSNanoForGeneration(nn.Module):
# vLLM boilerplate: embed_input_ids required by generation runner
# ------------------------------------------------------------------
def compute_logits(
self,
hidden_states: torch.Tensor,
) -> torch.Tensor | None:
# Generation stage does not produce logits; satisfy VllmModelForTextGeneration protocol.
return None
def embed_input_ids(
self,
input_ids: torch.Tensor,
+1 -1
View File
@@ -166,7 +166,7 @@ _OMNI_MODELS = {
## MOSS-TTS-Nano
"MossTTSNanoForCausalLM": (
"moss_tts_nano",
"moss_tts_nano",
"modeling_moss_tts_nano",
"MossTTSNanoForGeneration",
),
"DyninOmniForConditionalGeneration": (