[Model] Reduce MOSS-TTS Local intermediate output overhead

Keep internal talker control outputs FINAL_ONLY while codec audio remains incremental. Reapply pipeline sampling constraints after default streaming coercion, log routine coercions at debug level, and cover explicit/default parameters with regression tests. Document matched high-concurrency statistics settings.

Signed-off-by: Sy03 <1370724210@qq.com>
This commit is contained in:
Sy03 committed 2026-09-08 01:15:46 +08:00
1 parent dcd6588a36
commit 55855fee9c
6 files changed
+66 -7

No files matched your search

@@ -432,13 +432,19 @@ For a single H200, the optional
deployment places both the talker and codec on logical GPU 0:
```bash
CUDA_VISIBLE_DEVICES=0 vllm serve OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5 \
CUDA_VISIBLE_DEVICES=0 VLLM_OMNI_EVENT_DRIVEN_ORCH=1 \
vllm serve OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5 \
--omni --trust-remote-code \
--deploy-config vllm_omni/deploy/moss_tts_local_h200.yaml
--deploy-config vllm_omni/deploy/moss_tts_local_h200.yaml \
--disable-log-stats
```
Run from the repository root and select an available physical GPU with
`CUDA_VISIBLE_DEVICES`. This preset configures 256 request slots per stage,
`CUDA_VISIBLE_DEVICES`. The command enables event-driven orchestration and
disables detailed per-request statistics logging to reduce CPU overhead at
high concurrency. Remove `--disable-log-stats` when those statistics are needed;
keep these settings identical when comparing performance.
This preset configures 256 request slots per stage,
a 32 GiB talker KV cache, talker CUDA Graph buckets through 512 scheduled tokens,
and codec batch buckets through 256. It requires more than 80 GiB of GPU memory;
request capacity also depends on input and generated lengths. The default