anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Model] Default MOSS Local 1.5 to adaptive CUDA MRV2 with MPS (#8438)

Browse Source 
Signed-off-by: Sy03 <1370724210@qq.com>
This commit is contained in:
Sy03 authored and GitHub committed 2026-10-06 08:20:46 -07:00
1 parent ee9ab9ea88
commit 7be49aa0ce
30 files changed
+808 -152
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/configuration/stage_configs.md
+12 -7
View File
@@ -598,8 +598,9 @@ the intended GPU, and warm the complete pipeline before measuring performance.
## MOSS-TTS Local 1.5 with Model Runner V2
`OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5` can opt into CUDA MRV2 using
the shared runtime introduced for Qwen3-TTS:
`OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5` defaults to CUDA MRV2 using
the shared runtime introduced for Qwen3-TTS. To select the C64 profile without
MPS explicitly:
```bash
vllm serve OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5 --omni \
@@ -607,7 +608,7 @@ vllm serve OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5 --omni \
```
This profile inherits the batching, codec graph buckets and 1-frame/15-frame
chunk geometry from `moss_tts_local.yaml`, and selects V2 for both the Local
chunk geometry from `moss_tts_local_v1.yaml`, and selects V2 for both the Local
Talker and codec stages. The Local depth predictor exposes the MRV2 `mtp`
capabilities while retaining its V1 `talker_mtp` implementation, sampling
defaults and explicit request-seed handling. Codec chunks use the native data
@@ -631,8 +632,12 @@ guarantee that 128 maximum-length prompts fit simultaneously. See the
[MOSS recipe](gh-file:recipes/OpenMOSS/MOSS-TTS.md#local-15-mrv2-and-slot-attention)
for activation, backend comparisons, memory requirements and benchmark commands.
Omitting `--deploy-config`, or selecting `moss_tts_local.yaml`, retains V1.
NPU, XPU, ROCm and MUSA overrides also retain V1. This profile does not enable
Omitting `--deploy-config` selects the C128 system profile on CUDA GPUs with
at least 140 GiB memory and MPS available. Smaller GPUs or a failed memory
query use C64 with utilization-based memory budgets; missing MPS selects
C64 without MPS. Selecting `moss_tts_local.yaml` explicitly uses C64 with MPS.
Use `moss_tts_local_v1.yaml` for the previous V1 deployment.
NPU, XPU, ROCm and MUSA overrides retain V1. These profiles do not enable
MRV2 for MOSS Delay, Realtime or Nano. Local 1.5 outputs 48 kHz stereo audio;
set `VLLM_OMNI_BENCH_AUDIO_SAMPLE_RATE=48000` and
`VLLM_OMNI_BENCH_AUDIO_CHANNELS=2` when benchmarking raw PCM.
@@ -640,5 +645,5 @@ set `VLLM_OMNI_BENCH_AUDIO_SAMPLE_RATE=48000` and
Event-driven orchestration remains independently selectable with
`VLLM_OMNI_EVENT_DRIVEN_ORCH=0` or `1`. Keep the runner and deployment identical
when comparing these modes. Model-runner selection does not change the
orchestration default or enable experimental reference encoding, chunk ramps,
generation-output draining or MPS.
orchestration default. MPS, codec dispatch and graph settings belong to the
deployment profile; the automatic CUDA default selects MPS when available.