[Bugfix] Leave MiniCPM-o vocoder memory headroom (#5621)

Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
authored and GitHub committed 2026-07-31 03:35:13 +00:00
1 parent 499d930fe5
commit 0a51a18570
4 files changed
+20 -12

No files matched your search

+5 -3
View File
@@ -23,9 +23,11 @@ including `librosa`.
The deploy config auto-loads via `--omni`.
The default `vllm_omni/deploy/minicpmo_4_5.yaml` keeps all three stages on
logical device 0 with memory budgets of 55%, 22%, and 22%. The profile admits
at most four sequences per stage and bounds startup video profiling to 32
frames per video. For throughput,`minicpmo_4_5_2gpu.yaml` gives the Thinker
logical device 0 with memory budgets of 55%, 15%, and 15%, leaving headroom
for runtime kernels such as the HiFi-GAN vocoder's cuDNN workspace. The
profile admits at most four sequences per stage and bounds startup video
profiling to 32 frames per video. For throughput,
`minicpmo_4_5_2gpu.yaml` gives the Thinker
GPU 0 (90%) and colocates the Talker (55%) and Code2Wav (35%) on GPU 1. That
profile admits at most four concurrent sequences per stage.