[Bugfix] Leave MiniCPM-o vocoder memory headroom (#5621)
Signed-off-by: lishunyang12 <lishunyang03@gmail.com> Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com> Co-authored-by: lishunyang12 <lishunyang03@gmail.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This commit is contained in:
4 files changed
+20
-12
No files matched your search
@@ -23,9 +23,11 @@ including `librosa`.
|
||||
|
||||
The deploy config auto-loads via `--omni`.
|
||||
The default `vllm_omni/deploy/minicpmo_4_5.yaml` keeps all three stages on
|
||||
logical device 0 with memory budgets of 55%, 22%, and 22%. The profile admits
|
||||
at most four sequences per stage and bounds startup video profiling to 32
|
||||
frames per video. For throughput,`minicpmo_4_5_2gpu.yaml` gives the Thinker
|
||||
logical device 0 with memory budgets of 55%, 15%, and 15%, leaving headroom
|
||||
for runtime kernels such as the HiFi-GAN vocoder's cuDNN workspace. The
|
||||
profile admits at most four sequences per stage and bounds startup video
|
||||
profiling to 32 frames per video. For throughput,
|
||||
`minicpmo_4_5_2gpu.yaml` gives the Thinker
|
||||
GPU 0 (90%) and colocates the Talker (55%) and Code2Wav (35%) on GPU 1. That
|
||||
profile admits at most four concurrent sequences per stage.
|
||||
|
||||
|
||||
Reference in new issue
Block a user