[Deploy] Reduce MiniCPM-o single-GPU startup memory (#5447)

Signed-off-by: R2-Y <ruiruyang2@gmail.com>

examples/online_serving/minicpmo/README.md

@@ -23,10 +23,11 @@ including `librosa`.
 
 The deploy config auto-loads via `--omni`.
 The default `vllm_omni/deploy/minicpmo_4_5.yaml` keeps all three stages on
-logical device 0 with memory budgets of 65%, 15%, and 15%. For throughput,
+logical device 0 with memory budgets of 55%, 22%, and 22%. The profile admits
-`minicpmo_4_5_batching.yaml` gives the Thinker GPU 0 (90%) and colocates the
+at most four sequences per stage and bounds startup video profiling to 32
-Talker (55%) and Code2Wav (35%) on GPU 1. Each stage admits at most four
+frames per video. For throughput,`minicpmo_4_5_batching.yaml` gives the Thinker
-concurrent sequences.
+GPU 0 (90%) and colocates the Talker (55%) and Code2Wav (35%) on GPU 1. That
+profile admits at most four concurrent sequences per stage.
 
 | deploy config | GPUs | Notes |
 |---|---|---|
@@ -60,7 +61,7 @@ checkpoint path. To start the experimental native duplex backend, use
 
 ```bash
 vllm serve openbmb/MiniCPM-o-4_5 --omni --trust-remote-code --port 8099 \
-    --stage-overrides '{"0": {"gpu_memory_utilization": 0.65}}'
+    --stage-overrides '{"0": {"gpu_memory_utilization": 0.55}}'
 ```
 
 ## Send multimodal requests

recipes/OpenBMB/MiniCPM-o-4_5.md

@@ -79,9 +79,9 @@ a fallback because it would duplicate state machines and correctness paths.
 The default
 [`vllm_omni/deploy/minicpmo_4_5.yaml`](../../vllm_omni/deploy/minicpmo_4_5.yaml)
 co-locates Thinker, codec-only Talker, and Code2Wav on GPU 0. Their
-`gpu_memory_utilization` budgets are 0.65, 0.15, and 0.15. This layout
+`gpu_memory_utilization` budgets are 0.55, 0.22, and 0.22. All stages admit
-minimizes the GPU count; use a large-memory accelerator and leave the
+up to four sequences. Startup video profiling is bounded to 32 frames per
-remaining 5% for runtime overhead.
+video. Use the two-GPU profile for production throughput.
 
 #### Environment
 
@@ -202,8 +202,8 @@ speech output (TTS)"** checkbox on / off.
 
 #### Notes
 
-- Memory budget: Thinker, Talker, and Code2Wav reserve 0.65, 0.15, and
+- Memory budget: Thinker, Talker, and Code2Wav reserve 0.55, 0.22, and
-  0.15 of GPU 0. The larger Thinker share protects its multimodal KV cache;
+  0.22 of GPU 0. The larger Thinker share protects its multimodal KV cache;
   all three model processes still share one CUDA device.
 - `--trust-remote-code` is required — the HF repo ships a custom
   `MiniCPMO` config / model class.
@@ -214,6 +214,9 @@ speech output (TTS)"** checkbox on / off.
   contention. Talker AR
   state and Code2Wav caches are request-owned; Code2Wav batches only
   exact-shape-compatible chunks and does not fall back to serial decode.
+- `limit_mm_per_prompt.video.num_frames: 32` bounds startup dummy profiling,
+  not runtime media decoding. Use `media_io_kwargs.video.num_frames` when a
+  matching URL/file video sampling limit is required.
 - `StageRequestStats.batch_size` is a request-scoped placeholder, not the
   scheduler's execution batch.
 - Single-GPU co-location trades throughput for hardware density: Stage 0/1

tests/model_executor/models/minicpmo_4_5/test_pipeline.py

@@ -159,10 +159,11 @@ class TestDeployTopology:
         if filename == "minicpmo_4_5.yaml":
             assert [stage.yaml_engine_args["max_num_seqs"] for stage in stages] == [4, 4, 4]
             assert [stage.yaml_engine_args["gpu_memory_utilization"] for stage in stages] == [
-                0.65,
+                0.55,
-                0.15,
+                0.22,
-                0.15,
+                0.22,
             ]
+            assert stages[0].yaml_engine_args["limit_mm_per_prompt"] == {"video": {"count": 1, "num_frames": 32}}
         elif filename in {"minicpmo_4_5_batching.yaml", "minicpmo_4_5_2gpu.yaml"}:
             assert [stage.yaml_engine_args["gpu_memory_utilization"] for stage in stages] == [
                 0.9,

vllm_omni/deploy/minicpmo_4_5.yaml

@@ -18,13 +18,15 @@ connectors:
 stages:
   - stage_id: 0
     max_num_seqs: 4
-    gpu_memory_utilization: 0.65
+    gpu_memory_utilization: 0.55
     enforce_eager: false
     tensor_parallel_size: 1
     max_num_batched_tokens: 16384
     devices: "0"
     limit_mm_per_prompt:
-      video: 1
+      video:
+        count: 1
+        num_frames: 32
     default_sampling_params:
       temperature: 0.0
       top_p: 1.0
@@ -36,7 +38,7 @@ stages:
 
   - stage_id: 1
     max_num_seqs: 4
-    gpu_memory_utilization: 0.15
+    gpu_memory_utilization: 0.22
     enforce_eager: false
     max_num_batched_tokens: 8192
     devices: "0"
@@ -56,7 +58,7 @@ stages:
 
   - stage_id: 2
     max_num_seqs: 4
-    gpu_memory_utilization: 0.15
+    gpu_memory_utilization: 0.22
     enforce_eager: true
     enable_chunked_prefill: false
     max_num_batched_tokens: 65536