From e0142926b66b9b8a98e2f9fc59b25b4d7a472450 Mon Sep 17 00:00:00 2001 From: tzhouam Date: Wed, 23 Sep 2026 10:19:10 +0800 Subject: [PATCH] Document MiniCPM-o checkpoint preflight and current deploy layout --- .../results/e2e_recovery_20260923/README.md | 2 ++ examples/offline_inference/minicpmo/README.md | 10 ++++++---- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/benchmarks/edge_harness/results/e2e_recovery_20260923/README.md b/benchmarks/edge_harness/results/e2e_recovery_20260923/README.md index 6ef886176..01da0fee2 100644 --- a/benchmarks/edge_harness/results/e2e_recovery_20260923/README.md +++ b/benchmarks/edge_harness/results/e2e_recovery_20260923/README.md @@ -13,4 +13,6 @@ The code changes make vLLM-native `qwen3_5` a one-stage Omni pipeline, project e The missing primary MiniCPM-o 4.5 and InternVLA-A1 checkpoints are being acquired at pinned revisions; no E2E result is claimed until their transfers finish and real checkpoint runs pass. The required InternVLA Cosmos encoder/decoder and Qwen3-VL processor files have been downloaded separately. Existing Spark-X2.5 4B and 1.7B INT8, Qwen3-TTS, Qwen3.8 FP8 and Qwen3.8 NVFP4 artifacts already covered those model families. Additional official Spark 1.7B BF16 and Qwen3.8-27B BF16 transfers were paused when the Hugging Face data path slowed; they are distinct artifact variants, not replacements for the present weights. InternVLA source checkpoints carry a CC BY-NC-SA 4.0 license; the current download is for local research validation. +The pinned MiniCPM-o checkpoint (`openbmb/MiniCPM-o-4_5`, `503e754207c94da6bb26850b4469f367c9ea3582`) has four weight shards totaling 18.74 GB (decimal) within a 20.05 GB repository. InternVLA base (`InternRobotics/InternVLA-A1-3B`, `08cf716d413aa8208a34f346cd8824e6ff901205`) and Place_Markpen fine-tune (`Jia-Zeng/InternVLA-A1-3B-FineTuned-Place_Markpen`, `0a557c6a503e6545b369bff2f34c76a410bbd190`) each have a 6.72 GB weight file. Their partial Xet files are **not** complete artifacts. MiniCPM-o's small configuration and processor files did finish; offline `AutoConfig` and `AutoProcessor` construction passed against the local path. This is a dependency preflight, not a weight load or E2E run. The large weight transfers have been throttled by the Hugging Face data path and remain in progress. + The other rows still need actual local devices, a matching complete backend/artifact, memory admission, and model-specific quality and profiling. In particular, no ADB device is attached, native Windows CPU attention operations are missing from the installed extension, AMD iGPU/NPU routes lack a verified stateful whole-model deployment, and there is no qualified joint CPU+iGPU+NPU execution plan. AI Hub component results remain component evidence. diff --git a/examples/offline_inference/minicpmo/README.md b/examples/offline_inference/minicpmo/README.md index b827a4e98..0bde9dbf5 100644 --- a/examples/offline_inference/minicpmo/README.md +++ b/examples/offline_inference/minicpmo/README.md @@ -1,8 +1,8 @@ # MiniCPM-o 4.5: Offline inference -Two-stage pipeline: **thinker** (multimodal understanding) → **talker + Token2Wav** -(24 kHz speech). Deploy config auto-loads from -`vllm_omni/deploy/minicpmo_4_5.yaml` (2-GPU default). +Three-stage pipeline: **thinker** (multimodal understanding) → **talker** → +**Code2Wav** (24 kHz speech). Deploy config auto-loads from +`vllm_omni/deploy/minicpmo_4_5.yaml` (single-GPU compatibility layout). ## Setup @@ -83,7 +83,9 @@ python end2end.py --query-type text \ - Output WAV is **24 kHz mono**. - Placeholders in the prompt are MiniCPM-style: `(./)`, `()`, `()`. -- Default layout needs **2 GPUs**. Async chunking is off in the bundled YAMLs. +- The default layout places all three stages on one large-memory GPU. Check + admission on the actual device before running; a 24 GiB GPU is not presumed + sufficient for this checkpoint and its runtime state. ## Online serving