Qualify scoped MiniCPM-o CPU speech and Windows InternVLA policy paths

This commit is contained in:
tzhouam committed 2026-09-23 16:21:28 +08:00
1 parent 3f2cd52939
commit 2f5d0edcf7
37 files changed
+4618 -27

No files matched your search

+11 -2
View File
@@ -13,6 +13,15 @@ three-stage text-to-speech functional pass after correcting request embeddings,
and an InternVLA real-weight synthetic action forward. Model-quality and device gates
remain open; the 2026-09-22 audit remains the historical baseline.
The [WSL CPU expansion](../../benchmarks/edge_harness/results/e2e_expansion_20260923/README.md)
adds constrained three-stage MiniCPM-o text-to-speech requests and a CPU-only
InternVLA synthetic policy forward. A subsequent 20-request serial MiniCPM-o
profile measured p50 21.86 s/p95 22.97 s request wall time after one warmup, with
swap use under the 30.91 GiB WSL limit. It does not qualify speech quality,
concurrency, streaming, loading peak or sustained behavior. Native Windows
InternVLA direct-policy runs separately passed on CPU and RTX 5090 Laptop with
synthetic inputs; AMD and mobile cells do not inherit these passes.
## Architecture and prerequisites
Keep Omni's PipelineConfig, StageRuntime, StageClient, orchestration and admission
@@ -68,8 +77,8 @@ capability and lifecycle implementation; a state-handle type alone is insufficie
| M1 | S25 Spark | Device-local Omni-compatible controller; embedding, prefill, continuous decode, ring/full KV, head and sampler; resident state | Reference checks, at least 128 output tokens, 512-window/1024-bucket transitions, cancellation, resident memory and sustained profile |
| M2 | Qwen3-TTS | Resolve desktop playback and shutdown findings and CPU compatibility; then mobile talker/predictor plus GPU vocoder | Complete PCM and tail, history/chunk correctness, interruption/recovery, quality checks, no post-startup underruns and RTF below 1 in the declared workload |
| M3 | AMD stages | Connect compatible real encoders or other useful coarse stages through current Omni workers; start from the existing 890M vision evidence | Actual node/device placement, numerical and task quality, complete downstream output, shared-memory and handoff costs; reject unhelpful splits |
| M4a | MiniCPM-o 4.5 | Three-stage desktop text+WAV path produced coherent text and nonzero WAV under a constrained 24 GiB profile after fixing uninitialized request embeddings; add encoder and persistent-state reference before mobile artifacts | Speech intelligibility/alignment, separate text, image and audio-understanding suites, then combined streaming and interruption |
| M4b | InternVLA-A1 | Real weights, observation preprocessing, shared prefix state, iterative action head and versioned action metadata; one strict-load synthetic action forward now passes on RTX 5090 Laptop | Reference action agreement on real observations, units/order/timestamps, observation age, stale-input handling and application-specific deadline reporting |
| M4a | MiniCPM-o 4.5 | Three-stage desktop text+WAV produced coherent text and nonzero WAV on RTX 5090 Laptop and WSL CPU under separate constrained plans. The CPU plan needed vLLM 0.28/0.29 processor compatibility and used swap; 20 serial CPU requests measured p50 21.86 s/p95 22.97 s. Add encoder and persistent-state reference before mobile artifacts | Speech intelligibility/alignment, CPU loading-peak admission, concurrency, separate text, image and audio-understanding suites, then combined streaming and interruption |
| M4b | InternVLA-A1 | Real weights, observation preprocessing, shared prefix state, iterative action head and versioned action metadata; strict-load synthetic policy forwards now pass on WSL and native Windows, each on CPU and RTX 5090 Laptop | Reference action agreement on real observations, units/order/timestamps, observation age, stale-input handling and application-specific deadline reporting |
| Independent | Qwen3.8-27B | Public Omni pipeline binding; text, then image/video and longer contexts; same-model CPU/iGPU artifact and explicit offload evaluation | Modality-specific quality, actual loading/runtime memory and measured performance; each precision/context receives its own qualification |
The initial M0 desktop text acceptance remains scoped to its tested checkpoint,