anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

Track CPU talker controls and mobile speech-head profiles

Browse Source
This commit is contained in:
tzhouam committed 2026-09-24 08:01:05 +08:00
1 parent d71d39f656
commit f04fdf79da
9 files changed
+77 -6
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/design/omni_edge_e2e_roadmap.md
+1 -1
View File
@@ -19,7 +19,7 @@ The [Qwen3-TTS Code2Wav Qualcomm follow-up](../../benchmarks/edge_harness/result
The identical SA/RB3 TFLite vocoder binary also matched original ONNX CPU within 3.45e-6 relative L2 on a separate local HX370 x86-64 LiteRT 2.2.0 CPU control. Exact-SA CPU-requested parity narrows the failure to the GPU-requested route. The FP32-preserving GPU option did not fix it, so per-node placement and delegate-operation isolation remain open.
A separate [28-layer talker decode-step attempt](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) retains a synthetic fixed-cache fixture and CPU reference. Historical FP16 QNN context-binary inference executed on S24, S25, SA8775P and X Elite, but all four changed the CPU top-1 speech token with 8.57–9.43% logits relative L2. RB3 rejected that floating-point-input artifact. An explicit X Elite ONNX CPU fallback compile was pending. None advances M2 to numerical talker or full-stream qualification yet.
A separate [28-layer talker decode-step attempt](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/qwen_tts_talker_qualcomm/README.md) retains a synthetic fixed-cache fixture and CPU reference. Historical FP16 QNN context-binary inference executed on S24, S25, SA8775P and X Elite, but all four changed the CPU top-1 speech token with 8.57–9.43% logits relative L2. RB3 rejected that floating-point-input artifact. Explicit X Elite ONNX CPU and SA8775P TFLite CPU fallback compiles passed, with same-fixture inference pending. None advances M2 to numerical talker or full-stream qualification yet.
For M4, a [MiniCPM-o 4.5 speech-head study](../../benchmarks/edge_harness/results/e2e_expansion_20260924/evidence/minicpmo_tts_qualcomm/README.md) recovered the historical 20-layer ONNX source and matching S24/S25 fixtures. Both FP16 context-binary outputs preserved the ONNX CPU top-1 speech token on one decode step, with logits relative L2 0.346%/0.360%. New FP16 QNN DLC compiles passed for SA8775P, X Elite and RB3, while same-fixture NPU-requested inference remained pending. This is component C numerical evidence only, not thinker/encoder/vocoder or complete multimodal support.