Add native Windows CPU Qwen TTS Omni stage and profile evidence

This commit is contained in:
tzhouam committed 2026-09-23 21:18:21 +08:00
1 parent 8569995591
commit bb8735effc
24 files changed
+25090 -14

No files matched your search

+8 -2
View File
@@ -5,7 +5,9 @@ per request, with a fixed input bucket and copied host tensors. A second backend
`external.llamacpp.text.v1`, owns one pinned GGUF whole-model server process and
returns complete text requests. `external.crisp.tts.v1` owns a pinned
CustomVoice GGUF talker, codec and local hybrid CPU+Radeon process, returning
complete PCM audio. All use the normal `Omni` / `AsyncOmni` APIs,
complete PCM audio. `external.qwen_tts.cpu.v1` owns the original BF16 CustomVoice
checkpoint in an isolated native Windows CPU process and also returns complete
PCM audio. All use the normal `Omni` / `AsyncOmni` APIs,
`PipelineConfig`, `StageRuntime`, `StagePool` and orchestrator. Native vLLM stages
retain their model execution, KV cache, sampling and batching. No accelerator is
mandatory. The llama.cpp backend currently uses the complete-request graph
@@ -13,7 +15,9 @@ control path for admission and acknowledgement; it is not an ONNX graph and does
not yet support incremental token events or restart after active cancellation.
The CrispASR route likewise lacks playable incremental PCM and post-cancel
restart; its current placement gate is specific to the measured HX370 Radeon
890M shared-RAM configuration.
890M shared-RAM configuration. The Qwen CPU route likewise returns a terminal
WAV only; its worker pins an isolated Transformers 4.57.3 overlay and verifies
CPU placement and weight hashes before serving.
## Boundaries
@@ -64,6 +68,8 @@ This section preserves the 2026-09-22 audit. For the latest per-cell disposition
The [Qwen3-TTS CPU+Radeon whole-session path](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/qwen_tts_omni_hybrid/README.md) also completed two named text-to-audio inputs and 20 serial Omni requests with exact PCM parity to its standalone record. It verifies pinned executable and weights, logged hybrid placement, one-slot shared-RAM admission, terminal audio and cancellation drain. Playable streaming, post-cancel restart, loading/iGPU peaks, sustained use and broader speech quality remain open; RTF still exceeds 1. No joint CPU+iGPU+NPU claim follows from this result.
The [native Windows CPU Qwen3-TTS Omni path](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/qwen_tts_omni_cpu/README.md) completed two named text-to-WAV inputs and 20 serial Omni requests with exact PCM parity to the standalone PyTorch record. It verifies pinned weights, CPU-only placement, a 10 GiB host reservation, a 4 GiB pre-load refusal, terminal audio and cancellation drain. Request-wall p50/p95 were 15.32/15.60 s for 4.56 s of audio. Playable streaming, post-cancel restart, loading peak, broad voice quality and sustained use remain open. This CPU route does not qualify AMD or mobile execution.
**All 60 named pairings were checked on 2026-09-22** through actual runs,
raw-record review or artifact/runtime preflight. This is not 60 successful
executions. Each cell links to its evidence, finding and next required step.
+9 -2
View File
@@ -53,8 +53,15 @@ prompt took p50/p95 13.94/14.68 s to generate 4.56 s of audio. Whisper tiny.en
ASR gave word error rates 0.20 and 0.00 for the two WAVs, an intelligibility
proxy rather than a quality pass. The [raw record](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/qwen_tts_native_cpu/README.md)
documents the isolated Transformers 4.57.3 dependency and short process-memory
trace. Omni CPU stage binding, playable streaming, loading-peak admission and
sustained power/thermal tests remain M2 work.
trace. The subsequent [Omni CPU whole-session stage](../../benchmarks/edge_harness/results/e2e_expansion_20260923/evidence/qwen_tts_omni_cpu/README.md)
matched both standalone PCM hashes exactly, returned a public `AsyncOmni`
audio event, and completed one warmup plus 20 serial measured requests at
p50/p95 15.32/15.60 s for 4.56 s of audio. It reserves 10 GiB host RAM,
rejects an insufficient 4 GiB demand before worker launch, and drains an
in-flight cancellation without stale output or remaining reservation. This
closes the scoped CPU stage-binding task. Playable streaming, post-cancel
restart, loading-peak admission, broad speech quality and sustained
power/thermal tests remain M2 work.
A separate native Windows CPU+Radeon 890M Qwen3-TTS route now completes two
named text-to-WAV requests in standalone CrispASR using a Q8_0 talker, F16