tzhouamandClaude Fable 5.1
35e43b9983
[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)
* vllm_omni/platforms/cpu/platform.py: CPUOmniPlatform(OmniPlatform, CpuPlatform)
selected when VLLM_TARGET_DEVICE=cpu or a +cpu vLLM wheel is installed
(builtin plugin "cpu"; explicit CPU target short-circuits accelerator probes
like vLLM does). NUMA-based memory queries, one logical device, no graph
wrapper (sentinel type for isinstance checks), inductor opt-in via
VLLM_OMNI_CPU_INDUCTOR=1, diffusion stages unsupported.
* vllm_omni/platforms/cpu/worker: CPUARWorker / CPUGenerationWorker compose
vLLM's CPUWorker with the Omni mixin (XPU pattern); CPUARModelRunner /
CPUGenerationModelRunner compose the Omni runners with CPUModelRunner
(no graphs, no async omni output; sleep/wake return error ACKs).
* patch.py skips the CuMem patch on CPU-only torch; Qwen3-TTS speaker encoder
and prompt-embeds dtype follow the model dtype (float32/float16 on CPU).
* vllm_omni/deploy/qwen3_tts.yaml: `platforms: cpu:` overlay (eager, no async
scheduling, absolute kv_cache_memory_bytes, disjoint OMP thread ranges).
* requirements/cpu.txt (+psutil, intel-openmp), docs/getting_started/installation/cpu.md.
Measured (this x86 Xeon 8480C host, vllm 0.28.0+cpu, bf16, 12+4 threads):
Qwen3-TTS 0.6B CustomVoice end-to-end on CPU, 12 prompts: TTFA p50 292 ms,
RTF ~1.7 (25-frame chunk ~4.6 s), init 185 s. Result JSON + init timeline
committed under benchmarks/tts/edge_results/.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-09 11:02:14 +08:00
..
2026-09-09 11:01:40 +08:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-07-28 03:58:22 +00:00
2026-08-03 23:41:52 +08:00
2026-07-28 03:58:22 +00:00
2026-07-22 09:44:15 +08:00
2026-08-20 07:09:12 +08:00
2026-04-24 21:21:30 +08:00
2026-07-22 09:44:15 +08:00
2026-07-29 02:46:27 +00:00
2026-05-12 14:39:58 +08:00
2026-08-13 17:58:33 +00:00
2026-07-28 17:14:26 +08:00
2026-07-05 07:17:58 +08:00
2026-06-02 21:28:12 +00:00
2026-07-22 09:44:15 +08:00
2026-06-02 21:28:12 +00:00
2026-08-20 07:09:12 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-08-14 22:06:05 -07:00
2026-05-27 12:18:47 +12:00
2026-05-24 05:42:26 +08:00
2026-06-20 11:45:26 +08:00
2026-07-22 09:44:15 +08:00
2026-08-13 10:00:50 -07:00
2026-08-13 10:00:50 -07:00
2026-08-30 08:52:15 +00:00
2026-05-11 22:49:19 +08:00
2026-07-24 14:45:26 +00:00
2026-07-22 09:44:15 +08:00
2026-07-18 09:34:33 +00:00
2026-08-11 23:00:35 +00:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-06-01 16:01:25 +08:00
2026-09-07 13:38:23 +08:00
2026-08-28 08:32:32 +08:00
2026-08-28 08:32:32 +08:00
2026-08-26 16:26:24 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-05-03 14:43:49 +12:00
2026-05-03 14:43:49 +12:00
2026-05-03 14:43:49 +12:00
2026-07-12 19:50:51 -07:00
2026-07-13 06:45:28 +08:00
2026-08-29 23:27:47 +08:00
2026-08-26 10:56:51 +08:00
2026-08-29 23:27:47 +08:00
2026-08-26 10:56:51 +08:00
2026-08-26 10:56:51 +08:00
2026-08-26 10:56:51 +08:00
2026-09-06 22:25:35 +08:00
2026-08-28 14:27:05 +08:00
2026-08-28 14:27:05 +08:00
2026-08-18 04:33:13 +00:00
2026-08-18 04:33:13 +00:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-08-13 08:56:26 +00:00
2026-08-20 07:09:12 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-07-22 09:44:15 +08:00
2026-09-03 18:49:19 -07:00
2026-09-03 18:49:19 -07:00
2026-09-03 18:49:19 -07:00
2026-07-08 00:44:33 +00:00
2026-08-04 20:10:33 -07:00
2026-08-17 14:29:17 +08:00
2026-08-11 23:00:35 +00:00
2026-06-10 08:37:33 +08:00
2026-08-27 08:21:03 +00:00
2026-09-04 21:16:58 +08:00
2026-08-12 05:33:34 +00:00
2026-08-28 02:11:05 -07:00
2026-09-09 11:02:14 +08:00
2026-08-13 10:00:50 -07:00
2026-07-22 09:44:27 +00:00
2026-07-18 09:34:33 +00:00
2026-07-18 09:34:33 +00:00
2026-07-18 09:34:33 +00:00
2026-07-04 17:47:11 +00:00
2026-07-28 07:35:35 +08:00
2026-09-01 01:02:54 +00:00