 tzhouamandClaude Fable 5.1
|
0356b22e4c
|
edge results: chunk hop put->get stats, shm vs cuda_ipc (0.6B, chain3)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 16:08:22 +08:00 |
|
|
tzhouamandClaude Fable 5.1
|
5104fb6d67
|
edge results: edge_final2 (parallel init), GPU step-stats (default/edge/cuda_ipc), codec stream parity run 1
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 16:02:17 +08:00 |
|
|
tzhouamandClaude Fable 5.1
|
a786a88919
|
edge results: cold-cache init profile, GPU hardware emulation, CPU thread sweep, edge_final benches
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 15:23:08 +08:00 |
|
|
tzhouamandClaude Fable 5.1
|
285db2fa3c
|
edge results: WP1 edge-profile stream latency (0.6B/1.7B) + init timelines
1.7B: TTFA p50 43.9 ms, playback_start 43.9 ms, stall 0 (baseline 32.0 / 116.2 / 84.7).
0.6B: TTFA p50 46.1 ms, playback_start 46.1 ms, stall 0 (baseline 27.8 / 95.4 / 67.3).
RTF unchanged (~0.1). Measured on L20X via gpu run --gpus 1 (canhazgpu).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 11:21:48 +08:00 |
|
|
tzhouamandClaude Fable 5.1
|
35e43b9983
|
[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)
* vllm_omni/platforms/cpu/platform.py: CPUOmniPlatform(OmniPlatform, CpuPlatform)
selected when VLLM_TARGET_DEVICE=cpu or a +cpu vLLM wheel is installed
(builtin plugin "cpu"; explicit CPU target short-circuits accelerator probes
like vLLM does). NUMA-based memory queries, one logical device, no graph
wrapper (sentinel type for isinstance checks), inductor opt-in via
VLLM_OMNI_CPU_INDUCTOR=1, diffusion stages unsupported.
* vllm_omni/platforms/cpu/worker: CPUARWorker / CPUGenerationWorker compose
vLLM's CPUWorker with the Omni mixin (XPU pattern); CPUARModelRunner /
CPUGenerationModelRunner compose the Omni runners with CPUModelRunner
(no graphs, no async omni output; sleep/wake return error ACKs).
* patch.py skips the CuMem patch on CPU-only torch; Qwen3-TTS speaker encoder
and prompt-embeds dtype follow the model dtype (float32/float16 on CPU).
* vllm_omni/deploy/qwen3_tts.yaml: `platforms: cpu:` overlay (eager, no async
scheduling, absolute kv_cache_memory_bytes, disjoint OMP thread ranges).
* requirements/cpu.txt (+psutil, intel-openmp), docs/getting_started/installation/cpu.md.
Measured (this x86 Xeon 8480C host, vllm 0.28.0+cpu, bf16, 12+4 threads):
Qwen3-TTS 0.6B CustomVoice end-to-end on CPU, 12 prompts: TTFA p50 292 ms,
RTF ~1.7 (25-frame chunk ~4.6 s), init 185 s. Result JSON + init timeline
committed under benchmarks/tts/edge_results/.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 11:02:14 +08:00 |
|
|
tzhouamandClaude Fable 5.1
|
f804bbd679
|
[Edge][WP0] Record Qwen3-TTS streaming baselines on 1x L20X
Default deploy config (1 -> 25 frame schedule), 12 prompts x 3 repeats,
warm-up excluded, one GPU via `gpu run`:
1.7B CustomVoice: TTFA p50 32.0 ms, playback-start p50 116 ms (stall at
TTFA 85 ms), RTF_total 0.086, chunk gap p50 163 ms, init 271 s (cold caches)
0.6B CustomVoice: TTFA p50 27.8 ms, playback-start p50 95 ms (stall 67 ms),
RTF_total 0.077, chunk gap p50 146 ms, init 152 s
Per-request chunk arrivals, memory and the full summaries are in the JSON
(force-added: *.json is gitignored repo-wide).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 10:49:37 +08:00 |
|