vllm-omni/vllm_omni at 35e43b9983fc85fbc9e4650de3923ee3a9cb3614 - vllm-omni - Gitea: Git with a cup of tea

anodyine/vllm-omni

0 Fork 0

Files

T

History

tzhouamandClaude Fable 5.1 35e43b9983 [Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)

* vllm_omni/platforms/cpu/platform.py: CPUOmniPlatform(OmniPlatform, CpuPlatform)
  selected when VLLM_TARGET_DEVICE=cpu or a +cpu vLLM wheel is installed
  (builtin plugin "cpu"; explicit CPU target short-circuits accelerator probes
  like vLLM does). NUMA-based memory queries, one logical device, no graph
  wrapper (sentinel type for isinstance checks), inductor opt-in via
  VLLM_OMNI_CPU_INDUCTOR=1, diffusion stages unsupported.
* vllm_omni/platforms/cpu/worker: CPUARWorker / CPUGenerationWorker compose
  vLLM's CPUWorker with the Omni mixin (XPU pattern); CPUARModelRunner /
  CPUGenerationModelRunner compose the Omni runners with CPUModelRunner
  (no graphs, no async omni output; sleep/wake return error ACKs).
* patch.py skips the CuMem patch on CPU-only torch; Qwen3-TTS speaker encoder
  and prompt-embeds dtype follow the model dtype (float32/float16 on CPU).
* vllm_omni/deploy/qwen3_tts.yaml: `platforms: cpu:` overlay (eager, no async
  scheduling, absolute kv_cache_memory_bytes, disjoint OMP thread ranges).
* requirements/cpu.txt (+psutil, intel-openmp), docs/getting_started/installation/cpu.md.

Measured (this x86 Xeon 8480C host, vllm 0.28.0+cpu, bf16, 12+4 threads):
Qwen3-TTS 0.6B CustomVoice end-to-end on CPU, 12 prompts: TTFA p50 292 ms,
RTF ~1.7 (25-frame chunk ~4.6 s), init 185 s. Result JSON + init timeline
committed under benchmarks/tts/edge_results/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-09-09 11:02:14 +08:00

..

[Rebase] Rebase to vllm 0.26.0 (#5443 )

2026-07-27 23:35:49 +08:00

[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196 )

2026-09-05 08:20:21 +00:00

[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196 )

2026-09-05 08:20:21 +00:00

[Feature][LingBot World]stepwise execution (#6844 )

2026-09-07 13:38:23 +08:00

[Model] Preserve MiniCPM-o Talker full attention until capacity (#6799 )

2026-09-06 22:25:35 +08:00

[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)

2026-09-09 11:02:14 +08:00

[Perf][Diffusion] Keep LoRA weights resident across activation cycles (#7195 )

2026-09-07 14:27:24 +00:00

[Model] Preserve MiniCPM-o Talker full attention until capacity (#6799 )

2026-09-06 22:25:35 +08:00

[Edge][WP4] Add hardware adaptation: probe, derived overrides, auto profile, calibration

2026-09-09 11:01:40 +08:00

[Edge][WP2] Add initialization timeline and cold-start profiler

2026-09-09 11:01:39 +08:00

[Edge][WP2] Add initialization timeline and cold-start profiler

2026-09-09 11:01:39 +08:00

[Feature][LingBot World]stepwise execution (#6844 )

2026-09-07 13:38:23 +08:00

host_weight_runtime

[Bugfix] Make HWR filesystem lifecycle transitions fail-closed (#6692 )

2026-08-28 09:30:17 +08:00

[Core][Diffusion] Refactor prompt updates into a modality-ready interaction framework (#6294 )

2026-09-07 11:40:04 +08:00

[Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment (#758 )

2026-01-27 10:22:40 +08:00

[Bugfix][Qwen3-TTS] Handle codec generations that exhaust their token budget (#6728 )

2026-08-29 08:59:05 -07:00

[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)

2026-09-09 11:02:14 +08:00

[Model] Add native MAGI-2 Preview diffusion support (#5918 )

2026-09-05 15:32:30 +08:00

[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196 )

2026-09-05 08:20:21 +00:00

[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)

2026-09-09 11:02:14 +08:00

[Quantization] Migrate GGUF diffusion model support to OOT plugin (#4769 )

2026-07-11 18:48:43 +00:00

[Enhancement]modify profiling.md (#3051 )

2026-04-24 16:30:24 +08:00

[Model] FP8 Support For Boogu-Image (#6925 )

2026-09-06 04:48:38 +00:00

Step audio R1 reasoning parser (#2846 )

2026-06-10 21:03:19 +00:00

update dirs

2025-09-11 15:43:43 +08:00

[Fix] Restore MammothModa2 support (#6694 )

2026-08-28 08:32:32 +08:00

transformers_utils

[Feat][dots.tts] Add Online Serving Support (#6235 )

2026-09-04 14:44:10 -07:00

[Refactor] Move model-specific capability metadata into TTS adapters (#6138 )

2026-08-22 14:26:49 +08:00

[Edge][WP2] Add initialization timeline and cold-start profiler

2026-09-09 11:01:39 +08:00

__init__.py

[Rebase] Rebase to vllm 0.26.0 (#5443 )

2026-07-27 23:35:49 +08:00

data_entry_keys.py

[Perf][NemotronVoiceChat] Fast defaults for all three deploy profiles: 2.8x offline, realtime full-duplex audio (#6354 )

2026-09-03 18:49:19 -07:00

errors.py

[refactor] Refactor guardrail error handling - add 400 error code (#4297 )

2026-06-11 21:09:24 +00:00

logger.py

[BugFix] Failed to print DEBUG level log (#307 )

2025-12-15 07:13:36 +00:00

patch.py

[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)

2026-09-09 11:02:14 +08:00

request.py

[Rebase] Rebase to vllm 0.28.0 (#6606 )

2026-08-27 06:12:55 +00:00

version.py

[Misc] Warn When vLLM / vLLM-Omni Have Mismatched Versions (#2691 )

2026-04-17 20:15:16 +08:00