vllm-omni/vllm_omni/model_executor/models at 35e43b9983fc85fbc9e4650de3923ee3a9cb3614 - vllm-omni - Gitea: Git with a cup of tea

anodyine/vllm-omni

0 Fork 0

Files

T

History

tzhouamandClaude Fable 5.1 35e43b9983 [Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)

* vllm_omni/platforms/cpu/platform.py: CPUOmniPlatform(OmniPlatform, CpuPlatform)
  selected when VLLM_TARGET_DEVICE=cpu or a +cpu vLLM wheel is installed
  (builtin plugin "cpu"; explicit CPU target short-circuits accelerator probes
  like vLLM does). NUMA-based memory queries, one logical device, no graph
  wrapper (sentinel type for isinstance checks), inductor opt-in via
  VLLM_OMNI_CPU_INDUCTOR=1, diffusion stages unsupported.
* vllm_omni/platforms/cpu/worker: CPUARWorker / CPUGenerationWorker compose
  vLLM's CPUWorker with the Omni mixin (XPU pattern); CPUARModelRunner /
  CPUGenerationModelRunner compose the Omni runners with CPUModelRunner
  (no graphs, no async omni output; sleep/wake return error ACKs).
* patch.py skips the CuMem patch on CPU-only torch; Qwen3-TTS speaker encoder
  and prompt-embeds dtype follow the model dtype (float32/float16 on CPU).
* vllm_omni/deploy/qwen3_tts.yaml: `platforms: cpu:` overlay (eager, no async
  scheduling, absolute kv_cache_memory_bytes, disjoint OMP thread ranges).
* requirements/cpu.txt (+psutil, intel-openmp), docs/getting_started/installation/cpu.md.

Measured (this x86 Xeon 8480C host, vllm 0.28.0+cpu, bf16, 12+4 threads):
Qwen3-TTS 0.6B CustomVoice end-to-end on CPU, 12 prompts: TTFA p50 292 ms,
RTF ~1.7 (25-frame chunk ~4.6 s), init 185 s. Result JSON + init timeline
committed under benchmarks/tts/edge_results/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-09-09 11:02:14 +08:00

..

[Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149 )

2026-08-25 15:24:59 +08:00

[Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149 )

2026-08-25 15:24:59 +08:00

[Config] Return VllmOmniConfig from create_from_model (#4818 )

2026-07-18 09:21:07 +00:00

[Edge][WP2] Add initialization timeline and cold-start profiler

2026-09-09 11:01:39 +08:00

[Bugfix][TTS] Make CosyVoice3 TRT plan publication concurrency-safe (#6955 )

2026-09-03 06:44:42 +00:00

[Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149 )

2026-08-25 15:24:59 +08:00

[Feat][dots.tts] Add Online Serving Support (#6235 )

2026-09-04 14:44:10 -07:00

[Config] Return VllmOmniConfig from create_from_model (#4818 )

2026-07-18 09:21:07 +00:00

[Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149 )

2026-08-25 15:24:59 +08:00

[Config] Return VllmOmniConfig from create_from_model (#4818 )

2026-07-18 09:21:07 +00:00

[Model] Add Gepard-1.0 native-AR FSQ/NanoCodec TTS (offline inference) (#5666 )

2026-08-14 22:06:05 -07:00

[Perf][GLM-Image] Skip redundant hidden state D2H (#5346 )

2026-07-28 20:34:35 +08:00

[Perf][TTS] glm_tts: hoist loop-invariant text embed, RoPE, mask and CFG batch out of Euler loop (#5068 )

2026-08-12 17:02:02 +00:00

[Config] Return VllmOmniConfig from create_from_model (#4818 )

2026-07-18 09:21:07 +00:00

[Config] Return VllmOmniConfig from create_from_model (#4818 )

2026-07-18 09:21:07 +00:00

[Bugfix] Fix Higgs Audio v3 voice-clone token validation (#7065 )

2026-09-04 16:06:47 -07:00

[Bugfix] Fix Hunyuan Image-3 expert mapping contract (#7021 )

2026-09-06 21:43:37 +08:00

[Refactor] Migrate Legacy Stage Configs to the Pipeline Registry (#5031 )

2026-07-18 09:34:33 +00:00

[Bugfix] Fix fallback loading for IndexTTS external models (#6137 )

2026-09-04 19:58:33 -07:00

[Config] Return VllmOmniConfig from create_from_model (#4818 )

2026-07-18 09:21:07 +00:00

[Feature][LingBot World]stepwise execution (#6844 )

2026-09-07 13:38:23 +08:00

[Fix] Restore MammothModa2 support (#6694 )

2026-08-28 08:32:32 +08:00

[Bugfix][MiMo-Audio] Restore make_empty_intermediate_tensors on the talker for vLLM 0.28 (#6803 )

2026-08-31 15:26:35 +08:00

ming_flash_omni

[Ming] Cleanup Ming-family shared modules (#6119 )

2026-08-19 14:13:40 +08:00

[Ming] Cleanup Ming-family shared modules (#6119 )

2026-08-19 14:13:40 +08:00

[Model] Preserve MiniCPM-o Talker full attention until capacity (#6799 )

2026-09-06 22:25:35 +08:00

[Model] MiniMax-H3: validate versioned text-conditioning handoff (#6720 )

2026-09-06 21:45:10 +08:00

[Bugfix][MiniMax-Music3] Resolve stage subdirs against a real snapshot (#6640 )

2026-08-30 06:51:24 +00:00

[Feature][Moss-TTS] Reference Audio Preprocessing (#4982 )

2026-09-04 16:36:26 -07:00

[Config] Return VllmOmniConfig from create_from_model (#4818 )

2026-07-18 09:21:07 +00:00

nemotron_voicechat

[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196 )

2026-09-05 08:20:21 +00:00

[Perf][Bugfix][OmniVoice] Restore float16 serving, and fuse the generator hot loop (#6317 )

2026-09-02 19:54:55 +00:00

[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196 )

2026-09-05 08:20:21 +00:00

[Bugfix][Qwen2.5-Omni] Fix two stale call sites in the speech helpers (#6886 )

2026-09-02 21:41:05 -07:00

[Bugfix] Give exact-shape codec models an unpadded input_ids view (#6775 )

2026-09-04 15:28:49 -07:00

[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)

2026-09-09 11:02:14 +08:00

[Bugfix][TTS] Isolate concurrent StepAudio2 TRT builds (#6957 )

2026-09-02 19:53:02 +00:00

[Refactor] Move model-specific capability metadata into TTS adapters (#6138 )

2026-08-22 14:26:49 +08:00

[Perf][TTS] voxtral_tts hot path: cudagraph opt-out, host-sync removal, kernel caches (#5175 )

2026-08-10 21:03:32 -07:00

[FEAT]Add FastVideo VSA backend for Wan2.2 (#4820 )

2026-08-24 14:48:31 +08:00

__init__.py

[Core] Unified quantization framework (#1764 )

2026-03-24 18:56:55 +08:00

output_templates.py

[Refactor] Standardize data entry key names to {type}.{qualifier} format (#1829 )

2026-04-25 18:15:38 +08:00

registry.py

[model]Add experimental Mage-VL full-duplex adapter and offline integration (#6537 )

2026-09-02 20:25:32 -07:00

utils.py

[Rebase] Rebase to vllm 0.27.0 (#5976 )

2026-08-11 15:23:46 +00:00

whisper_utils.py

[Model] Add Ming-flash-omni-2.0 Thinker Stage (#1822 )

2026-04-17 19:10:02 +08:00