tzhouam and Claude Fable 5.1
8e912db6c8
[Edge][WP6] Slim the AR/TTS import path and add an edge dependency set
...
* diffusers is no longer imported by vllm_omni.diffusion.data (annotation only);
vllm_omni.diffusion.io_support (diffusion registry + distributed state) is
imported lazily by config_factory and async_omni_engine, so constructing a
Qwen3-TTS pipeline config / AsyncOmni no longer loads diffusers,
x_transformers or whisper (guarded by tests/config/test_import_guard.py).
* requirements/common.txt = core.txt (AR/speech runtime) + media.txt
(diffusion/video/extra audio); VLLM_OMNI_EDGE_BUILD=1 makes setup.py use
requirements/<device>-edge.txt (core only); `edge` extra adds websockets and
sounddevice for on-device clients.
Measured on this host (CUDA venv): `import vllm` 7.0-7.9 s, `import vllm_omni`
adds ~20 s of which ~17 s is transformers' own import (transformers.utils.import_utils)
pulled in by vllm_omni.patch / vllm_omni.version; that floor is not
vLLM-Omni's and is unchanged. Guard test: no diffusion module loaded on the TTS path.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 11:14:50 +08:00
tzhouam and Claude Fable 5.1
35e43b9983
[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)
...
* vllm_omni/platforms/cpu/platform.py: CPUOmniPlatform(OmniPlatform, CpuPlatform)
selected when VLLM_TARGET_DEVICE=cpu or a +cpu vLLM wheel is installed
(builtin plugin "cpu"; explicit CPU target short-circuits accelerator probes
like vLLM does). NUMA-based memory queries, one logical device, no graph
wrapper (sentinel type for isinstance checks), inductor opt-in via
VLLM_OMNI_CPU_INDUCTOR=1, diffusion stages unsupported.
* vllm_omni/platforms/cpu/worker: CPUARWorker / CPUGenerationWorker compose
vLLM's CPUWorker with the Omni mixin (XPU pattern); CPUARModelRunner /
CPUGenerationModelRunner compose the Omni runners with CPUModelRunner
(no graphs, no async omni output; sleep/wake return error ACKs).
* patch.py skips the CuMem patch on CPU-only torch; Qwen3-TTS speaker encoder
and prompt-embeds dtype follow the model dtype (float32/float16 on CPU).
* vllm_omni/deploy/qwen3_tts.yaml: `platforms: cpu:` overlay (eager, no async
scheduling, absolute kv_cache_memory_bytes, disjoint OMP thread ranges).
* requirements/cpu.txt (+psutil, intel-openmp), docs/getting_started/installation/cpu.md.
Measured (this x86 Xeon 8480C host, vllm 0.28.0+cpu, bf16, 12+4 threads):
Qwen3-TTS 0.6B CustomVoice end-to-end on CPU, 12 prompts: TTFA p50 292 ms,
RTF ~1.7 (25-frame chunk ~4.6 s), init 185 s. Result JSON + init timeline
committed under benchmarks/tts/edge_results/.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com >
2026-09-09 11:02:14 +08:00
SYLAR and hsliuustc0106
22cefe04b4
[Bugfix] Update Hub kernel dependency floor ( #7020 )
...
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-09-04 22:15:55 +08:00
Mu GuanLin and Hongsheng Liu
5e4daaf197
[Diffusion] Add LTX-2.5 Diffusion VAE decoder support ( #6189 )
...
Signed-off-by: mglyn <1203789601@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-27 23:54:40 +08:00
Nick Cao
faa12af454
[Misc] Bump diffusers pin to 0.40.0 ( #6459 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-08-22 12:20:34 +08:00
SYLAR
ce6ab0ca46
[Model] Add LTX-2.5 support ( #6070 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-08-15 18:27:23 +08:00
Hongsheng Liu
c57178c0e1
[Dependency] Upgrade Cache-DiT to 1.5.0 ( #6065 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-13 08:41:59 +08:00
MaciejBalaNV
e677360168
Add cosmos-guardrail dependency and update error message ( #6107 )
...
Signed-off-by: Maciej Bala <mbala@nvidia.com >
2026-08-12 17:04:07 +00:00
Ricardo Noriega
ca0ae7269c
Bump safetensors to the 0.8.0 release ( #4713 )
...
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com >
2026-07-09 15:09:23 +00:00
Zhou Taichang
a560ed184d
[Rebase] Rebase to vllm v0.24.0 ( #4709 )
2026-06-30 19:41:06 +08:00
Ian Eaves
951ae28eb0
[Frontend] Configurable video storage backends with TTL ( #2531 )
...
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com >
2026-06-19 17:41:22 -06:00
Yuzhu Dong
e2e79177db
Support DROID policy server for Cosmos3 OpenPI ( #4282 )
...
Signed-off-by: yuzhud <yuzhudong@gmail.com >
2026-06-15 21:22:59 +08:00
Zhou Taichang
7b837944a1
[Rebase] Rebase to vllm 0.23.0 ( #4286 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-06-13 16:43:02 +00:00
zyz111222 and Canlin Guo
6de0ca36bb
[Hardware][Ascend] Adapt Qwen3 TTS for 310P ( #4283 )
...
Signed-off-by: zouyizhou <zouyizhou@huawei.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
2026-06-12 09:39:21 +00:00
FED4 and Gao Han
6ae355b9a9
[BugFix] Remove pydub dependency for Python 3.13 compatibility ( #4035 )
...
Signed-off-by: FED4 <feiyyfrank@outlook.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-06-09 20:45:57 -07:00
Chendi.Xue
f174ee54b0
[XPU] Add sage_attn backend ( #3785 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
2026-06-09 19:28:18 +08:00
+1
f92d84fcf9
[Rebase] Rebase to vllm releases/v0.22.0 ( #3891 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: ZhengWG <zwg0606@gmail.com >
Signed-off-by: david6666666 <530634352@qq.com >
Signed-off-by: Yueqian Lin <linyueqian@outlook.com >
Signed-off-by: nainiu258 <cperfect02@163.com >
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com >
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com >
Co-authored-by: Zheng Wengang <zwg0606@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Yueqian Lin <linyueqian@outlook.com >
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com >
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2026-06-01 11:08:12 +08:00
Ricardo Noriega
83bbe39d39
Bump diffusers minimum version to >=0.38.0 ( #3349 )
...
Signed-off-by: Ricardo Noriega De Soto <rnoriega@redhat.com >
2026-05-13 10:25:02 +08:00
Zhou Taichang and Yuanheng Zhao
7f2b1e9386
[Rebase] Rebase to vllm 0.20.0 ( #3232 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com >
2026-04-30 09:56:59 +12:00
R0CKSTAR
5f53dccd15
[MUSA][Chore] Bump torchada ( #3132 )
...
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com >
2026-04-27 05:03:18 +00:00
R0CKSTAR
8384dead56
[MUSA][Feat] Upgrade MATE to match CUDA/ROCm behavior on FA ( #2766 )
...
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com >
2026-04-23 03:07:02 +00:00
Nick Cao
d2c23d7bd0
[Refactor] Remove resampy dependency ( #2891 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-04-18 03:20:36 +00:00
Nick Cao and Claude
1e3bb36521
[Refactor] Remove sox from dependencies ( #2745 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-17 01:24:47 +00:00
Nick Cao and Claude
2bc183f6f0
[Refactor] Remove dependency on librosa ( #2273 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
Co-authored-by: Claude <noreply@anthropic.com >
2026-04-11 02:11:31 +08:00
汪志鹏 and Didan Deng
0059ec878c
[Model Support]: Magihuman support ( #2301 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-04-04 09:29:09 +00:00
R0CKSTAR
3def008b32
[Feat] Add MUSA platform support for Moore Threads GPUs ( #2337 )
...
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com >
2026-04-01 17:55:00 +08:00
Lidang Jiang
168bea2440
[Misc] Make gradio an optional dependency and upgrade to >=6.7.0 ( #2221 )
...
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com >
Signed-off-by: Lidang Jiang <lidangjiang@gmail.com >
2026-03-28 12:33:55 -04:00
3158e9d28f
[Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring ( #1908 )
...
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn >
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: wuhang <whlbx@hotmail.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn >
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com >
Co-authored-by: wuhang <whlbx@hotmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: linyueqian <linyueqian@outlook.com >
Co-authored-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: lishunyang <lishunyang12@163.com >
2026-03-18 17:33:53 +08:00
Samit
4dbaa74f2b
[Enhancement] Upgrade cache-dit from 1.2.0 to 1.3.0 ( #1834 )
...
Signed-off-by: samithuang <285365963@qq.com >
2026-03-12 15:39:06 +08:00
dsinghvi and Hongsheng Liu
b63e7c7c9a
[Model] Fun cosy voice3-0.5-b-2512 ( #498 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-03-04 19:54:59 -05:00
36b8f80266
[Feature] Support Stage Based Deployment CLI ( #939 )
...
Signed-off-by: wuhang <wuhang6@huawei.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: wuhang <whlbx@hotmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-02-24 16:55:23 +08:00
75770c9309
[Rebase] Rebase to vllm v0.16.0 ( #1357 )
...
ComfyUI Integration -- Publish to Comfy registry / Publish Custom Node to registry (push) Failing after 44s
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <Isotr0py@outlook.com >
Co-authored-by: ZJY0516 <zhu.jiangyun@foxmail.com >
Co-authored-by: youkaichao <youkaichao@gmail.com >
2026-02-13 18:05:07 +08:00
ab1acb27ff
[Feature]async scheduling to overlap chunk IO and compute ( #951 )
...
Signed-off-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Bhanu068 <voutharoja.bhanu06@gmail.com >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2026-02-12 16:05:02 +08:00
Junhong Liu and Hongsheng Liu
9af6fb9a1f
[Feature] Opt metrics structure ( #891 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Junhong Liu <ljh_lbj@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-09 19:17:32 +08:00
78a5aae299
[Hardware] [Feat] Setup platform dependent package installation ( #1046 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
Co-authored-by: PopSoda2002 <zhouhp.me@gmail.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
2026-02-05 21:47:41 +08:00
