 tzhouamandClaude Fable 5.1
|
35e43b9983
|
[Edge][WP3] Add CPU OmniPlatform (AR + generation stages on vLLM's CPU backend)
* vllm_omni/platforms/cpu/platform.py: CPUOmniPlatform(OmniPlatform, CpuPlatform)
selected when VLLM_TARGET_DEVICE=cpu or a +cpu vLLM wheel is installed
(builtin plugin "cpu"; explicit CPU target short-circuits accelerator probes
like vLLM does). NUMA-based memory queries, one logical device, no graph
wrapper (sentinel type for isinstance checks), inductor opt-in via
VLLM_OMNI_CPU_INDUCTOR=1, diffusion stages unsupported.
* vllm_omni/platforms/cpu/worker: CPUARWorker / CPUGenerationWorker compose
vLLM's CPUWorker with the Omni mixin (XPU pattern); CPUARModelRunner /
CPUGenerationModelRunner compose the Omni runners with CPUModelRunner
(no graphs, no async omni output; sleep/wake return error ACKs).
* patch.py skips the CuMem patch on CPU-only torch; Qwen3-TTS speaker encoder
and prompt-embeds dtype follow the model dtype (float32/float16 on CPU).
* vllm_omni/deploy/qwen3_tts.yaml: `platforms: cpu:` overlay (eager, no async
scheduling, absolute kv_cache_memory_bytes, disjoint OMP thread ranges).
* requirements/cpu.txt (+psutil, intel-openmp), docs/getting_started/installation/cpu.md.
Measured (this x86 Xeon 8480C host, vllm 0.28.0+cpu, bf16, 12+4 threads):
Qwen3-TTS 0.6B CustomVoice end-to-end on CPU, 12 prompts: TTFA p50 292 ms,
RTF ~1.7 (25-frame chunk ~4.6 s), init 185 s. Result JSON + init timeline
committed under benchmarks/tts/edge_results/.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
2026-09-09 11:02:14 +08:00 |
|
|
|
fe4a2a7477
|
[Diffusion] Add paged KV cache support for HunyuanImage3 DiT on NPU and GPU (#6563)
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Signed-off-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Signed-off-by: zwhzzz0821 <2831474076@qq.com>
Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Co-authored-by: zwhzzz0821 <2831474076@qq.com>
|
2026-08-31 10:15:46 +00:00 |
|
|
Wang fuyinandHongsheng Liu
|
72ee535fcd
|
[Diffusion] Add native paged KV backend for diffusion workers (#6102)
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-08-22 21:45:45 +08:00 |
|
|
|
e04210d690
|
[Hardware][MUSA] Enable Qwen3-Omni ModelOpt FP8 inference (#5671)
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
|
2026-08-11 17:19:09 +08:00 |
|
|
Zijun HuangandWeiQing Chen
|
93457f29ff
|
[NPU][Quantization] Add RainFusion attention and INT8 online quantization for MiniMax H3 (#5706)
Signed-off-by: Zijun Huang <zijunhua@usc.edu>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
|
2026-08-06 14:15:21 +08:00 |
|
|
zzhanddengyunyang
|
dba05c3517
|
Enable Diffusion Image models asynchronous output (#4981)
Signed-off-by: zzh <943967662@qq.com>
Co-authored-by: dengyunyang <584797741@qq.com>
|
2026-07-29 17:21:55 +00:00 |
|
|
Joshna-MedisettyandChendi.Xue
|
7fa40681ff
|
Xpu/vllm 0.26 fixes (#5476)
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
|
2026-07-29 08:15:36 +08:00 |
|
|
|
e108cbb510
|
[NPU][HunyuanImage3] Adapt diffusion FusedMoE for vLLM >= 0.24.0 (#5167)
Signed-off-by: betta18 <jiangmengyu1@huawei.com>
Signed-off-by: jiangmengyu18 <56633611+jiangmengyu18@users.noreply.github.com>
Co-authored-by: betta18 <jiangmengyu1@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-07-27 22:41:34 +08:00 |
|
|
|
5db89b41c4
|
[Core] hunyuan-image prefetch kv (#4448)
Signed-off-by: z00806815 <zhuhonghan@huawei.com>
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
Signed-off-by: dph97 <nenbaying@163.com>
Signed-off-by: zhh <zhh@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: zhuhh97 <zhuhh97@users.noreply.github.com>
|
2026-06-25 21:31:49 +08:00 |
|
|
Nick Cao
|
67d14b5ac4
|
[Refactor] Replace hardcoded CUDA device selection with platform-agnostic API (#4496)
Signed-off-by: Nick Cao <ncao@redhat.com>
|
2026-06-23 14:38:32 +08:00 |
|
|
Y. Fisher
|
64efa66869
|
[NPU] Initialize Ascend config for diffusion workers (#4386)
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
|
2026-06-23 03:12:44 +00:00 |
|
|
Samit
|
f44f3ef9f7
|
[platform] fix: set UnspecifiedOmniPlatform device_type to cpu (#4357)
Signed-off-by: samithuang <285365963@qq.com>
|
2026-06-12 08:40:00 +00:00 |
|
|
+1
|
f92d84fcf9
|
[Rebase] Rebase to vllm releases/v0.22.0 (#3891)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Signed-off-by: nainiu258 <cperfect02@163.com>
Signed-off-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Signed-off-by: JasonJ2021 <jasonj@zju.edu.cn>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: skf <54565339+skf-1999@users.noreply.github.com>
Co-authored-by: Zheng Wengang <zwg0606@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: nainiu258 <101917677+nainiu258@users.noreply.github.com>
Co-authored-by: Jason <72191212+JasonJ2021@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
|
2026-06-01 11:08:12 +08:00 |
|
|
Canlin Guo
|
412bf42a75
|
[Refactor] Unify _talker_mtp_forward across GPU and NPU model runners (#3476)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
|
2026-05-31 07:58:00 +08:00 |
|
|
汪志鹏
|
c7178d89bb
|
[Bugfix] UnspecifiedOmniPlatform.get_device_count returns 0 instead o… (#3636)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
|
2026-05-15 05:28:40 +00:00 |
|
|
|
ea4cf77f56
|
[Hardware] Extend diffusion engine plugin extensibility for out-of-tree hardware backends (#3239)
Signed-off-by: Yuchen Jiang <yucjiang@amazon.com>
Co-authored-by: Yuchen Jiang <yucjiang@amazon.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
|
2026-05-08 14:29:15 +08:00 |
|
|
R0CKSTARandCanlin Guo
|
52f17bf292
|
[Refactor] Extract has_flash_attn_pkg in OmniPlatform (#3068)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
|
2026-04-27 04:07:34 +00:00 |
|
|
jiangmengyu18andbetta18
|
c174c95ab0
|
[NPU] [Quant] Support HunyuanImage3 offline quantization with vLLM-Ascend on diffusion path (#2979)
Signed-off-by: betta18 <jiangmengyu1@huawei.com>
Signed-off-by: jiangmengyu18 <56633611+jiangmengyu18@users.noreply.github.com>
Co-authored-by: betta18 <jiangmengyu1@huawei.com>
|
2026-04-25 06:14:01 +00:00 |
|
|
TaffyOfficial
|
1ad726f495
|
[Fix] HunyuanImage-3.0: unify naming hunyuan_image_3 → hunyuan_image3 (#2712)
|
2026-04-14 21:53:47 -07:00 |
|
|
Y. FisherandCanlin Guo
|
ec082add35
|
[Perf] Wan2.2 I2V optimization: convert datatype from FP32 to BF16 in vae (#2391)
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
|
2026-04-08 03:26:39 +00:00 |
|
|
R0CKSTAR
|
7a72f34ce4
|
Add supports_float64() to OmniPlatform and clean up MPS (#2488)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
|
2026-04-07 11:15:02 +08:00 |
|
|
R0CKSTAR
|
3def008b32
|
[Feat] Add MUSA platform support for Moore Threads GPUs (#2337)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com>
|
2026-04-01 17:55:00 +08:00 |
|
|
|
7217557950
|
[Refactor] Unify torch profiler for omni and diffusion models (#2099)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: lishunyang lishunyang12@163.com
|
2026-03-24 15:25:52 +08:00 |
|
|
TJian
|
c85acb1773
|
[CI] [ROCm] Bugfix device environment issue (#1984)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2026-03-19 05:11:28 +08:00 |
|
|
|
28dd1a699d
|
[Model] Extend NPU support for HunyuanImage3 Diffusion Model (#1689)
Signed-off-by: ElleElleWu <1608928702@qq.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: skf1999 <13234016272@163.com>
Co-authored-by: Just-it <1161406585@qq.com>
Co-authored-by: Semmer2 <semmer@live.cn>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
|
2026-03-12 20:49:17 +08:00 |
|
|
Chendi.Xueandgcanlin
|
482f9b89a1
|
[XPU][NPU][ROCM] enable cpu_offloading flag for non_cuda (#1488)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
|
2026-02-27 14:52:15 +08:00 |
|
|
Canlin Guo
|
4eeea68cff
|
[Platform] Add supports_torch_inductor interface (#1108)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
|
2026-01-30 15:21:43 +00:00 |
|
|
Fanli LinandCopilot
|
bfbf3e5b69
|
[Feat] Support XPU Backend in vLLM-Omni (#191)
Signed-off-by: Fanli Lin <fanli.lin@intel.com>
Signed-off-by: Fanli Lin <fanli0116@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
2026-01-30 04:33:05 +13:00 |
|
|
Canlin Guo
|
c9392dff01
|
[Hardware] Support platforms and plugin system (#774)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
|
2026-01-28 22:12:52 +08:00 |
|