anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,628 Commits 235 Branches 31 Tags
codex/dlo-runtime-cache-phase-b
Commit Graph
76 Commits
Author SHA1 Message Date
Hongsheng Liu 6bcd350301 [Diffusion] Prefer host-cache registration when pinned 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 18:13:44 +08:00
Hongsheng Liu 859003fb97 [Diffusion] Gate CUDA read-only host registration 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:36 +08:00
Hongsheng Liu 42c6ae9007 [Diffusion] Document TP host layout identity 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:36 +08:00
Hongsheng Liu 16642dc2e8 [Diffusion] Make DLO host weight cache opt-in 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:36 +08:00
Hongsheng Liu 010b3c1d74 [Diffusion] Make no-AllGather host weight cache automatic 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:36 +08:00
Hongsheng Liu 3bb7547a0f [Diffusion] Rename runtime cache to host weight cache 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:36 +08:00
Hongsheng Liu ee665fcfc3 [Docs] Add AllGather DLO architecture diagram 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:16 +08:00
Hongsheng Liu 649558e1a1 [Docs] Show TP2 devices in DLO architecture 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:16 +08:00
Hongsheng Liu 47153d86e6 [Docs] Visualize no-AllGather shared host weights 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:16 +08:00
Hongsheng Liu 1ccdc2e80e [Docs] Separate DLO transfer mode design 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:16 +08:00
Hongsheng Liu 8f93a74951 [Docs] Remove stale DLO validation results 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:22:52 +08:00
Hongsheng Liu 4acb3b1387 [Docs] Add DLO component architecture 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:21:59 +08:00
Hongsheng Liu 6751c1f404 [Diffusion] Address runtime cache review feedback 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:21:59 +08:00
Hongsheng Liu 23ab14a15b [Diffusion] Register runtime cache for direct DLO H2D 
Avoid recurrent mmap-to-pinned packing in no-AllGather DLO by registering transform-complete runtime-cache mappings under an explicit per-worker budget. Copy mapped tensors directly into the rotating device buffers while preserving bounded-staging fallback and ordered shutdown cleanup.

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:21:28 +08:00
Hongsheng Liu 67f15e73cd [Diffusion] Document runtime cache hardware tradeoffs 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:21:28 +08:00
Hongsheng Liu 59549968cf [Diffusion] Tighten runtime cache compatibility guards 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:21:28 +08:00
Hongsheng Liu d30ccdc315 [Diffusion] Add node-local runtime weight cache for DLO 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:21:01 +08:00
Hongsheng Liu 284e05c88b [Core] Support online FP8 with DLO AllGather (#6279) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-18 23:48:07 +08:00
Hongsheng Liu 9d2bb23ff6 [Diffusion] Add loader-owned host-weight plans for DLO (TP=1) (#6213) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 11:02:38 +08:00
Jack Chenzhangbin.327Hongsheng Liu
 		3875f8ded4 [Core] Add realtime AR-Diffusion tick sessions for LingBot World 2.0 (#5491) 
Signed-off-by: zhangbin.327 <zhangbin.327@bytedance.com>
Signed-off-by: Jack47 <scsvip@gmail.com>
Co-authored-by: zhangbin.327 <zhangbin.327@bytedance.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 09:11:20 +08:00
SYLARandHongsheng Liu d1e230c95b [Diffusion][Quantization] Enable MiniMax-H3 global FP8 with DLO (#5910) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-15 23:01:26 +08:00
Samit 136cb27b05 [Bugfix] Fix lost async diffusion outputs in request-level batching (#6023) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-08-15 18:27:51 +08:00
Hongsheng Liu 1260ceaae6 [Docs] Split diffusion attention and CPU offload guides (#6075) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-15 14:41:09 +08:00
Hongsheng Liu 62bf8d630d docs: document distributed layerwise offload compatibility (#5839) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-06 23:21:43 +08:00
bjf-frz 9f978923f2 [docs] consolidate diffusion execution modes (#5599) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-08-05 22:41:18 +08:00
Chenguang Zheng d5d47e9be1 [Doc] Document async Omni output materialization (#5610) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-08-04 14:50:38 +08:00
zzhanddengyunyang dba05c3517 Enable Diffusion Image models asynchronous output (#4981) 
Signed-off-by: zzh <943967662@qq.com>
Co-authored-by: dengyunyang <584797741@qq.com>
 2026-07-29 17:21:55 +00:00
wangyu 7c17111ff5 [Refactor][CI] Reorganize test tree, and move slow e2e to weekly (#4951) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-07-29 23:53:03 +08:00
SYLARandlishunyang12 7762d664d8 [Attention] Add trtllm diffusion attention backend with Skip-Softmax (#5283) 
Signed-off-by: lishunyang12 <lishunyang03@gmail.com>
Co-authored-by: lishunyang12 <lishunyang03@gmail.com>
 2026-07-27 22:48:11 +08:00
Yang_wdrs 8927ec057c [Refactor][1/N]diffusion/cache: cache-dit (#5226) 
Signed-off-by: yangjianjuan <66548561+yangjianjuan@users.noreply.github.com>
 2026-07-24 22:42:45 +08:00
Diya Peng acf941da4d [Diffusion] Add VAE patch parallel support for FLUX.2-dev (#5292) 
Signed-off-by: dpeng123 <diyapeng123@gmail.com>
 2026-07-24 07:00:57 +00:00
bjf-frzandHongsheng Liu be667a8067 [Refact] Refactor diffusion output streaming (#5166) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-23 10:06:08 +08:00
Zeyu Huang | 黃澤宇Copilot Autofix powered by AICursor
 		e39f4dddfb [Refactor]: clean up diffusion scheduler interfaces (#5216) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-07-22 14:05:49 +08:00
Bingyu (Spencer) Liu dcf8b07d39 [Core] Cleanup Omni connector runtime and transfer backends (#5096) 
Signed-off-by: spencerr221 <liubingyu62@gmail.com>
 2026-07-22 09:44:15 +08:00
Zeyu Huang | 黃澤宇andHongsheng Liu 1b0de21c55 [Chore]: refactor out unused/redundant params in diffusion pipelines (#1235) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-07-02 16:32:05 +02:00
JiangJie ZhangandSamit c45ac74d43 [Core][Frontend] Support request-level batching for diffusion pipelines (#4079) 
Signed-off-by: jader <yjader@foxmail.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-06-29 18:09:19 +02:00
Rahul SteigerandHongsheng Liu 327e9dcaf2 [Feature] Spatially-sharded (SP) decode for the Wan VAE (#4620) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-06-27 09:01:25 +08:00
inkcherryzejwangjunkang1991Zejian WangZejian WangCursor
 		e1b0b83238 add moriio transfer engine (#1742) 
Signed-off-by: inkcherry <mingzhi.liu@amd.com>
Signed-off-by: zejwang  <zejwang@amd.com>
Signed-off-by: junkang1991 <junkangchow@gmail.com>
Signed-off-by: Zejian Wang <zejianwang@sjtu.edu.cn>
Co-authored-by: zejwang <zejwang@amd.com>
Co-authored-by: junkang1991 <junkangchow@gmail.com>
Co-authored-by: Zejian Wang <zejianwang@sjtu.edu.cn>
Co-authored-by: Zejian Wang <Zejian.Wang@amd.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-06-10 08:37:33 +08:00
Samit 083b5e363f [Diffusion] Support LoRA in step-wise execution (#3639) 
Signed-off-by: samithuang <285365963@qq.com>
 2026-05-20 22:26:18 +08:00
yangsonglin13 1c530d984d Yuanrong TransferEngine Connector for NPU (#3180) 
Signed-off-by: yangsonglin13 <yangsonglin566@gmail.com>
 2026-05-20 14:51:33 +08:00
Rustam Khadipash 89f8819525 [Feature] Add support for Pipeline Parallel and integrate it into Wan 2.2 (#2322) 
Signed-off-by: Rustam Khadipash <16683750+hadipash@users.noreply.github.com>
 2026-05-19 09:42:17 +08:00
Ting FUasukaqaq-sjaderHongsheng LiuGao Han
 		e49fbd8a2d [Feat] DiffusionEngine Support async batch infer (#2729) 
Signed-off-by: Semmer <semmer@live.cn>
Signed-off-by: jader <yjader@foxmail.com>
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: jader <yjader@foxmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-05-06 09:47:13 +12:00
rein yang d435fe0707 [Docs] update documents (#2921) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-04-20 12:55:04 +08:00
Alex Brooks f346f2fdc6 Add Teacache Support for LongCat Image (#1487) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-04-18 07:52:35 +08:00
Alex Brooks f1e3f03726 [feature] Hidden State Prefix Caching (#2164) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-04-15 17:01:45 +00:00
TaffyOfficial 1ad726f495 [Fix] HunyuanImage-3.0: unify naming hunyuan_image_3 → hunyuan_image3 (#2712) 2026-04-14 21:53:47 -07:00
amy-why-3459 2a1d5060ab [skip ci][doc]Update async_chunk design diagram (#2420) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-13 20:43:40 +08:00
zhou zhuoxin 16041ab550 [Refactor] Extend CFG Parallel to support 3 or 4 branch dispatch across M GPUs (#2423) 2026-04-12 22:06:49 +08:00
Jinhengahengljhnatureofnature
 		c3c736d488 [Feature] Port Bagel RDMA flow to latest main (#2000) 
Signed-off-by: Jinheng Li <ahengljh@gmail.com>
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
Co-authored-by: ahengljh <ahengljh@users.noreply.github.com>
Co-authored-by: natureofnature <wzliu@connect.hku.hk>
 2026-04-08 14:55:35 +08:00
skfandCanlin Guo 6433847249 [skipCI][Docs] Add expert_parallel.md (#2471) 
Signed-off-by: skf1999 <13234016272@163.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-04-08 02:35:14 +00:00