anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,706 Commits 235 Branches 31 Tags
c9bb65902e4e575abde837fd7941bc9415ebb32b
Commit Graph
785 Commits
Author SHA1 Message Date
hsliuustc0106 c9bb65902e [Model] Learn the retention budget directly from observed peaks 
Drop the staged-bytes seed per review direction: the budget is now
min(observed_peak * 1.25, 25% of device capacity), nothing else. A cold
start pays one release per observation step while the peak climbs (two on
the validated MiniMax workload), then boundaries retain steadily; the 5%
free floor and capacity cap are unchanged. The staged-byte ledger remains
only to detect full detachment, which now resets the observed peak so one
workload cannot permanently influence another. Trade-off accepted per
review: lower reserved memory and a simpler policy over a flush-free
first request.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 13:43:00 +08:00
hsliuustc0106 27f9726a6a [Model] Derive the retention budget from observed boundary peaks 
The staged-bytes seed left the transient half of the working set to a
guessed multiplier, and that half scales with the running workload rather
than the model topology. Track the largest cached-but-unallocated value
observed at any component boundary instead: the budget is the larger of
the seed and the observed peak times a 1.25 margin, still capped at a
quarter of device capacity. A workload that outgrows the seed pays one
release while the observation lands, then retains steadily. No warmup
flag is needed and no user-chosen fraction exists.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 12:52:19 +08:00
hsliuustc0106 c5295f3ffe [Model] Cap derived cache budget and account for stager detach 
Address review on #6629:

- Cap the derived retention budget at a quarter of device capacity so a
  large staged total cannot monopolize a small device; the multiplier
  itself stays pending an instrumented target run.
- Log every release decision (reason, cached bytes, effective budget and
  cap, staged total, free/total memory) and every staging OOM retry with
  its staged byte total, so a single target run validates the headroom
  multiplier directly.
- Make stager detach symmetric: set_cache_retention now shrinks the
  previous cache's budget before growing the new one, so cache -> None ->
  same cache counts the staged bytes exactly once (regression test added).

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 11:12:09 +08:00
hsliuustc0106 ba5943dddb [Model] Derive MiniMax-H3 DLO component cache budget from staged bytes 
Replace BoundedAllocatorCache's device-capacity fraction bound (25% of
total memory) with a budget derived from the model topology: each attached
PinnedModuleStager reports its unique staged storage bytes once, and the
retention ceiling is that total times a fixed 2.0 headroom multiplier
covering allocator fragmentation and transient storages freed at component
boundaries. The 5% physically-free floor is unchanged.

The budget therefore scales with the staged components instead of device
capacity, so no user-chosen fraction is needed and behavior is identical
across differently sized devices with the same model. Re-attaching the
same cache does not double count; a second stager grows the budget by its
own bytes.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 09:49:55 +08:00
Hongsheng Liuandhsliuustc0106 c608910d20 [Diffusion] Register HWR mmap for direct DLO H2D (#6591) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 23:30:11 +08:00
Hongsheng Liu 5a8b81ebeb [Model] Optimize MiniMax-H3 DLO component lifecycle (#6526) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-25 23:13:41 +08:00
Bo Li 51b7565f33 [perf] Add mask-free TRTLLM packed-padding path (#6542) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-08-25 07:45:21 +00:00
yzong-rh 710024fa6f [Bugfix] Honor height/width overrides in LongCatImageEditPipeline (#6222) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-08-25 02:09:00 +00:00
leoandleo 50b8a150f9 [Model] Add native SANA-Video 2B T2V and I2V support (#5508) 
Signed-off-by: leo <92263211+napleon-liu@users.noreply.github.com>
Signed-off-by: leo <92263211+liuyao0322@users.noreply.github.com>
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com>
Co-authored-by: leo <92263211+napleon-liu@users.noreply.github.com>
 2026-08-25 09:46:02 +08:00
Hongsheng Liuandhsliuustc0106 8360346a66 [Diffusion] Integrate no-AllGather DLO with Host Weight Runtime (#6486) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 08:03:53 +08:00
汪志鹏Claude Opus 5lishunyang12Cursor
 		d150a4fde7 [Feature][MiniMax-H3] Support diffusion continuous batching (#5810) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-24 21:08:53 +08:00
Mu GuanLinandHongsheng Liu aa69acbd09 [Bugfix] Prevent LTX-2.5 from silently loading unindexed Diffusers shards (#6234) 
Signed-off-by: mglyn <1203789601@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 20:57:24 +08:00
bjf-frzandHongsheng Liu 072bfc02dd [FEAT]Add FastVideo VSA backend for Wan2.2 (#4820) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 14:48:31 +08:00
Mu GuanLinandHongsheng Liu 3d035bfa19 [Feature] Support MiniMax-H3 Turbo LoRA with the legacy manager (#6476) 
Signed-off-by: mglyn <1203789601@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 06:21:42 +08:00
wangyu 1e2a6446f7 [CI] Remove AudioX and MammothModa2 support (#6353) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-08-23 04:22:05 +00:00
wangyu 2e096788a6 [CI]Remove SoulX-Singer support (#6362) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-08-23 07:16:32 +08:00
hurukawaandHongsheng Liu 855719e331 [Bugfix] Release GPU memory after diffusion execution failures (#6385) 
Signed-off-by: nagisa <1434936049@qq.com>
Signed-off-by: nagisa-kun <1434936049@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 18:32:48 +00:00
ivanustoandHongsheng Liu f09bdcbc7d [Bugfix] Make the async-output wait bound configurable and default it higher (#6255) 
Signed-off-by: ivanusto <ivanusto@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 22:10:30 +08:00
Rahul SteigerHongsheng LiuRahul Steiger
 		b39f599b09 [Core] Add UniProc diffusion executor for single-GPU (#6308) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Signed-off-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-08-22 21:58:21 +08:00
Wang fuyinandHongsheng Liu 72ee535fcd [Diffusion] Add native paged KV backend for diffusion workers (#6102) 
Signed-off-by: Acerak01-fy <wfy2003324@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 21:45:45 +08:00
Hongsheng Liu c2daa471c3 fix: keep DLO hooks outside regional compilation (#6073) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 21:44:03 +08:00
Hongsheng Liu 493fc64397 [Diffusion] Add final-layout BF16 Host Weight Runtime artifacts (#6445) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 21:37:20 +08:00
Zhang BowenandHongsheng Liu 4882a36c41 [Model][Frontend] MiniMax-H3: Add opt-in planar video response encoding (#6288) 
Signed-off-by: MosCloud <bwzhang1991@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 04:04:14 +00:00
yzong-rh 3d34b8b977 [Bugfix] Fix LongCat TeaCache CFG negative-branch guidance kwargs (#6181) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-08-22 01:22:26 +00:00
R0CKSTAR 260db8bab5 [Bugfix][Diffusion] Fix MiniMax-H3 model-level CPU offload residency (#6072) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-08-21 10:30:53 +00:00
Shin 70bdd49402 [Model] Add HiDream-O1-Image support (#5194) 
Signed-off-by: yixiaoer <shin@yixiaoer.sg>
 2026-08-21 14:15:37 +08:00
MaciejBalaNVandRoger Wang 13c2de684c Cosmos3 logging and prompt improvements (#6325) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2026-08-20 12:04:49 -07:00
MaciejBalaNV b3270dec8d Cosmos3 transfer fix (#5614) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
 2026-08-20 11:40:51 -07:00
dengyunyang 8d34d3b5de [Perf][Hunyuan] Optimize for vae and patch_embed (#6306) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-08-20 16:29:21 +00:00
wangyu 273ffce130 [CI/Build] Strengthen pre-commit with markdownlint, SPDX, and policy hooks (#6273) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-08-20 21:07:04 +08:00
WeiQing Chen 53df9fe62b [Bugfix] Use dedicated WORLD group for distributed VAE communication (#6401) 
Signed-off-by: david6666666 <530634352@qq.com>
 2026-08-20 16:13:34 +08:00
R0CKSTAR b58feb1420 [MUSA] Fix SwiGLU to use fused op (#6364) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-08-20 14:19:14 +08:00
linzhenpl07andClaude Opus 5 38cf9adc38 [BugFix][Diffusion] Fix MiniMax-H3 VAE hang when decoder tiles are fewer than ranks (#6345) 
Signed-off-by: linzhenpl07 <linzhenpl07@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-08-20 12:42:14 +08:00
Joshna-MedisettyCursorChendi.Xue
 		ea24deefcc [Bugfix][FLUX.2-klein] Fix duplicated image position ids (#6130) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Signed-off-by: Joshna-Medisetty <joshna.medisetty@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
 2026-08-19 16:30:12 -05:00
BruceLoveDecimaland汪志鹏 adf6d0c1ea [Model] Add SANA-WM support(Stage-1 only) (#4061) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-08-18 23:57:55 -06:00
bjf-frzandHongsheng Liu 14bd8f8772 [REFACT]Refactor diffusion parallel state (#5531) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-19 11:23:10 +08:00
Canlin Guo 14f2c16fb2 [Perf] Fuse MiniMax H3 modulation with FP32 accumulation (#6281) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-08-18 20:13:54 -07:00
Canlin Guo 9683b7f4b4 [Perf] Fuse MiniMax H3 SwiGLU activation (#6283) 
Signed-off-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-08-18 20:12:10 -07:00
Alex Brooks b4b907e7b6 [Bugfix | Model] Fix SenseNova & Use Well-defined Model Configs (#5877) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-08-18 16:43:05 +00:00
Weiming Liao f208f907bc [BugFix][NPU] Wan2.2-S2V RoPE: replace complex64 advanced indexing with index_select (#6320) 
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
 2026-08-19 00:06:29 +08:00
Hongsheng Liu 284e05c88b [Core] Support online FP8 with DLO AllGather (#6279) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-18 23:48:07 +08:00
Hongsheng Liu c3de04e127 [Bugfix] Avoid eager pi0 runtime import in pipeline registry (#6322) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-18 23:42:03 +08:00
Joshna-MedisettyHongsheng LiuCursorNick Cao
 		4cd667875f [XPU][SDXL] Fix text encoder input device under CPU offload (#6125) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
 2026-08-18 14:03:31 +00:00
TJian 02069a5bbf [ROCm] [CI] Fix circular import for rocm due to patch (#6287) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-08-18 09:45:05 +00:00
Yukim1andzwhzzz0821 0a1846aa45 [Diffusion] Add native KV cache initialization and Scheduler-managed block allocation (#6094) 
Signed-off-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
Co-authored-by: zwhzzz0821 <zwhzzz0821@users.noreply.github.com>
 2026-08-18 07:00:24 +00:00
TJian 170d498771 [ROCm] [CI] Fix cpu test on rocm (#6267) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-08-17 14:46:30 +08:00
Yichen WangandDidan Deng 2a9f6aaa8e Add π0 VLA model support (#4222) 
Signed-off-by: Yichen Wang <yichen.wang@bytedance.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-08-17 14:29:17 +08:00
Hongsheng Liu 9d2bb23ff6 [Diffusion] Add loader-owned host-weight plans for DLO (TP=1) (#6213) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 11:02:38 +08:00
Jack Chenzhangbin.327Hongsheng Liu
 		3875f8ded4 [Core] Add realtime AR-Diffusion tick sessions for LingBot World 2.0 (#5491) 
Signed-off-by: zhangbin.327 <zhangbin.327@bytedance.com>
Signed-off-by: Jack47 <scsvip@gmail.com>
Co-authored-by: zhangbin.327 <zhangbin.327@bytedance.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-16 09:11:20 +08:00
Anurag 68bf8c2c42 [Bugfix] Prevent DiffusionResultPump crash on cancelled futures (#5793) (#5983) 
Signed-off-by: ANURAG KANADE <anuragkanade6@gmail.com>
 2026-08-15 16:19:45 +00:00