anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,706 Commits 235 Branches 31 Tags
c9bb65902e4e575abde837fd7941bc9415ebb32b
Commit Graph
2706 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
hsliuustc0106 c9bb65902e [Model] Learn the retention budget directly from observed peaks 
Drop the staged-bytes seed per review direction: the budget is now
min(observed_peak * 1.25, 25% of device capacity), nothing else. A cold
start pays one release per observation step while the peak climbs (two on
the validated MiniMax workload), then boundaries retain steadily; the 5%
free floor and capacity cap are unchanged. The staged-byte ledger remains
only to detect full detachment, which now resets the observed peak so one
workload cannot permanently influence another. Trade-off accepted per
review: lower reserved memory and a simpler policy over a flush-free
first request.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 13:43:00 +08:00
hsliuustc0106 b78db6036a [Model] Fix adaptation test fixture so the capacity cap does not bind 
The fixture reported device capacity 100, making the 25% cap 25 — below
the learned budget of 37 — so the post-adaptation checks released again
and the test failed. Raise the fixture capacity to 400 so the cap stays
above the learned budget and the test isolates peak adaptation; verified
against the real suite on the L20X validation host (15 passed).

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 13:12:55 +08:00
hsliuustc0106 27f9726a6a [Model] Derive the retention budget from observed boundary peaks 
The staged-bytes seed left the transient half of the working set to a
guessed multiplier, and that half scales with the running workload rather
than the model topology. Track the largest cached-but-unallocated value
observed at any component boundary instead: the budget is the larger of
the seed and the observed peak times a 1.25 margin, still capped at a
quarter of device capacity. A workload that outgrows the seed pays one
release while the observation lands, then retains steadily. No warmup
flag is needed and no user-chosen fraction exists.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 12:52:19 +08:00
hsliuustc0106 c5295f3ffe [Model] Cap derived cache budget and account for stager detach 
Address review on #6629:

- Cap the derived retention budget at a quarter of device capacity so a
  large staged total cannot monopolize a small device; the multiplier
  itself stays pending an instrumented target run.
- Log every release decision (reason, cached bytes, effective budget and
  cap, staged total, free/total memory) and every staging OOM retry with
  its staged byte total, so a single target run validates the headroom
  multiplier directly.
- Make stager detach symmetric: set_cache_retention now shrinks the
  previous cache's budget before growing the new one, so cache -> None ->
  same cache counts the staged bytes exactly once (regression test added).

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 11:12:09 +08:00
hsliuustc0106 ba5943dddb [Model] Derive MiniMax-H3 DLO component cache budget from staged bytes 
Replace BoundedAllocatorCache's device-capacity fraction bound (25% of
total memory) with a budget derived from the model topology: each attached
PinnedModuleStager reports its unique staged storage bytes once, and the
retention ceiling is that total times a fixed 2.0 headroom multiplier
covering allocator fragmentation and transient storages freed at component
boundaries. The 5% physically-free floor is unchanged.

The budget therefore scales with the staged components instead of device
capacity, so no user-chosen fraction is needed and behavior is identical
across differently sized devices with the same model. Re-attaching the
same cache does not double count; a second stager grows the budget by its
own bytes.

Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-26 09:49:55 +08:00
Hongsheng Liu 780e8d40aa [Doc] Add environment variables configuration reference (#6217) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-26 04:58:54 +08:00
Satellite 6639c82a17 [CI/Build] Pin GGUF plugin for diffusion nightly tests (#6303) 
Signed-off-by: Xin Wei <14196318+wxwxwwxxx@users.noreply.github.com>
 2026-08-25 09:16:46 -07:00
Alex Brooks 0eb47b76d0 [Bugfix] Handle DoS Overflow Cases (#6598) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-08-25 16:09:03 +00:00
NATURE 497c537c6f [Benchmark] Add local OmniInteract realtime benchmark (#6522) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-25 15:32:58 +00:00
Hongsheng Liuandhsliuustc0106 c608910d20 [Diffusion] Register HWR mmap for direct DLO H2D (#6591) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 23:30:11 +08:00
Hongsheng Liu 5a8b81ebeb [Model] Optimize MiniMax-H3 DLO component lifecycle (#6526) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-08-25 23:13:41 +08:00
NumberWan dd527b4245 [CI] Reduce Qwen-Image Function and share step-execution perf server (#6613) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-08-25 22:55:15 +08:00
jl9876andChendi.Xue 0d5d1115d6 [AutoRound] Add offline quantized MXFP4 model support (#5544) 
Signed-off-by: jl9876 <jia.li@intel.com>
Co-authored-by: Chendi.Xue <chendi.xue@intel.com>
 2026-08-25 14:44:38 +00:00
psv666 683f3c92a4 [CI/Build] Clean up Qwen3-Omni nightly tests (#6570) 
Signed-off-by: psv666 <2693925048@qq.com>
 2026-08-25 22:14:43 +08:00
Jae 72beeaee86 [Misc] move text-to-audio online examples to a unified folder (#4807) 
Signed-off-by: zzehli <jaeli_ottawa@outlook.com>
 2026-08-25 22:02:26 +08:00
Bhavyansh Sharma d6b66faba7 [CI] Fix response_format json_schema error expectation (#6290) 
Signed-off-by: Bhavyansh Sharma <66562679+clumsylad21@users.noreply.github.com>
 2026-08-25 19:47:09 +08:00
NumberWan bc8ea489ef [BugFix] Qwen-image performance regressed - Avoid mapping diffusion_batch_size onto scheduler max_num_seqs (#6525) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-08-25 03:36:11 -07:00
zhumingjue138andGao Han 29e3165bb9 [Test] add marker for stability and add Wan2.2 npu test case (#6343) 
Signed-off-by: zhumingjue <zhumingjue@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-08-25 17:08:58 +08:00
Bo Li 51b7565f33 [perf] Add mask-free TRTLLM packed-padding path (#6542) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-08-25 07:45:21 +00:00
wy17003andHongsheng Liu b454e34966 [Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. (#6149) 
Signed-off-by: wy17003 <yaow17003@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-25 15:24:59 +08:00
Nguyen Kha Nhat LongandSamit bae58f84b8 [minor, fix] Remove cuda sync on wake_up in AsyncOmni (#4092) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-25 03:13:42 +00:00
WeiGao HanSamit
 		f0808c9793 [Bugfix] Fix remote replica membership lifecycle races (#5277) 
Signed-off-by: hbhflw2000 <417911774@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Samit <285365963@qq.com>
 2026-08-25 10:34:59 +08:00
NATURE 702a409664 [Feature][MiniCPM-o] Support configurable concurrent duplex sessions (#6021) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-08-25 02:27:30 +00:00
yzong-rh 710024fa6f [Bugfix] Honor height/width overrides in LongCatImageEditPipeline (#6222) 
Signed-off-by: Yifan Zong <yzong@redhat.com>
 2026-08-25 02:09:00 +00:00
Pranav ThakurandZhou Taichang 53b60bd9bc [Bugfix] Invalidate stale ref_audio cache on local file modification (#4873) (#5670) 
Signed-off-by: pranavthakur0-0 <pranavthakurdew@gmail.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-08-25 10:06:32 +08:00
leoandleo 50b8a150f9 [Model] Add native SANA-Video 2B T2V and I2V support (#5508) 
Signed-off-by: leo <92263211+napleon-liu@users.noreply.github.com>
Signed-off-by: leo <92263211+liuyao0322@users.noreply.github.com>
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com>
Co-authored-by: leo <92263211+napleon-liu@users.noreply.github.com>
 2026-08-25 09:46:02 +08:00
wangyuandamy-why-3459 4af0ddc1c2 [Test] Cleanup for helpers (#6523) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-25 09:10:17 +08:00
Hongsheng Liuandhsliuustc0106 8360346a66 [Diffusion] Integrate no-AllGather DLO with Host Weight Runtime (#6486) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
 2026-08-25 08:03:53 +08:00
Hongsheng Liu 8fdb719a41 [CI] Resume AR admission after sleep-mode wake (#6581) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-25 07:18:05 +08:00
Hongsheng Liu 678aa25864 Revert "Revert "[CI]Fix Wan DMD pipeline test alignment"" (#6578) 2026-08-24 23:40:01 +08:00
Gao Han a94b2580a1 Revert "[CI]Fix Wan DMD pipeline test alignment" (#6574) 2026-08-24 22:26:55 +08:00
汪志鹏Claude Opus 5lishunyang12Cursor
 		d150a4fde7 [Feature][MiniMax-H3] Support diffusion continuous batching (#5810) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: lishunyang12 <lishunyang12@163.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-24 21:08:53 +08:00
Mu GuanLinandHongsheng Liu aa69acbd09 [Bugfix] Prevent LTX-2.5 from silently loading unindexed Diffusers shards (#6234) 
Signed-off-by: mglyn <1203789601@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 20:57:24 +08:00
bjf-frz 0e4b28f491 [CI]Fix Wan DMD pipeline test alignment (#6557) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
 2026-08-24 17:43:24 +08:00
WeiQing Chen f446468c03 [Doc] Update vLLM-Omni WeChat QR code (#6535) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-08-24 16:46:02 +08:00
Nguyen Kha Nhat LongandCursor d15b7ca3e1 [Feature]: Support pause / resume and sleep / wake for AR stages in AsyncOmni (#6084) 
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-08-24 16:21:32 +08:00
bjf-frzandHongsheng Liu 072bfc02dd [FEAT]Add FastVideo VSA backend for Wan2.2 (#4820) 
Signed-off-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 14:48:31 +08:00
Mu GuanLinandHongsheng Liu 3d035bfa19 [Feature] Support MiniMax-H3 Turbo LoRA with the legacy manager (#6476) 
Signed-off-by: mglyn <1203789601@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-24 06:21:42 +08:00
SYLAR 1c917b2751 [Bugfix] Fix Qwen3-Omni AWQ quantization name mapping (#5687) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-08-23 15:13:36 +00:00
Jerry Song 17285c2f55 [Feature] Add FP8 quantization for FLUX.2-dev (#3027) 
Signed-off-by: Songrui625 <songrui625@gmail.com>
 2026-08-23 04:31:34 +00:00
wangyu 1e2a6446f7 [CI] Remove AudioX and MammothModa2 support (#6353) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-08-23 04:22:05 +00:00
akshatvishu 39c16d75a9 [tts][bugfix] Fix CosyVoice3 sampling and stage handoff (#6424) 
Signed-off-by: akshatvishu <akshatnayak197@gmail.com>
 2026-08-22 21:20:57 -07:00
wangyuandamy-why-3459 0db262de2a [CI/Build] Move L1/E2E coverage to weekly and split scheduled L4/L5 pipelines (#6311) 
Signed-off-by: wangyu <410167048@qq.com>
Signed-off-by: [Your Name] <your.email@example.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-08-23 11:49:37 +08:00
wangyu 2e096788a6 [CI]Remove SoulX-Singer support (#6362) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-08-23 07:16:32 +08:00
Nick Cao b2c9376539 [BugFix] Eliminate distributed test port race by switching to file:// rendezvous (#6468) 
Signed-off-by: Nick Cao <ncao@redhat.com>
 2026-08-23 07:08:44 +08:00
Zhou Taichang 3ad64c6a12 [Bugfix][Qwen3-Omni] Stabilize thinker MRoPE compilation (#6449) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-08-22 19:42:56 +00:00
hurukawaandHongsheng Liu 855719e331 [Bugfix] Release GPU memory after diffusion execution failures (#6385) 
Signed-off-by: nagisa <1434936049@qq.com>
Signed-off-by: nagisa-kun <1434936049@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 18:32:48 +00:00
maithilijoshi20andHongsheng Liu bd4f9acfd3 [Bugfix] Preserve pipeline sampling constraints (#6182) 
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 14:57:09 +00:00
ivanustoandHongsheng Liu f09bdcbc7d [Bugfix] Make the async-output wait bound configurable and default it higher (#6255) 
Signed-off-by: ivanusto <ivanusto@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-08-22 22:10:30 +08:00
Rahul SteigerHongsheng LiuRahul Steiger
 		b39f599b09 [Core] Add UniProc diffusion executor for single-GPU (#6308) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Signed-off-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-08-22 21:58:21 +08:00