hsliuustc0106
c9bb65902e
[Model] Learn the retention budget directly from observed peaks
...
Drop the staged-bytes seed per review direction: the budget is now
min(observed_peak * 1.25, 25% of device capacity), nothing else. A cold
start pays one release per observation step while the peak climbs (two on
the validated MiniMax workload), then boundaries retain steadily; the 5%
free floor and capacity cap are unchanged. The staged-byte ledger remains
only to detect full detachment, which now resets the observed peak so one
workload cannot permanently influence another. Trade-off accepted per
review: lower reserved memory and a simpler policy over a flush-free
first request.
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-08-26 13:43:00 +08:00
hsliuustc0106
27f9726a6a
[Model] Derive the retention budget from observed boundary peaks
...
The staged-bytes seed left the transient half of the working set to a
guessed multiplier, and that half scales with the running workload rather
than the model topology. Track the largest cached-but-unallocated value
observed at any component boundary instead: the budget is the larger of
the seed and the observed peak times a 1.25 margin, still capped at a
quarter of device capacity. A workload that outgrows the seed pays one
release while the observation lands, then retains steadily. No warmup
flag is needed and no user-chosen fraction exists.
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-08-26 12:52:19 +08:00
hsliuustc0106
c5295f3ffe
[Model] Cap derived cache budget and account for stager detach
...
Address review on #6629 :
- Cap the derived retention budget at a quarter of device capacity so a
large staged total cannot monopolize a small device; the multiplier
itself stays pending an instrumented target run.
- Log every release decision (reason, cached bytes, effective budget and
cap, staged total, free/total memory) and every staging OOM retry with
its staged byte total, so a single target run validates the headroom
multiplier directly.
- Make stager detach symmetric: set_cache_retention now shrinks the
previous cache's budget before growing the new one, so cache -> None ->
same cache counts the staged bytes exactly once (regression test added).
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-08-26 11:12:09 +08:00
hsliuustc0106
ba5943dddb
[Model] Derive MiniMax-H3 DLO component cache budget from staged bytes
...
Replace BoundedAllocatorCache's device-capacity fraction bound (25% of
total memory) with a budget derived from the model topology: each attached
PinnedModuleStager reports its unique staged storage bytes once, and the
retention ceiling is that total times a fixed 2.0 headroom multiplier
covering allocator fragmentation and transient storages freed at component
boundaries. The 5% physically-free floor is unchanged.
The budget therefore scales with the staged components instead of device
capacity, so no user-chosen fraction is needed and behavior is identical
across differently sized devices with the same model. Re-attaching the
same cache does not double count; a second stager grows the budget by its
own bytes.
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-08-26 09:49:55 +08:00
Hongsheng Liu
780e8d40aa
[Doc] Add environment variables configuration reference ( #6217 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-26 04:58:54 +08:00
Alex Brooks
0eb47b76d0
[Bugfix] Handle DoS Overflow Cases ( #6598 )
...
Signed-off-by: Alex Brooks <albrooks@redhat.com >
2026-08-25 16:09:03 +00:00
NATURE
497c537c6f
[Benchmark] Add local OmniInteract realtime benchmark ( #6522 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-08-25 15:32:58 +00:00
Hongsheng Liu and hsliuustc0106
c608910d20
[Diffusion] Register HWR mmap for direct DLO H2D ( #6591 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
2026-08-25 23:30:11 +08:00
Hongsheng Liu
5a8b81ebeb
[Model] Optimize MiniMax-H3 DLO component lifecycle ( #6526 )
...
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-08-25 23:13:41 +08:00
jl9876 and Chendi.Xue
0d5d1115d6
[AutoRound] Add offline quantized MXFP4 model support ( #5544 )
...
Signed-off-by: jl9876 <jia.li@intel.com >
Co-authored-by: Chendi.Xue <chendi.xue@intel.com >
2026-08-25 14:44:38 +00:00
NumberWan
bc8ea489ef
[BugFix] Qwen-image performance regressed - Avoid mapping diffusion_batch_size onto scheduler max_num_seqs ( #6525 )
...
Signed-off-by: NumberWan <wantszkin2003@gmail.com >
2026-08-25 03:36:11 -07:00
Bo Li
51b7565f33
[perf] Add mask-free TRTLLM packed-padding path ( #6542 )
...
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com >
2026-08-25 07:45:21 +00:00
wy17003 and Hongsheng Liu
b454e34966
[Refactor][2/N Scheduler]Replace _FULL_PAYLOAD_INPUT_STAGES with a resolved stage transport capability. ( #6149 )
...
Signed-off-by: wy17003 <yaow17003@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-25 15:24:59 +08:00
Nguyen Kha Nhat Long and Samit
bae58f84b8
[minor, fix] Remove cuda sync on wake_up in AsyncOmni ( #4092 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-08-25 03:13:42 +00:00
f0808c9793
[Bugfix] Fix remote replica membership lifecycle races ( #5277 )
...
Signed-off-by: hbhflw2000 <417911774@qq.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: Samit <285365963@qq.com >
2026-08-25 10:34:59 +08:00
NATURE
702a409664
[Feature][MiniCPM-o] Support configurable concurrent duplex sessions ( #6021 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-08-25 02:27:30 +00:00
yzong-rh
710024fa6f
[Bugfix] Honor height/width overrides in LongCatImageEditPipeline ( #6222 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-08-25 02:09:00 +00:00
Pranav Thakur and Zhou Taichang
53b60bd9bc
[Bugfix] Invalidate stale ref_audio cache on local file modification ( #4873 ) ( #5670 )
...
Signed-off-by: pranavthakur0-0 <pranavthakurdew@gmail.com >
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk >
2026-08-25 10:06:32 +08:00
leo and leo
50b8a150f9
[Model] Add native SANA-Video 2B T2V and I2V support ( #5508 )
...
Signed-off-by: leo <92263211+napleon-liu@users.noreply.github.com >
Signed-off-by: leo <92263211+liuyao0322@users.noreply.github.com >
Signed-off-by: liuyao0322 <yaoliu548926@gmail.com >
Co-authored-by: leo <92263211+napleon-liu@users.noreply.github.com >
2026-08-25 09:46:02 +08:00
Hongsheng Liu and hsliuustc0106
8360346a66
[Diffusion] Integrate no-AllGather DLO with Host Weight Runtime ( #6486 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
Co-authored-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com >
2026-08-25 08:03:53 +08:00
d150a4fde7
[Feature][MiniMax-H3] Support diffusion continuous batching ( #5810 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: Claude Opus 5 <noreply@anthropic.com >
Co-authored-by: lishunyang12 <lishunyang12@163.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-08-24 21:08:53 +08:00
Mu GuanLin and Hongsheng Liu
aa69acbd09
[Bugfix] Prevent LTX-2.5 from silently loading unindexed Diffusers shards ( #6234 )
...
Signed-off-by: mglyn <1203789601@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-24 20:57:24 +08:00
Nguyen Kha Nhat Long and Cursor
d15b7ca3e1
[Feature]: Support pause / resume and sleep / wake for AR stages in AsyncOmni ( #6084 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-08-24 16:21:32 +08:00
bjf-frz and Hongsheng Liu
072bfc02dd
[FEAT]Add FastVideo VSA backend for Wan2.2 ( #4820 )
...
Signed-off-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-24 14:48:31 +08:00
Mu GuanLin and Hongsheng Liu
3d035bfa19
[Feature] Support MiniMax-H3 Turbo LoRA with the legacy manager ( #6476 )
...
Signed-off-by: mglyn <1203789601@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-24 06:21:42 +08:00
SYLAR
1c917b2751
[Bugfix] Fix Qwen3-Omni AWQ quantization name mapping ( #5687 )
...
Signed-off-by: lishunyang12 <lishunyang12@163.com >
2026-08-23 15:13:36 +00:00
wangyu
1e2a6446f7
[CI] Remove AudioX and MammothModa2 support ( #6353 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-08-23 04:22:05 +00:00
akshatvishu
39c16d75a9
[tts][bugfix] Fix CosyVoice3 sampling and stage handoff ( #6424 )
...
Signed-off-by: akshatvishu <akshatnayak197@gmail.com >
2026-08-22 21:20:57 -07:00
wangyu
2e096788a6
[CI]Remove SoulX-Singer support ( #6362 )
...
Signed-off-by: wangyu <410167048@qq.com >
2026-08-23 07:16:32 +08:00
Zhou Taichang
3ad64c6a12
[Bugfix][Qwen3-Omni] Stabilize thinker MRoPE compilation ( #6449 )
...
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-08-22 19:42:56 +00:00
hurukawa and Hongsheng Liu
855719e331
[Bugfix] Release GPU memory after diffusion execution failures ( #6385 )
...
Signed-off-by: nagisa <1434936049@qq.com >
Signed-off-by: nagisa-kun <1434936049@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 18:32:48 +00:00
maithilijoshi20 and Hongsheng Liu
bd4f9acfd3
[Bugfix] Preserve pipeline sampling constraints ( #6182 )
...
Signed-off-by: maithilijoshi20 <97733343+maithilijoshi20@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 14:57:09 +00:00
ivanusto and Hongsheng Liu
f09bdcbc7d
[Bugfix] Make the async-output wait bound configurable and default it higher ( #6255 )
...
Signed-off-by: ivanusto <ivanusto@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 22:10:30 +08:00
b39f599b09
[Core] Add UniProc diffusion executor for single-GPU ( #6308 )
...
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com >
Signed-off-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster >
2026-08-22 21:58:21 +08:00
Wang fuyin and Hongsheng Liu
72ee535fcd
[Diffusion] Add native paged KV backend for diffusion workers ( #6102 )
...
Signed-off-by: Acerak01-fy <wfy2003324@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 21:45:45 +08:00
Hongsheng Liu
c2daa471c3
fix: keep DLO hooks outside regional compilation ( #6073 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 21:44:03 +08:00
Hongsheng Liu
493fc64397
[Diffusion] Add final-layout BF16 Host Weight Runtime artifacts ( #6445 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 21:37:20 +08:00
boatman and Hongsheng Liu
3781214675
[Refactor] Move model-specific capability metadata into TTS adapters ( #6138 )
...
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 14:26:49 +08:00
amy-why-3459
42059f19d5
[Bugfix][MiniCPM-o] Cap offline Talker generation at remaining context ( #6458 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-22 12:42:49 +08:00
Zhang Bowen and Hongsheng Liu
4882a36c41
[Model][Frontend] MiniMax-H3: Add opt-in planar video response encoding ( #6288 )
...
Signed-off-by: MosCloud <bwzhang1991@163.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-22 04:04:14 +00:00
yzong-rh
3d34b8b977
[Bugfix] Fix LongCat TeaCache CFG negative-branch guidance kwargs ( #6181 )
...
Signed-off-by: Yifan Zong <yzong@redhat.com >
2026-08-22 01:22:26 +00:00
Nick Cao
cfbf1392c5
[Bugfix] Drop librosa reintroduced by Step-Audio2 ( #6467 )
...
Signed-off-by: Nick Cao <ncao@redhat.com >
2026-08-21 21:33:07 +00:00
汪志鹏
dc72559d5d
[BugFix]: CosyVoice3 STFT window device mismatch ( #6454 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-08-21 19:45:46 +08:00
R0CKSTAR
260db8bab5
[Bugfix][Diffusion] Fix MiniMax-H3 model-level CPU offload residency ( #6072 )
...
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com >
2026-08-21 10:30:53 +00:00
NATURE and amy-why-3459
4f7c08c0eb
[Bugfix][MiniCPM-o] Fix async-chunk snapshot replacement and prompt cleanup ( #6406 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-08-21 16:37:21 +08:00
Hongsheng Liu
15e38ac173
[Core] Add explicit post-load Host Weight Runtime publication ( #6427 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-21 14:26:49 +08:00
Shin
70bdd49402
[Model] Add HiDream-O1-Image support ( #5194 )
...
Signed-off-by: yixiaoer <shin@yixiaoer.sg >
2026-08-21 14:15:37 +08:00
Hongsheng Liu
73b623f2f7
[Core] Add Host Weight Runtime foundation ( #6419 )
...
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-08-21 08:50:59 +08:00
Ian Eaves
5730af0a95
[Bugfix] Preserve prompt token usage details ( #5181 )
...
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com >
2026-08-20 20:05:05 +00:00
37c11e6e4f
[Perf][Qwen3-TTS] Fuse QKV and gate_up projections in code predictor ( #5791 )
...
Signed-off-by: sheenlin <sheenlin@tencent.com >
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: sheenlin <sheenlin@tencent.com >
Co-authored-by: yurain <yurain26@users.noreply.github.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-08-20 19:22:40 +00:00