lishunyang
69bcc6416d
[New Model] Lance (ByteDance) — unified AR+diffusion multimodal ( #3697 )
...
Adds Lance (bytedance-research/Lance), a 3B unified AR+diffusion
multimodal model on a Qwen2.5-VL backbone. Lance is BAGEL-lineage:
it reuses vLLM-Omni's BAGEL Mixture-of-Transformers core verbatim and
specializes only the ViT (Qwen2.5-VL vision), the VAE (Wan2.2) and the
checkpoint layout.
All six single-stage Lance modalities verified end-to-end on 1x B300:
t2i, t2v, image_edit, video_edit, x2t_image (image understanding) and
x2t_video (video understanding). 3-D mRoPE is wired through
BagelRotaryEmbedding for the x2t / video / edit paths. Includes the
upstream gap-analysis quality fixes (task-specific system prompts, full
Qwen chat template, upstream inference defaults and aspect-ratio
buckets).
Ships an offline example (examples/offline_inference/lance/), an
OpenAI-compatible online-serving example
(examples/online_serving/lance/) and an e2e test
(tests/e2e/online_serving/test_lance.py).
See LANCE_PR.md for the architecture rationale, per-modality status and
the two remaining follow-ups (two-stage AR+DiT, video_edit polish).
Closes #3697 .
Signed-off-by: lishunyang <lishunyang03@gmail.com >
2026-05-18 22:09:14 +00:00
Zeyu Huang | 黃澤宇 and Copilot Autofix powered by AI
d18168ccbb
[bugfix] Fix diffusers backend input bug after #2913 ( #3644 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-15 09:55:09 +00:00
lyj-jjj and Cursor
bbc00f9f86
[BugFix] fix(omni): isolate diffusion KV-cache dtype from vLLM --kv-cache-dtype #3585 ( #3596 )
...
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-05-15 11:22:59 +08:00
Y. Fisher
4d3eed152a
[Feat][Config] Support additional_config for diffusion worker ( #3020 )
...
Signed-off-by: KexiongYu <yukexiong1@huawei.com >
Signed-off-by: Y. Fisher <yukexiong1@huawei.com >
2026-05-13 14:22:43 +00:00
56ca7dd612
support online FP8 quantization for FA on NPU #2236 ( #2640 )
...
Signed-off-by: lyj-jjj <liuyingjun5@huawei.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
2026-05-13 11:46:00 +08:00
汪志鹏
c9a8556c24
[New Model]: Add sensenova u1 support ( #3319 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-05-11 22:19:00 +08:00
Canlin Guo
5e263b6929
[BugFix] Rename attention_config to diffusion_attention_config ( #3489 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-05-11 11:27:35 +08:00
Canlin Guo and Hongsheng Liu
11c4c7f0ff
[Diffusion][Attention] Support per-role attention backend via CLI ( #2681 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-10 11:59:05 +08:00
baonudesifeizhai and roG0d
c4a0990044
（Phase 1）Add ModelOpt FP8 auto-detect support for diffusion checkpoints #2709 ( #2913 )
...
Signed-off-by: roG0d <rodgarcas98@gmail.com >
Signed-off-by: roG0d <baonudesifeizhai@gmail.com >
Signed-off-by: baonudesifeizhai <85092850+baonudesifeizhai@users.noreply.github.com >
Co-authored-by: roG0d <rodgarcas98@gmail.com >
2026-05-09 16:55:05 +08:00
Chendi.Xue and Hongsheng Liu
4e41b7bc23
Enable Wan2.2-S2V modeling to vLLM-omni ( #2751 )
...
Signed-off-by: Chendi Xue <chendi.xue@intel.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-05-06 08:03:48 +08:00
Zeyu Huang | 黃澤宇 and Copilot Autofix powered by AI
f19891e4f1
[bugfix][CI] Diffusers backend update ( #3096 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
Signed-off-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com >
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com >
2026-05-05 20:54:39 +08:00
NATURE
bfb730ed50
[Bugfix]pass TP size to diffusion config ( #2867 )
...
Signed-off-by: natureofnature <wzliu@connect.hku.hk >
2026-04-30 11:56:51 +08:00
jiangmengyu18 and betta18
c174c95ab0
[NPU] [Quant] Support HunyuanImage3 offline quantization with vLLM-Ascend on diffusion path ( #2979 )
...
Signed-off-by: betta18 <jiangmengyu1@huawei.com >
Signed-off-by: jiangmengyu18 <56633611+jiangmengyu18@users.noreply.github.com >
Co-authored-by: betta18 <jiangmengyu1@huawei.com >
2026-04-25 06:14:01 +00:00
Lancer and Hongsheng Liu
345504b68f
[Feat] support quantization for GLM-IMAGE ( #2292 )
...
Signed-off-by: Lancer <maruixiang6688@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-24 13:49:49 +08:00
Zeyu Huang | 黃澤宇
d8cc7a0409
[feat]: General diffusers adapter backend to run diffusion models ( #2724 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-04-23 00:07:18 +08:00
Vensen
e076378df1
[Feat][sleepmode] add omni sleepmode and ack protocol ( #2022 )
2026-04-21 08:19:53 +08:00
WeiQing Chen and Gao Han
f658bcb0bf
[Bugfix] Limit Qwen-Image-Edit-2511 input image count ( #2840 )
...
Signed-off-by: david6666666 <530634352@qq.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
2026-04-17 20:07:30 +08:00
Samit
61a3cbdff5
[Perf] Eliminate Hop 3 IPC overhead for single-stage diffusion via inline execution ( #2736 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
2026-04-15 11:46:06 +08:00
cb6a8739c1
[Feature] Add trajectory recording to BAGEL denoising loop ( #2483 )
...
Signed-off-by: Zhengyuan Su <su.zhengyuan@u.nus.edu >
Signed-off-by: Zhengyuan Su (苏政渊) <su.zhengyuan@u.nus.edu >
Co-authored-by: Claude <noreply@anthropic.com >
Co-authored-by: Samit <285365963@qq.com >
2026-04-08 10:43:01 +08:00
Yi Liu and Hongsheng Liu
c1d2dcc3ac
[AutoRound] Add offline quantized W4A16 model support ( #1777 )
...
Signed-off-by: yiliu30 <yi4.liu@intel.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-04-02 13:57:53 +08:00
Wu JIAZHEN and jader
3fd4a4dc27
[Feat] Support step-boundary abort in diffusion ( #1769 )
...
Signed-off-by: jader <yjader@foxmail.com >
Signed-off-by: asukaqaq-s <1311722138@qq.com >
Co-authored-by: jader <yjader@foxmail.com >
2026-04-01 17:25:02 +08:00
SYLAR
6b93459826
[Core] Unified quantization framework ( #1764 )
2026-03-24 18:56:55 +08:00
7217557950
[Refactor] Unify torch profiler for omni and diffusion models ( #2099 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: lishunyang lishunyang12@163.com
2026-03-24 15:25:52 +08:00
dongbo910220 and Didan Deng
50bd1961bf
[Feature]: add Ulysses advanced_uaa mode ( #1379 )
...
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-03-24 14:55:54 +08:00
Samit and Didan Deng
9ead0d890d
[Enhancement] Add force_refresh support for GLM-Image for cache-dit 1.3.0 upgrade ( #1858 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-03-24 14:32:18 +08:00
Didan Deng
1f5eca5686
[Perf] Qwen-Image Performance Nightly CI test ( #1805 )
...
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-03-24 00:19:20 +08:00
Wu JIAZHEN
da7a8f8014
reafator pipeline stage/step pipeline ( #1368 )
...
Signed-off-by: asukaqaq-s <1311722138@qq.com >
2026-03-20 19:21:50 +08:00
fde4808723
[Performance] diffusion timing ( #1757 )
...
Signed-off-by: dengyunyang <584797741@qq.com >
Signed-off-by: bjf-frz <frz123db@gmail.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: bjf-frz <frz123db@gmail.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-03-19 12:26:12 +08:00
5377619c0b
fix: propagate parallel_config through create_default_diffusion ( #1878 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-03-14 17:07:13 +08:00
dengyunyang
3e8cb6c631
[Diffusion] [Model] Dreamid omni ( #1855 )
...
Signed-off-by: dengyunyang <584797741@qq.com >
2026-03-13 21:53:06 +08:00
28dd1a699d
[Model] Extend NPU support for HunyuanImage3 Diffusion Model ( #1689 )
...
Signed-off-by: ElleElleWu <1608928702@qq.com >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: skf1999 <13234016272@163.com >
Co-authored-by: Just-it <1161406585@qq.com >
Co-authored-by: Semmer2 <semmer@live.cn >
Co-authored-by: gcanlin <canlinguosdu@gmail.com >
2026-03-12 20:49:17 +08:00
Ting FU
6ac85d4607
[Feat] Enable expert parallel for diffusion MoE layers ( #1323 )
...
Signed-off-by: Semmer2 <semmer@live.cn >
2026-03-11 10:45:36 +08:00
Canlin Guo
29357027b0
[Bugfix] Allow to enable HSDP alone ( #1567 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-03-09 06:36:13 +00:00
knlnguyen1802 and Samit
92e1a54419
[RL] Support collective rpc api to entrypoint && Support custom input output ( #1646 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-03-09 10:12:43 +08:00
Ding Zuhao and 汪志鹏
128e6f4b97
[Feature][Bagel] Add CFG parallel mode ( #1578 )
...
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-03-06 09:55:54 +08:00
Samit
22a51a7cdb
[Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading ( #1504 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
2026-03-02 14:53:25 +00:00
汪志鹏 and Cursor
e37a89ffcb
[Feature]: Support cfg kv-cache transfer in multi-stage ( #1422 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: Cursor <cursoragent@cursor.com >
2026-03-02 05:06:28 +00:00
0b884ebc6c
[Feature]: Native GGUF Quantization Support for DiT ( #1285 )
...
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Isotr0py <2037008807@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-27 11:39:14 +08:00
Canlin Guo and Hongsheng Liu
f53152d340
[Feature] Support HSDP for diffusion models ( #1339 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-02-26 21:31:10 +08:00
knlnguyen1802
9994594782
[Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker ( #764 )
...
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com >
2026-02-12 19:23:51 +08:00
Yuanheng Zhao and Samit
8bf849c399
Refactor CPU Offloading Backend Pattern ( #1223 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-02-12 17:29:55 +08:00
SYLAR
830d261f9d
[Feature]: FP8 Quantization Support for DiT ( #1034 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-02-11 21:01:35 +08:00
dongbo910220 and hsliuustc0106
43e8747c00
[Feat]: support VAE patch parallelism ( #756 )
...
Signed-off-by: dongbo910220 <1275604947@qq.com >
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com >
2026-02-10 17:36:10 +08:00
Didan Deng and Hongsheng Liu
23cf48d7bb
[Perf]: CFG parallel abstraction ( #851 )
...
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-30 16:23:44 +00:00
2923d5a91e
[Feature] Support DiT Layerwise (Blockwise) CPU Offloading ( #858 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
2026-01-30 17:35:09 +08:00
Samit
c0beebbfe3
[Feature] Support cache-dit for Wan 2.2 inference ( #1021 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
2026-01-28 20:07:50 +08:00
5037af1033
[Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment ( #758 )
...
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk >
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk >
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: dongbo910220 <1275604947@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-01-27 10:22:40 +08:00
汪志鹏 and wzliu
7f821be57b
Support Bagel Model ( #726 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: wzliu <wzliu@connect.hku.hk >
2026-01-22 23:09:55 +13:00
Prajwal A and zjy0516
779f5987d9
[feature] cpu offloading support for diffusion ( #497 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Prajwal A <prajwalanagani@gmail.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-01-12 10:41:18 +00:00
Jiangyun Zhu
c784b3e07d
[core] add torch compile for diffusion ( #684 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-01-09 04:05:44 +13:00
lishunyang