tzhouam
31d127edcd
[Refactor] Remove unused MRotaryEmbedding and vision position ID functions
...
- Deleted the MRotaryEmbedding class and its associated methods from mrope.py to streamline the codebase.
- Removed the get_llm_pos_ids_for_vision function from vision.py as it is no longer needed.
- Updated import statements in qwen2_5_omni.py to reflect the removal of the vision module.
Signed-off-by: [Your Name] <your.email@example.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-01-28 15:42:35 +08:00
tzhouam
52d0122fb5
[Enhancement] Refactor OmniInputProcessor and related classes for improved prompt handling
...
- Updated OmniInputProcessor to streamline input processing, including enhanced handling of prompt embeddings and additional information.
- Introduced OmniTextPrompt and OmniEmbedsInputs to support new input structures with optional metadata.
- Modified input preprocessing methods to accommodate new prompt types and ensure compatibility with multimodal inputs.
- Improved error handling and type checking for data parallel configurations.
Signed-off-by: [Your Name] <your.email@example.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-01-28 15:42:05 +08:00
tzhouam
b26a9735e6
[Enhancement] Update type hint for stage_connector_spec in OmniEngineArgs and AsyncOmniEngineArgs
...
- Changed type hint for stage_connector_spec from 'any' to 'Any' for improved type checking and consistency.
- Ensured compatibility with typing standards in the arg_utils module.
Signed-off-by: [Your Name] <your.email@example.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-01-28 15:41:20 +08:00
tzhouam
d7cfe195ea
[Enhancement] Refactor OmniARScheduler and OmniGenerationScheduler to include OmniSchedulerOutput
...
- Updated OmniARScheduler to wrap output in OmniSchedulerOutput, including transfer metadata for finished requests needing KV cache transfer.
- Enhanced OmniGenerationScheduler to handle routed experts and finish reasons more effectively, ensuring proper output structure.
- Introduced performance statistics handling in both schedulers for improved monitoring.
Signed-off-by: [Your Name] <your.email@example.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-01-28 15:40:59 +08:00
tzhouam
e1c6639f18
[Enhancement] Add mm_encoder_only parameter and deprecation warning for convert option
...
- Introduced mm_encoder_only parameter to OmniModelConfig.
- Added a deprecation warning for the `--convert mm_encoder_only` option, advising users to switch to `--mm-encoder-only`.
Signed-off-by: [Your Name] <your.email@example.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-01-28 15:40:29 +08:00
tzhouam
ce7430b562
[Enhancement] Update OmniRequest to support torch.Tensor for prompt_embeds and add decoding method
...
- Modified the constructor to accept torch.Tensor as an option for prompt_embeds.
- Introduced a new static method _maybe_decode_prompt_embeds to handle the conversion from PromptEmbedsPayload to torch.Tensor.
- Preserved the original prompt_embeds payload for reference.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-01-28 15:40:17 +08:00
Jiangyun Zhu and Hongsheng Liu
1c8fcf436f
[Perf] replace torch's conv with vLLM's conv to fix torch 2.9 performance regression ( #982 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-28 10:41:33 +08:00
dongbo910220
e8aa32b82b
[Docs] Fix GLM-Image docstring indentation to resolve CI failure ( #992 )
...
Signed-off-by: dongbo910220 <1275604947@qq.com >
2026-01-28 12:55:17 +13:00
Yueqian Lin
9bca8880fe
Fix TTS speaker typo and add supported languages ( #990 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-01-27 23:19:51 +08:00
amy-why-3459
d2b3a24af3
[BugFix] Modify the method of obtaining external_request_id ( #961 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
2026-01-27 20:56:02 +08:00
Yuanheng Zhao
2f5e32477c
[NFC] Remove redundant torch.no_grad in models and pipelines ( #854 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
2026-01-27 12:36:54 +00:00
yangsonglin13
16be03b909
[Feature] Implement YuanrongConnector based on OmniConnectorBase ( #716 )
...
Signed-off-by: yangsonglin13 <yangsonglin566@gmail.com >
2026-01-27 20:17:02 +08:00
TJian
46e96e18e3
[ROCm] [Perf] Add AITER Flash Attention ( #941 )
...
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com >
2026-01-27 20:14:49 +08:00
Canlin Guo
cadd77262d
[NPU] Upgrade to v0.14.0 ( #820 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
2026-01-27 16:45:32 +08:00
Jiangyun Zhu
50b44303f1
[diffusion] add tp for FLUX.2-klein ( #973 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-01-27 08:37:25 +00:00
Yueqian Lin
77ff875170
Support Qwen3 tts online serving ( #968 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-01-27 14:37:21 +08:00
Jiangyun Zhu
4a3a1b6de8
[Perf] avoid cpu op in QwenImageCrossAttention ( #942 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-01-27 12:15:35 +08:00
XU Mingshi and Hongsheng Liu
1fe8e71433
[Diffusion][Feature] Non-Intrusive Sequence Parallelism (SP) Support for Wan2.2 ( #966 )
...
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-27 11:33:38 +08:00
5037af1033
[Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment ( #758 )
...
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk >
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk >
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: dongbo910220 <1275604947@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-01-27 10:22:40 +08:00
Samit and Hongsheng Liu
978a290d6e
[Bugfix] Fix diffusion pipeline CFG (guidance_scale parsing failure bug) ( #922 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 05:09:03 +00:00
52bf1e2f40
[Feature]Support async computation and communication across stages by chunks ( #727 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 11:21:24 +08:00
Jiangyun Zhu and Hongsheng Liu
578ed1967e
[Bugfix] raise error in diffusion engine and fix offload test ( #933 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-25 23:04:11 +08:00
汪志鹏
06c9d9b463
[Bagel] Support TeaCache ( #848 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
2026-01-25 14:52:27 +08:00
YinHanke and Hongsheng Liu
f9c69a8c40
[BugFix] Standardize StableAudio audio output ( #842 )
...
Signed-off-by: LudovicoYIN <hankeyin@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-24 22:27:58 +13:00
21ee1ff6e6
[bugfix] qwen3-tts check_model_inputs ( #924 )
...
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com >
Co-authored-by: shijin zhang <75300765+Dovis01@users.noreply.github.com >
Co-authored-by: ning ding <nndding@gmail.com >
2026-01-23 09:37:53 +00:00
5d9918fa67
[Misc] Fix error log for the diffusion stage timeout ( #915 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-01-23 15:16:40 +08:00
Roger Wang
b8499e4839
Bump version to 0.14.0rc1 ( #910 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
2026-01-22 17:09:48 -08:00
Zhou Taichang
e3b6254090
debug Qwen TTS ( #902 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
2026-01-22 15:28:56 +00:00
汪志鹏 and wzliu
7f821be57b
Support Bagel Model ( #726 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: wzliu <wzliu@connect.hku.hk >
2026-01-22 23:09:55 +13:00
Gao Han
a54c323952
[Model] Support Qwen3-TTS model series ( #895 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2026-01-22 16:57:37 +08:00
Jiangyun Zhu
0df8e80574
[diffusion] use fa3 by default when device supports it ( #783 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-01-22 15:00:29 +08:00
NATURE
c9d7cd189f
[Core]Add Diffusion executor ( #865 )
...
Signed-off-by: wzliu <wzliu@connect.hku.hk >
2026-01-22 05:32:44 +13:00
Jiangyun Zhu
58efc5dde3
[Misc] fix qwen image family redundant computation ( #868 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-01-21 09:58:08 +00:00
Sihao Li
0aa72b952f
[bugfix] support text + audio mixed output ( #843 )
...
Signed-off-by: GG-li <3226868735@qq.com >
2026-01-21 17:44:17 +08:00
John Liu BUAA
c3894585b5
[Perf] Fuse Q/K/V Linear with QKVParallelLinear in Qwen2.5Omni DiTAttention ( #884 )
...
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
2026-01-21 09:24:05 +00:00
Canlin Guo and Hongsheng Liu
b564132e44
[Diffusion][Acceleration] Support TeaCache for Z-Image ( #817 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-21 16:53:18 +08:00
Chenguang Zheng
72a8553310
[Bugfix] Remove the duplicate api registration in vllm-omni ( #880 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
2026-01-21 07:00:31 +00:00
XU Mingshi
25b08b3371
[Diffusion] Non-Intrusive Sequence Parallelism (SP) Model Support Abstraction for vLLM-Omni Framework ( #779 )
...
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
2026-01-21 14:43:09 +08:00
+13
776c3a794e
Dev/rebase 0.14.0 ( #847 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: root <root@hk01dgx039.cm.cluster >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: TangPeng <85704592@qq.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
Signed-off-by: root <root@hk01dgx028.cm.cluster >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: root <root@hk01dgx039.cm.cluster >
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com >
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com >
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-21 00:25:25 +08:00
Zeyu Huang | 黃澤宇
430c60b9b5
[Bugfix] Diffusion model fails to load when stage config is present ( #860 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-01-20 12:42:26 +08:00
SYLAR
a1c37cc6e3
[Core] Implement Diffusion Profiler Support ( #709 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-01-20 11:35:48 +08:00
Jiangyun Zhu
3fc4f988ea
[diffusion] add tp support for qwen-image and refactor some tests ( #830 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2026-01-19 14:20:31 +08:00
D!NE$H
5e7035e183
[Feature]: Add CFG param to online serving ( #824 )
...
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
2026-01-19 04:23:15 +00:00
汪志鹏
36c28767cd
[Core]Add GPU Diffusion Runner ( #822 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-01-17 19:49:13 +08:00
John Liu BUAA
bb24e07a7a
[Perf] Optimize the Qwen2.5-Omni Model thinker-to-talker-proj with nn.Linear ( #825 )
...
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
2026-01-17 19:47:11 +08:00
汪志鹏
0888520875
[Bagel] Support Cache-Dit ( #736 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-01-17 15:21:49 +08:00
Yueqian Lin and Hongsheng Liu
3fb6adc40a
[bugfix] use unipc scheduler for Wan 2.2 ( #804 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-16 12:31:25 +00:00
WeiQing Chen
a7f9926013
[Model] add flux2 klein ( #809 )
...
Signed-off-by: David Chen <530634352@qq.com >
2026-01-16 16:32:28 +08:00
Didan Deng
43ec7f701b
[Feature] Flash Attention to Support Attention Mask ( #760 )
...
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
2026-01-16 11:01:11 +08:00
Peiqi Yin
35f994e255
[BugFix] Qwen2.5-omni supress end token and won't stop ( #773 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
2026-01-16 00:41:59 +08:00