anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
514 Commits 235 Branches 31 Tags
31d127edcdcf38e3a69d0dabf03397347406d8d2
Commit Graph
514 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam 31d127edcd [Refactor] Remove unused MRotaryEmbedding and vision position ID functions 
- Deleted the MRotaryEmbedding class and its associated methods from mrope.py to streamline the codebase.
- Removed the get_llm_pos_ids_for_vision function from vision.py as it is no longer needed.
- Updated import statements in qwen2_5_omni.py to reflect the removal of the vision module.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-28 15:42:35 +08:00
tzhouam 52d0122fb5 [Enhancement] Refactor OmniInputProcessor and related classes for improved prompt handling 
- Updated OmniInputProcessor to streamline input processing, including enhanced handling of prompt embeddings and additional information.
- Introduced OmniTextPrompt and OmniEmbedsInputs to support new input structures with optional metadata.
- Modified input preprocessing methods to accommodate new prompt types and ensure compatibility with multimodal inputs.
- Improved error handling and type checking for data parallel configurations.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-28 15:42:05 +08:00
tzhouam b26a9735e6 [Enhancement] Update type hint for stage_connector_spec in OmniEngineArgs and AsyncOmniEngineArgs 
- Changed type hint for stage_connector_spec from 'any' to 'Any' for improved type checking and consistency.
- Ensured compatibility with typing standards in the arg_utils module.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-28 15:41:20 +08:00
tzhouam d7cfe195ea [Enhancement] Refactor OmniARScheduler and OmniGenerationScheduler to include OmniSchedulerOutput 
- Updated OmniARScheduler to wrap output in OmniSchedulerOutput, including transfer metadata for finished requests needing KV cache transfer.
- Enhanced OmniGenerationScheduler to handle routed experts and finish reasons more effectively, ensuring proper output structure.
- Introduced performance statistics handling in both schedulers for improved monitoring.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-28 15:40:59 +08:00
tzhouam e1c6639f18 [Enhancement] Add mm_encoder_only parameter and deprecation warning for convert option 
- Introduced mm_encoder_only parameter to OmniModelConfig.
- Added a deprecation warning for the `--convert mm_encoder_only` option, advising users to switch to `--mm-encoder-only`.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-28 15:40:29 +08:00
tzhouam ce7430b562 [Enhancement] Update OmniRequest to support torch.Tensor for prompt_embeds and add decoding method 
- Modified the constructor to accept torch.Tensor as an option for prompt_embeds.
- Introduced a new static method _maybe_decode_prompt_embeds to handle the conversion from PromptEmbedsPayload to torch.Tensor.
- Preserved the original prompt_embeds payload for reference.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-28 15:40:17 +08:00
tzhouam ed8b03224b update the docker setup to vllm nightly 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-28 11:15:54 +08:00
Jiangyun ZhuandHongsheng Liu 1c8fcf436f [Perf] replace torch's conv with vLLM's conv to fix torch 2.9 performance regression (#982) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-28 10:41:33 +08:00
dongbo910220 e8aa32b82b [Docs] Fix GLM-Image docstring indentation to resolve CI failure (#992) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
 2026-01-28 12:55:17 +13:00
dongbo910220andHongsheng Liu 40cbd8ef7f [Fix] make images LoRA e2e less flaky (#978) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-27 23:55:05 +08:00
Yueqian Lin 9bca8880fe Fix TTS speaker typo and add supported languages (#990) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-01-27 23:19:51 +08:00
amy-why-3459 d2b3a24af3 [BugFix] Modify the method of obtaining external_request_id (#961) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-01-27 20:56:02 +08:00
Yuanheng Zhao 2f5e32477c [NFC] Remove redundant torch.no_grad in models and pipelines (#854) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-01-27 12:36:54 +00:00
yangsonglin13 16be03b909 [Feature] Implement YuanrongConnector based on OmniConnectorBase (#716) 
Signed-off-by: yangsonglin13 <yangsonglin566@gmail.com>
 2026-01-27 20:17:02 +08:00
wangyuandwangyu31577 2cc4375df8 [Test] Add chunk e2e test case for CI (#956) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
 2026-01-27 20:15:56 +08:00
TJian 46e96e18e3 [ROCm] [Perf] Add AITER Flash Attention (#941) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-27 20:14:49 +08:00
Canlin Guo cadd77262d [NPU] Upgrade to v0.14.0 (#820) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-27 16:45:32 +08:00
Andy Zhou 8efd912d32 [test] revert test flash attn file (#972) 
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
 2026-01-27 08:39:07 +00:00
Jiangyun Zhu 50b44303f1 [diffusion] add tp for FLUX.2-klein (#973) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-27 08:37:25 +00:00
Andy Zhou d12d3d711b [test] fix test_image_generation_lora CI timeout (#975) 
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
 2026-01-27 15:51:51 +08:00
Yueqian Lin 77ff875170 Support Qwen3 tts online serving (#968) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-01-27 14:37:21 +08:00
Jiangyun Zhu 4a3a1b6de8 [Perf] avoid cpu op in QwenImageCrossAttention (#942) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-27 12:15:35 +08:00
Jiangyun ZhuandHongsheng Liu 17e79336aa [doc] add some additional information for the diffusion model support (#952) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-27 12:14:19 +08:00
XU MingshiandHongsheng Liu 1fe8e71433 [Diffusion][Feature] Non-Intrusive Sequence Parallelism (SP) Support for Wan2.2 (#966) 
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-27 11:33:38 +08:00
Andy Zhoudongbo910220Samit
 		5037af1033 [Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment (#758) 
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk>
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-01-27 10:22:40 +08:00
zhou zhuoxin 4f3fd4de37 [doc]Add Text-To-Audio Readme documentation (#958) 
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
 2026-01-26 09:23:48 +00:00
SamitandHongsheng Liu 978a290d6e [Bugfix] Fix diffusion pipeline CFG (guidance_scale parsing failure bug) (#922) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-26 05:09:03 +00:00
amy-why-3459Rein YangCHENHongsheng Liu
 		52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-26 11:21:24 +08:00
Jiangyun ZhuandHongsheng Liu 578ed1967e [Bugfix] raise error in diffusion engine and fix offload test (#933) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-25 23:04:11 +08:00
Ding Zuhao 7f0825832a [Doc] Add Bagel model support to TeaCache documentation (#943) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
 2026-01-25 23:42:02 +13:00
汪志鹏 06c9d9b463 [Bagel] Support TeaCache (#848) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-01-25 14:52:27 +08:00
YinHankeandHongsheng Liu f9c69a8c40 [BugFix] Standardize StableAudio audio output (#842) 
Signed-off-by: LudovicoYIN <hankeyin@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-24 22:27:58 +13:00
WeiQing Chen 3d9cd16df7 [examples] add --enable-cpu-offload args (#930) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-01-23 22:43:00 +08:00
Jiangyun Zhu 28a6597c4c [Misc] Fix t2i online serving example (#928) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-23 19:32:06 +08:00
Baoyuan Qishijin zhangning ding
 		21ee1ff6e6 [bugfix] qwen3-tts check_model_inputs (#924) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
Co-authored-by: shijin zhang <75300765+Dovis01@users.noreply.github.com>
Co-authored-by: ning ding <nndding@gmail.com>
 2026-01-23 09:37:53 +00:00
SamitHongsheng LiuJiangyun Zhu
 		5d9918fa67 [Misc] Fix error log for the diffusion stage timeout (#915) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-01-23 15:16:40 +08:00
wangyuwangyu31577Hongsheng Liu
 		34c9d8f2a2 [Test] Add precision test cases for Qwen3-Omni-30B-A3B-Instruct in CI (#828) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-23 17:49:34 +13:00
WeiQing Chen 8f4f091387 [Misc] update wechat image (#914) 
Signed-off-by: David Chen <530634352@qq.com>
 2026-01-23 17:14:48 +13:00
TJian 28d13e0dee [Doc] Fix the version in the documentation to v0.14.0rc1; fix dockerfile.rocm entrypoint (#913) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-23 03:37:54 +00:00
Roger Wang b8499e4839 Bump version to 0.14.0rc1 (#910) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-01-22 17:09:48 -08:00
TJian 75dddcb2b5 [ROCm] [Doc] Add instructions to install ROCm dependencies to run Qwen3 TTS model (#905) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-23 10:55:50 +13:00
Zhou Taichang e3b6254090 debug Qwen TTS (#902) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-01-22 15:28:56 +00:00
汪志鹏andwzliu 7f821be57b Support Bagel Model (#726) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
 2026-01-22 23:09:55 +13:00
Gao Han a54c323952 [Model] Support Qwen3-TTS model series (#895) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2026-01-22 16:57:37 +08:00
Jiangyun Zhu 0df8e80574 [diffusion] use fa3 by default when device supports it (#783) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-22 15:00:29 +08:00
Samit f266b04534 [Bugfix] Fix video saving bug under certain fps (#893) 
Signed-off-by: SamitHuang <285365963@qq.com>
 2026-01-22 18:25:47 +13:00
NATURE c9d7cd189f [Core]Add Diffusion executor (#865) 
Signed-off-by: wzliu <wzliu@connect.hku.hk>
 2026-01-22 05:32:44 +13:00
TJian 4869311f2f [ROCm] [CI] Optimize Dockerfilerocm and reduce build time on CI (#885) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-01-22 03:39:45 +13:00
Jiangyun Zhu 58efc5dde3 [Misc] fix qwen image family redundant computation (#868) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-21 09:58:08 +00:00
Sihao Li 0aa72b952f [bugfix] support text + audio mixed output (#843) 
Signed-off-by: GG-li <3226868735@qq.com>
 2026-01-21 17:44:17 +08:00