anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
568 Commits 235 Branches 31 Tags
2b009b3cefcdcdebd55cd3260150d83d89c05f21
Commit Graph
35 Commits
Author SHA1 Message Date
tzhouam d7cfe195ea [Enhancement] Refactor OmniARScheduler and OmniGenerationScheduler to include OmniSchedulerOutput 
- Updated OmniARScheduler to wrap output in OmniSchedulerOutput, including transfer metadata for finished requests needing KV cache transfer.
- Enhanced OmniGenerationScheduler to handle routed experts and finish reasons more effectively, ensuring proper output structure.
- Introduced performance statistics handling in both schedulers for improved monitoring.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-01-28 15:40:59 +08:00
amy-why-3459 d2b3a24af3 [BugFix] Modify the method of obtaining external_request_id (#961) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-01-27 20:56:02 +08:00
amy-why-3459Rein YangCHENHongsheng Liu
 		52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-26 11:21:24 +08:00
汪志鹏andwzliu 7f821be57b Support Bagel Model (#726) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
 2026-01-22 23:09:55 +13:00
Gao Han a54c323952 [Model] Support Qwen3-TTS model series (#895) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2026-01-22 16:57:37 +08:00
+13
Zhou TaichangJaredforRealrootJustQJXU MingshiSihyeon JangJiangyun ZhucatcatZiming HuangYuhan LiuAliciaSamitYueqian LinWeiQing ChenPeiqi YinDidan DengHongsheng Liuwangyuwangyu31577汪志鹏John Liu BUAAD!NE$Hroot
 		776c3a794e Dev/rebase 0.14.0 (#847) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
 2026-01-21 00:25:25 +08:00
Zhou TaichangRein YangIsotr0py
 		8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:42:02 +08:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-16 15:19:40 +08:00
Lancer 31fedcfa2b [Misc] fix yaml config (#320) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2025-12-16 02:40:30 +00:00
汪志鹏 6c333161a4 [BugFix]Remove duplicate code (#309) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2025-12-15 01:42:55 +00:00
Zhou TaichangandGao Han c70b167f70 [Engine] Remove Diffusion_XX which duplicates with Generation_XX (#163) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-12-05 03:04:23 +00:00
Samit 04987ffa7a [Bugfix] Fix Qwen-omni Online Inference Bug caused by check_stop and long sequence (#112) 
Signed-off-by: samithuang <285365963@qq.com>
 2025-11-30 23:23:45 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-30 20:39:06 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-11-29 20:42:39 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-28 11:03:52 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Gaohan123 a04e945166 add precommit 2025-10-24 16:20:58 +08:00
tzhouam eb006f66e5 Enhance OmniScheduler error handling with logging 
- Added logging for exceptions when wrapping scheduled requests in OmniScheduler.
- Removed unnecessary blank lines for cleaner code structure.
 2025-10-24 12:00:36 +08:00
tzhouam 87b3ae7e70 Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers 2025-10-24 11:57:45 +08:00
tzhouam c23d24ca19 Refactor comments in diffusion_scheduler.py for clarity and consistency 
- Translated comments from Chinese to English for better accessibility.
- Improved clarity of comments related to request handling and scheduling logic.
- Ensured comments align with the code functionality and maintain a uniform style.
 2025-10-24 11:57:39 +08:00
Zhou TaichangandCopilot 9ebc1dd652 Apply suggestion from @Copilot 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-24 10:41:54 +08:00
tzhouam 4acaef3ccb Merge branch 'feat/omni-schedulers' of https://github.com/tzhouam/vllm-omni into feat/omni-schedulers 2025-10-24 09:41:29 +08:00
tzhouam c37cd6f0e3 delete unused imports 2025-10-24 09:41:20 +08:00
Zhou TaichangandCopilot b207bf89f2 Update vllm_omni/core/sched/diffusion_scheduler.py 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-22 17:44:20 +08:00
Zhou TaichangandCopilot 8d6a360f96 Update vllm_omni/core/sched/diffusion_scheduler.py 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2025-10-22 17:13:45 +08:00
tzhouam 19220aae82 Add scheduling components for vLLM-omni 
- Introduced `DiffusionScheduler` to handle scheduling for diffusion models, allowing immediate request completion.
- Added `OmniNewRequestData` to encapsulate additional request data, including prompt embeddings and extra information.
- Updated `OmniScheduler` to ensure compatibility with new request data structures.
- Created `output.py` to define new data classes for enriched request handling.
- Enhanced `__init__.py` in the engine module to support new payload structures for prompt embeddings and additional information.
 2025-10-22 15:05:23 +08:00
hsliu 81f04d3cf9 update ar_dit_test 2025-09-28 21:58:48 +08:00
hsliu 51d2570464 feat: Complete vLLM-omni implementation with conda environment setup 
- Add comprehensive PRD, architecture design, and test design documents
- Implement core modules: OmniLLM, AsyncOmniLLM, stage configurations
- Add DiT scheduler and cache manager for diffusion models
- Implement CLI integration with --omni flag support
- Add API server and plugin system for vLLM integration
- Create comprehensive test suite with fixtures
- Update dependencies to vLLM 0.10.2 and PyTorch 2.8.0
- Add conda environment setup and package installation
- Implement stage-based processing architecture
- Add multimodal output processing capabilities

This commit establishes the foundation for multi-modality models
inference and serving with non-autoregressive structures.
 2025-09-25 09:10:54 +08:00
hsliu 0c5caf6ab1 scheduler api design 2025-09-16 16:05:47 +08:00
hsliu dcb8868477 update config and entrypoints 2025-09-11 16:15:48 +08:00
hsliu 154d9ca4ff Initial vLLM-omni repository structure 
- Add complete directory structure for multimodal and non-autoregressive processing
- Include AsyncLLM, OmniEngine, and DiffusionEngine modules
- Add processors, schedulers, executors, and output processors
- Add API interfaces for Higress, Gradio, and ComfyUI
- Add configuration management and CLI interface
- Add comprehensive test structure and documentation
- Add pyproject.toml with dependencies and project configuration
- Add examples and documentation structure
 2025-09-11 14:32:51 +08:00