anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
568 Commits 235 Branches 31 Tags
2b009b3cefcdcdebd55cd3260150d83d89c05f21
Commit Graph
32 Commits
Author SHA1 Message Date
Zeyu Huang | 黃澤宇 d64bbde561 [Frontend][Model] Support batch request with refined OmniDiffusionReq… (#797) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-01-28 16:07:42 +08:00
Andy Zhoudongbo910220Samit
 		5037af1033 [Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment (#758) 
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk>
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-01-27 10:22:40 +08:00
amy-why-3459Rein YangCHENHongsheng Liu
 		52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-26 11:21:24 +08:00
SamitHongsheng LiuJiangyun Zhu
 		5d9918fa67 [Misc] Fix error log for the diffusion stage timeout (#915) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2026-01-23 15:16:40 +08:00
汪志鹏andwzliu 7f821be57b Support Bagel Model (#726) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
 2026-01-22 23:09:55 +13:00
SYLAR a1c37cc6e3 [Core] Implement Diffusion Profiler Support (#709) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-01-20 11:35:48 +08:00
dsinghvi 8c1259383c omni entrypoint support tokenizer arg (#572) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
 2026-01-08 15:44:14 +08:00
Canlin Guoandlishunyang 52afcb6055 [Feature] Support torch profiler across omni stages (#553) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-01-05 17:32:20 +08:00
Gao HanandHongsheng Liu 6b18d31aaa [Bugfix] Support both list and generator mode & fix unnecessary error log (#599) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-05 09:17:06 +08:00
Hongsheng Liu d7062550d7 [Core] omni refactor: optimize stage initialization polling and troubleshooting logs (#575) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-02 11:31:31 +08:00
Canlin GuoandHongsheng Liu 4f8cb95571 [Metric] Fix the computation of e2e_total_tokens (#519) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-01 00:13:17 +08:00
Gao Han 9c7de5373a [Feature]Basic version of supporting streaming output (#367) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2026-01-01 03:02:04 +13:00
Ziming HuangandHongsheng Liu ad0224f2e6 [Feature] Support Omni serving abort request (#486) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-31 16:14:03 +08:00
SYLARandHongsheng Liu 380c43eb86 [Feature] Add tqdm in Omni pipeline to align with Vllm (#552) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-31 04:44:11 +08:00
Peiqi YinandHongsheng Liu 2eebd22e32 [BugFix] __del__ exit don't join the spawned process (#555) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-31 03:59:25 +08:00
Gao Han 194fa159a3 [Bugfix]fix proces close problem for omni class (#554) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2025-12-31 08:23:54 +13:00
Zhou TaichangCopilotCanlin GuoPeiqi YinHongsheng LiuSYLARJiangyun ZhudengyunyangSamit
 		f0b8d3db82 [Feature] Control the stage init timeout threshold by --stage-init-timeout (#393) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Li Shunyang <gmsv2544@nus.edu.sg>
Signed-off-by: Li Shunyang <lishunyang12@163.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2025-12-30 23:37:10 +08:00
WeiQing ChenandHongsheng Liu 1753a878e0 [Bugfix] t2i online fix cache configuration failure and errors in /v1/images/generation endpoint. (#499) 
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-30 16:50:30 +08:00
Peiqi YinJiangyun ZhuHongsheng Liu
 		ebe37e9f57 [Refactor] let the AsyncOmni class inherit from Omni class (#511) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-29 21:04:17 +08:00
Peiqi YinandHongsheng Liu 60143adc52 [Core] remove deparated code from PR391 (#502) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-28 21:11:37 +13:00
Chenguang ZhengyinpeiqiHongsheng Liu
 		595e7c0032 [Core] Supports stage abstraction in the diffusion model (#391) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 01:59:33 +13:00
Jiangyun Zhu 9c03b80428 [Misc] move omni_diffusion.py to vllm_omni.entrypoints (#312) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-12-15 12:54:31 +08:00
Jiangyun Zhu aa689e6af0 [Core] add clean up method for diffusion engine (#219) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-12-08 03:52:07 +00:00
Mengqing Cao 00b1a9a1a0 [Misc] Temporarily support downloading models from ModelScope by snapshot download (#132) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-12-01 17:13:23 +00:00
Jiangyun Zhu bcc6bd9642 [diffusion] add doc and fix qwen-image (#96) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 18:07:44 +08:00
Jiangyun Zhu 4049f356f2 [Diffusion] Qwen image support (#82) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 14:22:11 +08:00
hsliu 05d23676ff change omni.py to serve.py and mv to cli 2025-09-29 18:10:37 +08:00
hsliu 81f04d3cf9 update ar_dit_test 2025-09-28 21:58:48 +08:00
hsliu e8b9425bab refactor: move omni_llm.py to entrypoints directory 
- Move vllm_omni/core/omni_llm.py to vllm_omni/entrypoints/omni_llm.py
- Update all import statements across the codebase to reflect new location
- Fix relative imports within the moved file
- Maintain functionality while improving code organization
- All imports and functionality tested and working correctly

This change better reflects that OmniLLM and AsyncOmniLLM are the main entry points
for vLLM-omni functionality, rather than core implementation details.
 2025-09-26 17:39:00 +08:00
hsliu afe5c2692c feat: enhance serving functionality and add comprehensive examples 
- Improve API server with better error handling and response formatting
- Enhance CLI with additional options for DiT stages and configuration
- Add comprehensive examples in examples/basic/ including:
  - API client with health checks and text generation
  - Docker setup and usage examples
  - Simple usage patterns for different scenarios
- Add utility scripts for model downloading and Docker setup
- Update documentation with implementation details and testing guidelines
- Fix configuration validation issues in OmniLLM
- Improve stage configuration handling for AR and DiT stages
- Add proper error handling and fallback mechanisms

Tested with Qwen3-0.6B model:
- Server starts successfully on port 8000
- Health and info endpoints working correctly
- Text generation with various parameters functioning
- API client examples working as expected
- CLI help and configuration options working properly
 2025-09-26 15:47:55 +08:00
hsliu 51d2570464 feat: Complete vLLM-omni implementation with conda environment setup 
- Add comprehensive PRD, architecture design, and test design documents
- Implement core modules: OmniLLM, AsyncOmniLLM, stage configurations
- Add DiT scheduler and cache manager for diffusion models
- Implement CLI integration with --omni flag support
- Add API server and plugin system for vLLM integration
- Create comprehensive test suite with fixtures
- Update dependencies to vLLM 0.10.2 and PyTorch 2.8.0
- Add conda environment setup and package installation
- Implement stage-based processing architecture
- Add multimodal output processing capabilities

This commit establishes the foundation for multi-modality models
inference and serving with non-autoregressive structures.
 2025-09-25 09:10:54 +08:00
hsliu dcb8868477 update config and entrypoints 2025-09-11 16:15:48 +08:00