anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,317 Commits 235 Branches 31 Tags
b5ddff72429b2bf2485bfc0e98d2c28b4ec80310
Commit Graph
44 Commits
Author SHA1 Message Date
amy-why-3459 227bab3038 [Benchmark]Omni-modality model accuracy benchmark(Daily-Omni & seed-tts-eval) (#2558) 
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-04-15 11:07:35 +08:00
wuhangandDidan Deng cb91cbe61e [Entrypoint][Refactor]Stage CLI Refactor (#2020) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-04-10 14:40:37 +08:00
Zhou Taichang 2804a85e58 [Rebase] Rebase to vllm v0.19.0 (#2475) 2026-04-04 15:03:32 +08:00
Wu JIAZHENandjader 3fd4a4dc27 [Feat] Support step-boundary abort in diffusion (#1769) 
Signed-off-by: jader <yjader@foxmail.com>
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: jader <yjader@foxmail.com>
 2026-04-01 17:25:02 +08:00
dongbo910220andDidan Deng 50bd1961bf [Feature]: add Ulysses advanced_uaa mode (#1379) 
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: dongbo910220 <32610838+dongbo910220@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-24 14:55:54 +08:00
Zhou Taichangandgcanlin a90a7690b8 [Rebase] Rebase to vllm v0.18.0 (#2037) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-21 20:37:38 +08:00
dengyunyangbjf-frzDidan Deng
 		fde4808723 [Performance] diffusion timing (#1757) 
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-19 12:26:12 +08:00
Chenguang ZhengyinpePeiqi YinwuhangPeiqi Yinlinyueqianprincepridelishunyang
 		3158e9d28f [Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring (#1908) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com>
Co-authored-by: wuhang <whlbx@hotmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-03-18 17:33:53 +08:00
Didan Deng 30191ed112 [Feature]: Add vae-patch-parallel CLI argument in online serving (#1716) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-07 01:45:02 +08:00
Jack Leegoogle-labs-jules[bot]JackLeeHal
 		c0b714a8ac [Feature] Support flexible task_type configuration for Qwen3-TTS models (#1197) 
Signed-off-by: Jack Lee <695697442@qq.com>
Signed-off-by: jackleehal <695697442@qq.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com>
 2026-03-05 13:39:09 -05:00
WeiQing Chen 83fe7d005b [Model] support LTX-2 text-to-video image-to-video (#841) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-05 23:31:48 +08:00
Samit 22a51a7cdb [Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading (#1504) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-02 14:53:25 +00:00
WeiQing ChenIsotr0pyIsotr0pyHongsheng Liu
 		0b884ebc6c [Feature]: Native GGUF Quantization Support for DiT (#1285) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 11:39:14 +08:00
Canlin GuoandHongsheng Liu f53152d340 [Feature] Support HSDP for diffusion models (#1339) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:31:10 +08:00
Yanick Schraner 97f4c5b1e7 [Bugfix] Add TTS configuration options (#1177) 
Signed-off-by: Yanick Schraner <yanick.schraner@bs.ch>
 2026-02-26 00:14:44 +08:00
wuhang汪志鹏Copilot
 		36b8f80266 [Feature] Support Stage Based Deployment CLI (#939) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-24 16:55:23 +08:00
zhou zhuoxin 40f72bb870 [Feat]: add ASCII art logo for vLLM-Omni (#1430) 2026-02-23 22:10:09 +08:00
Yuanheng ZhaoandSamit 8bf849c399 Refactor CPU Offloading Backend Pattern (#1223) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2026-02-12 17:29:55 +08:00
Chenguang Zheng c6b526bc2e fix: add diffusion offload args to OmniConfig group instead of serve_parser (#1271) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-02-09 11:46:00 +08:00
ApsarasX 10f841bb98 [Bugfix] Fix '--max-generated-image-size' cli args type (#1249) 
Signed-off-by: ApsarasX <apsarax@outlook.com>
 2026-02-06 21:44:51 +08:00
Zhou Taichang d6f93b0f37 [Debug] Correct Unreasonable Long Timeout (#1175) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-03 08:49:51 +00:00
dengyunyang 70a5de9dd4 [FEATURE] /v1/images/edit interface (#1101) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-01-31 14:13:36 +08:00
Didan DengandHongsheng Liu 23cf48d7bb [Perf]: CFG parallel abstraction (#851) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 16:23:44 +00:00
Yuanheng ZhaoHongsheng LiuCopilot
 		2923d5a91e [Feature] Support DiT Layerwise (Blockwise) CPU Offloading (#858) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-30 17:35:09 +08:00
wangyuwangyu31577Hongsheng Liu
 		7bfd3c069d [Benchmark] Add vLLM-Omni Omni model online benchmark (#780) 
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-30 11:41:09 +08:00
Samit c0beebbfe3 [Feature] Support cache-dit for Wan 2.2 inference (#1021) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-01-28 20:07:50 +08:00
D!NE$H 5e7035e183 [Feature]: Add CFG param to online serving (#824) 
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
 2026-01-19 04:23:15 +00:00
Chenguang Zheng 7e14c943d5 [Misc] Group omni arguments into OmniConfig section (#744) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: ZEHNG Chenguang <645327136@qq.com>
Signed-off-by: ZHENG Chenguang <645327136@qq.com>
 2026-01-12 23:20:20 +08:00
Prajwal Aandzjy0516 779f5987d9 [feature] cpu offloading support for diffusion (#497) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-12 10:41:18 +00:00
XU MingshiJiangyun ZhuHongsheng Liu
 		680742bc33 [Diffusion]: Diffusion Ring Attention support (#273) 
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-31 14:26:30 +00:00
Zhou TaichangCopilotCanlin GuoPeiqi YinHongsheng LiuSYLARJiangyun ZhudengyunyangSamit
 		f0b8d3db82 [Feature] Control the stage init timeout threshold by --stage-init-timeout (#393) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Li Shunyang <gmsv2544@nus.edu.sg>
Signed-off-by: Li Shunyang <lishunyang12@163.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2025-12-30 23:37:10 +08:00
Zhou TaichangRein YangIsotr0py
 		8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:42:02 +08:00
WeiQing Chen 9464e1479d [Entrypoints] online serving support usp args (#366) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-19 11:59:07 +08:00
Chenguang Zhengandyinpe acf66ee2cc [Entrypoints] Support Online Serving for Diffusion-only Models (#259) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
 2025-12-17 17:40:45 +08:00
NATURE f995211da0 [Feature] Omni Connector + ray supported (#215) 
Signed-off-by: wzliu <wzliu@connect.hku.hk>
 2025-12-10 20:15:03 +08:00
Gao Han 45dc189d8d [Bugfix] Support passing vllm cli args to online serving in vLLM-Omni (#206) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2025-12-08 14:59:53 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-11-29 20:42:39 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
Gaohan123 a04e945166 add precommit 2025-10-24 16:20:58 +08:00
hsliu 05d23676ff change omni.py to serve.py and mv to cli 2025-09-29 18:10:37 +08:00
hsliu afe5c2692c feat: enhance serving functionality and add comprehensive examples 
- Improve API server with better error handling and response formatting
- Enhance CLI with additional options for DiT stages and configuration
- Add comprehensive examples in examples/basic/ including:
  - API client with health checks and text generation
  - Docker setup and usage examples
  - Simple usage patterns for different scenarios
- Add utility scripts for model downloading and Docker setup
- Update documentation with implementation details and testing guidelines
- Fix configuration validation issues in OmniLLM
- Improve stage configuration handling for AR and DiT stages
- Add proper error handling and fallback mechanisms

Tested with Qwen3-0.6B model:
- Server starts successfully on port 8000
- Health and info endpoints working correctly
- Text generation with various parameters functioning
- API client examples working as expected
- CLI help and configuration options working properly
 2025-09-26 15:47:55 +08:00
hsliu 51d2570464 feat: Complete vLLM-omni implementation with conda environment setup 
- Add comprehensive PRD, architecture design, and test design documents
- Implement core modules: OmniLLM, AsyncOmniLLM, stage configurations
- Add DiT scheduler and cache manager for diffusion models
- Implement CLI integration with --omni flag support
- Add API server and plugin system for vLLM integration
- Create comprehensive test suite with fixtures
- Update dependencies to vLLM 0.10.2 and PyTorch 2.8.0
- Add conda environment setup and package installation
- Implement stage-based processing architecture
- Add multimodal output processing capabilities

This commit establishes the foundation for multi-modality models
inference and serving with non-autoregressive structures.
 2025-09-25 09:10:54 +08:00