7bfd3c069d
[Benchmark] Add vLLM-Omni Omni model online benchmark ( #780 )
...
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-30 11:41:09 +08:00
Samit
c0beebbfe3
[Feature] Support cache-dit for Wan 2.2 inference ( #1021 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
2026-01-28 20:07:50 +08:00
D!NE$H
5e7035e183
[Feature]: Add CFG param to online serving ( #824 )
...
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
2026-01-19 04:23:15 +00:00
Chenguang Zheng
7e14c943d5
[Misc] Group omni arguments into OmniConfig section ( #744 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: ZEHNG Chenguang <645327136@qq.com >
Signed-off-by: ZHENG Chenguang <645327136@qq.com >
2026-01-12 23:20:20 +08:00
Prajwal A and zjy0516
779f5987d9
[feature] cpu offloading support for diffusion ( #497 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Prajwal A <prajwalanagani@gmail.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-01-12 10:41:18 +00:00
680742bc33
[Diffusion]: Diffusion Ring Attention support ( #273 )
...
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-31 14:26:30 +00:00
f0b8d3db82
[Feature] Control the stage init timeout threshold by --stage-init-timeout ( #393 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Li Shunyang <gmsv2544@nus.edu.sg >
Signed-off-by: Li Shunyang <lishunyang12@163.com >
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Signed-off-by: dengyunyang <584797741@qq.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: dengyunyang <584797741@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2025-12-30 23:37:10 +08:00
8939eb835d
[Rebase] Rebase to vllm 0.12.0 ( #335 )
...
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn >
2025-12-19 16:42:02 +08:00
WeiQing Chen
9464e1479d
[Entrypoints] online serving support usp args ( #366 )
...
Signed-off-by: David Chen <530634352@qq.com >
2025-12-19 11:59:07 +08:00
Chenguang Zheng and yinpe
acf66ee2cc
[Entrypoints] Support Online Serving for Diffusion-only Models ( #259 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn >
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn >
2025-12-17 17:40:45 +08:00
NATURE
f995211da0
[Feature] Omni Connector + ray supported ( #215 )
...
Signed-off-by: wzliu <wzliu@connect.hku.hk >
2025-12-10 20:15:03 +08:00
Gao Han
45dc189d8d
[Bugfix] Support passing vllm cli args to online serving in vLLM-Omni ( #206 )
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2025-12-08 14:59:53 +08:00
Roger Wang and Gao Han
a5800cd2c2
[Doc] Misc documentation polishing ( #98 )
...
Signed-off-by: Roger Wang <hey@rogerw.io >
Signed-off-by: Gao Han <gaohan19@huawei.com >
Co-authored-by: Gao Han <gaohan19@huawei.com >
2025-11-29 20:42:39 +08:00
Jiangyun Zhu
5707fc78d5
[CI] fix ci and format existing code ( #71 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
2025-11-18 20:54:26 +08:00
Gao Han
c5a89302f1
[Feature] Support online inference ( #64 )
...
Signed-off-by: Gaohan123 <gaohan19@huawei.com >
2025-11-17 14:44:10 +08:00
Gaohan123
6cf73c4e80
Remove unused files and fix some bugs
2025-10-25 18:02:11 +08:00
Gaohan123
a04e945166
add precommit
2025-10-24 16:20:58 +08:00
hsliu
05d23676ff
change omni.py to serve.py and mv to cli
2025-09-29 18:10:37 +08:00
hsliu
afe5c2692c
feat: enhance serving functionality and add comprehensive examples
...
- Improve API server with better error handling and response formatting
- Enhance CLI with additional options for DiT stages and configuration
- Add comprehensive examples in examples/basic/ including:
- API client with health checks and text generation
- Docker setup and usage examples
- Simple usage patterns for different scenarios
- Add utility scripts for model downloading and Docker setup
- Update documentation with implementation details and testing guidelines
- Fix configuration validation issues in OmniLLM
- Improve stage configuration handling for AR and DiT stages
- Add proper error handling and fallback mechanisms
Tested with Qwen3-0.6B model:
- Server starts successfully on port 8000
- Health and info endpoints working correctly
- Text generation with various parameters functioning
- API client examples working as expected
- CLI help and configuration options working properly
2025-09-26 15:47:55 +08:00
hsliu
51d2570464
feat: Complete vLLM-omni implementation with conda environment setup
...
- Add comprehensive PRD, architecture design, and test design documents
- Implement core modules: OmniLLM, AsyncOmniLLM, stage configurations
- Add DiT scheduler and cache manager for diffusion models
- Implement CLI integration with --omni flag support
- Add API server and plugin system for vLLM integration
- Create comprehensive test suite with fixtures
- Update dependencies to vLLM 0.10.2 and PyTorch 2.8.0
- Add conda environment setup and package installation
- Implement stage-based processing architecture
- Add multimodal output processing capabilities
This commit establishes the foundation for multi-modality models
inference and serving with non-autoregressive structures.
2025-09-25 09:10:54 +08:00
