anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,775 Commits 235 Branches 31 Tags
c69ed4037dcbae7306cba9094cb42bf94de464f0
Commit Graph
498 Commits
Author SHA1 Message Date
Cyrus Leung 574e1fb68d [Doc] Update Security Advisory link (#173) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-03 12:39:08 +08:00
wjcwjc77wangjiangchengGao Han
 		8ed9716d5b [Doc] Fix wrong examples URLs (#166) 
Signed-off-by: wangjiangcheng <wangjiangcheng@kuaishou.com>
Co-authored-by: wangjiangcheng <wangjiangcheng@kuaishou.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-12-03 03:25:58 +00:00
Jiangyun ZhuandSamit 7a6d2fe3f6 [diffusion] z-image support (#149) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2025-12-03 02:16:36 +00:00
Canlin Guo 7e60c49ce4 [Bugfix] Add Fake VllmConfig on NPU and add slicing/tiling args in Qwen-Image (#145) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-01 16:40:01 +00:00
Roger Wang 510aeb7da4 [Misc] Misc description updates (#146) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-12-01 12:09:24 +00:00
Roger Wang 5939089c98 [Doc] Logo (#143) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-12-01 02:26:39 -08:00
Roger Wang a6c5a6aca4 [Doc] Update developer & user channel (#138) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-12-01 00:25:44 -08:00
Gao Han d53155f45c [Bugfix] Fix the curl bug of qwen3-omni and doc errors (#135) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-12-01 08:01:54 +00:00
youkaichao e8fb85d38c Update README with vllm-omni blogpost link (#137) 
Signed-off-by: youkaichao <youkaichao@gmail.com>
 2025-11-30 23:31:32 -08:00
Roger Wang a1914e1176 [Doc] Fix arch pic reference (#136) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 23:17:21 -08:00
Roger Wang 3b12ed9cf4 [Doc] Cleanup reference to deleted files (#134) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 22:50:21 -08:00
Roger Wang c852267a1a [Misc] Update default stage config for qwen3-omni (#124) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 20:01:20 -08:00
Roger Wang 7391db62d1 [Doc] Cleanup non-english comments (#125) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 16:45:30 -08:00
Yikun Jiang 90ba189232 Refresh supported models and address nits in doc (#119) 
Signed-off-by: Yikun Jiang <yikunkero@gmail.com>
 2025-11-30 16:36:04 -08:00
Alicia a33bbcedea [Fix] Resolve comments & update vLLM-Omni name usages. (#122) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-12-01 08:05:24 +08:00
Alicia 3427fb4774 [Feat] Rename AsyncOmniLLM -> AsyncOmni (#103) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-11-30 22:26:33 +08:00
Hongsheng LiuandRoger Wang 0c245dd1f9 [Doc] Fix readme typos (#108) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-30 21:32:04 +08:00
Gao Han bfd3cad5ee [Bugfix] Fix bug of online serving for qwen2.5-omni (#118) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 21:30:17 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-30 20:39:06 +08:00
Gao Han 88c00fb271 [Bugfix] Remove mandatory flash-attn dependency and optimzie docs (#113) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 18:20:44 +08:00
Samit f77e44d5bf [Docs] Add qwen image missing doc in user guide (#111) 
Signed-off-by: samithuang <285365963@qq.com>
 2025-11-30 17:46:39 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-11-29 20:42:39 +08:00
Samit 1f1be495b0 [Diffusion] Support Multi-image Generation and Add Web UI Demo for QwenImage (#97) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-11-29 20:19:50 +08:00
Jiangyun Zhu bcc6bd9642 [diffusion] add doc and fix qwen-image (#96) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 18:07:44 +08:00
Jiangyun Zhu 4049f356f2 [Diffusion] Qwen image support (#82) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 14:22:11 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-28 11:03:52 +08:00
Hongsheng LiuandRoger Wang 1fb46ddd62 update readme for v0.11.0rc1 release (#69) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-24 14:28:25 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
Samit 8cafb4294d [Docs] Setup Documentation System and Re-organize Dependencies (#49) 
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-11-15 21:45:43 +08:00
tzhouam 8f6280f4f6 adjust the doc 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-12 15:51:56 +08:00
tzhouam 26fcae5def Debug and Update doc 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 14:49:45 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 fed8785eb9 fix some docs 2025-10-25 18:02:16 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
Gaohan123 a04e945166 add precommit 2025-10-24 16:20:58 +08:00
hsliu c330e5e8f3 del unnessearcy files 2025-09-29 17:49:32 +08:00
hsliu 81f04d3cf9 update ar_dit_test 2025-09-28 21:58:48 +08:00
Hongsheng Liuandgemini-code-assist[bot] ffeb0dc407 Update docs/architecture/detailed arch design.md 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-26 15:52:18 +08:00
Hongsheng Liuandgemini-code-assist[bot] 91f60bee94 Update docs/architecture/detailed arch design.md 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-26 15:51:58 +08:00
Hongsheng Liuandgemini-code-assist[bot] e58e62fcbe Update docs/architecture/detailed arch design.md 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2025-09-26 15:51:49 +08:00
hsliu afe5c2692c feat: enhance serving functionality and add comprehensive examples 
- Improve API server with better error handling and response formatting
- Enhance CLI with additional options for DiT stages and configuration
- Add comprehensive examples in examples/basic/ including:
  - API client with health checks and text generation
  - Docker setup and usage examples
  - Simple usage patterns for different scenarios
- Add utility scripts for model downloading and Docker setup
- Update documentation with implementation details and testing guidelines
- Fix configuration validation issues in OmniLLM
- Improve stage configuration handling for AR and DiT stages
- Add proper error handling and fallback mechanisms

Tested with Qwen3-0.6B model:
- Server starts successfully on port 8000
- Health and info endpoints working correctly
- Text generation with various parameters functioning
- API client examples working as expected
- CLI help and configuration options working properly
 2025-09-26 15:47:55 +08:00
hsliu 51d2570464 feat: Complete vLLM-omni implementation with conda environment setup 
- Add comprehensive PRD, architecture design, and test design documents
- Implement core modules: OmniLLM, AsyncOmniLLM, stage configurations
- Add DiT scheduler and cache manager for diffusion models
- Implement CLI integration with --omni flag support
- Add API server and plugin system for vLLM integration
- Create comprehensive test suite with fixtures
- Update dependencies to vLLM 0.10.2 and PyTorch 2.8.0
- Add conda environment setup and package installation
- Implement stage-based processing architecture
- Add multimodal output processing capabilities

This commit establishes the foundation for multi-modality models
inference and serving with non-autoregressive structures.
 2025-09-25 09:10:54 +08:00
hsliu c1da072744 del api md 2025-09-16 16:54:27 +08:00
hsliu 0c5caf6ab1 scheduler api design 2025-09-16 16:05:47 +08:00
hsliu 154d9ca4ff Initial vLLM-omni repository structure 
- Add complete directory structure for multimodal and non-autoregressive processing
- Include AsyncLLM, OmniEngine, and DiffusionEngine modules
- Add processors, schedulers, executors, and output processors
- Add API interfaces for Higress, Gradio, and ComfyUI
- Add configuration management and CLI interface
- Add comprehensive test structure and documentation
- Add pyproject.toml with dependencies and project configuration
- Add examples and documentation structure
 2025-09-11 14:32:51 +08:00