anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
208 Commits 235 Branches 31 Tags
863cdb82b6d5db2a67351a8bae289774acd9e6aa
Commit Graph
50 Commits
Author SHA1 Message Date
Samit 5a0d7882c5 [Misc] Qwen-Omni support offline inference with local files (#167) 
Signed-off-by: samithuang <285365963@qq.com>
 2025-12-02 19:05:45 +08:00
Canlin Guo 7e60c49ce4 [Bugfix] Add Fake VllmConfig on NPU and add slicing/tiling args in Qwen-Image (#145) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-01 16:40:01 +00:00
Samit 98d94e0e3d [Bugfix] Fix Qwen3-Omni gradio audio input bug (#147) 
Signed-off-by: samithuang <285365963@qq.com>
 2025-12-01 13:52:31 +00:00
Samit 1a23d70a97 [Misc][WIP] Support qwen-omni online inference with local video/audio/image path (#131) 
Signed-off-by: samithuang <285365963@qq.com>
 2025-12-01 00:30:22 -08:00
Gao Han d53155f45c [Bugfix] Fix the curl bug of qwen3-omni and doc errors (#135) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-12-01 08:01:54 +00:00
Roger Wang c852267a1a [Misc] Update default stage config for qwen3-omni (#124) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 20:01:20 -08:00
Roger Wang 7391db62d1 [Doc] Cleanup non-english comments (#125) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 16:45:30 -08:00
Alicia a33bbcedea [Fix] Resolve comments & update vLLM-Omni name usages. (#122) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-12-01 08:05:24 +08:00
Alicia 3427fb4774 [Feat] Rename AsyncOmniLLM -> AsyncOmni (#103) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-11-30 22:26:33 +08:00
Hongsheng LiuandRoger Wang 0c245dd1f9 [Doc] Fix readme typos (#108) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Roger Wang <hey@rogerw.io>
 2025-11-30 21:32:04 +08:00
Gao Han bfd3cad5ee [Bugfix] Fix bug of online serving for qwen2.5-omni (#118) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 21:30:17 +08:00
Samit 3ea7fd280b [Feature] Support Gradio Demo for Qwen3-Omni (#116) 
Signed-off-by: samithuang <285365963@qq.com>
 2025-11-30 20:44:04 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-30 20:39:06 +08:00
Gao Han 88c00fb271 [Bugfix] Remove mandatory flash-attn dependency and optimzie docs (#113) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 18:20:44 +08:00
Jiangyun Zhu 425cbd49c1 [Fix] fix qwen3-omni example (#109) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-30 14:30:08 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-11-29 20:42:39 +08:00
Samit 1f1be495b0 [Diffusion] Support Multi-image Generation and Add Web UI Demo for QwenImage (#97) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-11-29 20:19:50 +08:00
Samit befc7fdfd7 Update gradio docs (#95) 
Signed-off-by: samithuang <285365963@qq.com>
 2025-11-28 15:06:33 +08:00
Jiangyun Zhu 4049f356f2 [Diffusion] Qwen image support (#82) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 14:22:11 +08:00
Samit b2d845742a [Feature] Add Gradio Demo for Qwen2.5Omni (#60) 
Signed-off-by: SamitHuang <285365963@qq.com>
Signed-off-by: samithuang <285365963@qq.com>
 2025-11-26 09:23:46 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
Samit 8cafb4294d [Docs] Setup Documentation System and Re-organize Dependencies (#49) 
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-11-15 21:45:43 +08:00
tzhouam c12bd006bc Make Example Easier To Use 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 11:56:08 +08:00
tzhouam b943abb22f add data download instructions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:49:53 +08:00
tzhouam edae58b137 modify code based on review suggestions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:03:26 +08:00
tzhouam c83d223e91 Squash 7 commits: 36ada0478a 0ef1fefd45 519c140d20 c18fc40f10 5f3f53a39d da385277a2 edefcc244b 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 16:42:58 +08:00
tzhouam 08aef1f964 remove text output for audio output 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 14:41:44 +08:00
tzhouam 8e8783d78a Refactor end-to-end inference script and enhance argument parsing. Added support for loading prompts from a .pt file, introduced new command-line arguments for initialization and output handling, and improved error handling for prompt loading. Removed deprecated files related to previous implementations. 2025-11-10 11:58:47 +08:00
tzhouam c5631ca753 Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 2025-11-07 18:05:00 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Gaohan123 1aa5ae61cb fix reviews 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 23:26:58 +08:00
Gaohan123 c80348ce71 supplement some new infomation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 10:47:30 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 03ae52a064 add installation of uv 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:18:19 +08:00
Gaohan123 1702033c3e fix some typos 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 17:16:25 +08:00
Gaohan123 27f06e27fc Further supplement installation guide 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-30 14:45:54 +08:00
Gaohan123 78830c8ad9 fix problem of installation 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-10-27 22:06:35 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Gaohan123 a04e945166 add precommit 2025-10-24 16:20:58 +08:00
Gaohan123 1714777c8d Add example of audio file 2025-10-23 14:33:33 +08:00
Gaohan123 9edce9e12a Add end2end example and documentation for qwen2.5-omni 2025-10-22 15:26:42 +08:00
hsliu 772583140f del reamds 2025-09-29 17:53:03 +08:00
hsliu 81f04d3cf9 update ar_dit_test 2025-09-28 21:58:48 +08:00
hsliu e8b9425bab refactor: move omni_llm.py to entrypoints directory 
- Move vllm_omni/core/omni_llm.py to vllm_omni/entrypoints/omni_llm.py
- Update all import statements across the codebase to reflect new location
- Fix relative imports within the moved file
- Maintain functionality while improving code organization
- All imports and functionality tested and working correctly

This change better reflects that OmniLLM and AsyncOmniLLM are the main entry points
for vLLM-omni functionality, rather than core implementation details.
 2025-09-26 17:39:00 +08:00
hsliu afe5c2692c feat: enhance serving functionality and add comprehensive examples 
- Improve API server with better error handling and response formatting
- Enhance CLI with additional options for DiT stages and configuration
- Add comprehensive examples in examples/basic/ including:
  - API client with health checks and text generation
  - Docker setup and usage examples
  - Simple usage patterns for different scenarios
- Add utility scripts for model downloading and Docker setup
- Update documentation with implementation details and testing guidelines
- Fix configuration validation issues in OmniLLM
- Improve stage configuration handling for AR and DiT stages
- Add proper error handling and fallback mechanisms

Tested with Qwen3-0.6B model:
- Server starts successfully on port 8000
- Health and info endpoints working correctly
- Text generation with various parameters functioning
- API client examples working as expected
- CLI help and configuration options working properly
 2025-09-26 15:47:55 +08:00
hsliu 154d9ca4ff Initial vLLM-omni repository structure 
- Add complete directory structure for multimodal and non-autoregressive processing
- Include AsyncLLM, OmniEngine, and DiffusionEngine modules
- Add processors, schedulers, executors, and output processors
- Add API interfaces for Higress, Gradio, and ComfyUI
- Add configuration management and CLI interface
- Add comprehensive test structure and documentation
- Add pyproject.toml with dependencies and project configuration
- Add examples and documentation structure
 2025-09-11 14:32:51 +08:00