anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
318 Commits 235 Branches 31 Tags
e1f96911203bcafb8aed81e04ea751b99b22bcdc
Commit Graph
71 Commits
Author SHA1 Message Date
Ziming Huang 1a7dcdf65c [Feature] Support Qwen Omni online batch inference (#438) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2025-12-25 19:38:48 +08:00
Wonjae Leeandwonjae.lee0 341457a121 [Feature] Support standard OpenAI API sampling parameters for thinker stage. (max_tokens...) (#450) 
Signed-off-by: wonjae.lee0 <wonjae.lee0@navercorp.com>
Co-authored-by: wonjae.lee0 <wonjae.lee0@navercorp.com>
 2025-12-24 20:39:01 +08:00
Doug Smith 74c142460d DALL-E compatible image generation endpoint (#292) 
Signed-off-by: dougbtv <dosmith@redhat.com>
 2025-12-23 09:05:37 +08:00
Peiqi Yin 0e878a92d2 [BugFix] raise error when request.stream=True. (#350) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
 2025-12-23 08:52:18 +08:00
dsinghvi 92c20afdc1 [Feature]: Adding traceback for error in background worker (#385) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
 2025-12-22 13:21:53 +13:00
TJianandHongxia Yang 13584fd98e [ROCm] [CI] Add ROCm Dockerfile and AMD CI (vllm v0.12.0) (#280) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
 2025-12-21 11:51:37 +08:00
rein yang c591ea124f [Bugfix] fix GPU VRAM calculation problem (#328) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2025-12-19 19:21:33 +08:00
WeiQing Chen 628c0a9217 [Entrypoints] Support Qwen-Image-Edit 2509 online serving & update doc (#368) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-19 16:50:32 +08:00
Zhou TaichangRein YangIsotr0py
 		8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:42:02 +08:00
WeiQing Chen 9464e1479d [Entrypoints] online serving support usp args (#366) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-19 11:59:07 +08:00
Gao Han c4cd502d82 [Misc] Fix left comments for PR298 (#346) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2025-12-17 22:40:20 +08:00
Chenguang Zhengandyinpe acf66ee2cc [Entrypoints] Support Online Serving for Diffusion-only Models (#259) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
 2025-12-17 17:40:45 +08:00
Gao Han 8722c6374c [Feature] Support output modalities control per request (#298) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2025-12-17 17:19:56 +08:00
TJian f785a5d401 [Bugfix] [Example] Add fallback to worker_backend and ray_address in AsyncOmni to fix gradio demos (#337) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2025-12-17 01:37:58 +00:00
dengyunyang bd53347615 [Feature] Send response with request id (#301) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2025-12-16 15:41:42 +00:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-16 15:19:40 +08:00
dengyunyang bd5c81cb7e [BugFix] Failed to print DEBUG level log (#307) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2025-12-15 07:13:36 +00:00
Jiangyun Zhu 9c03b80428 [Misc] move omni_diffusion.py to vllm_omni.entrypoints (#312) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-12-15 12:54:31 +08:00
Prajwal Azjy0516Samit
 		65ca13102d [Feature] teacache integration (#179) 
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2025-12-12 04:29:56 +00:00
LancerandLancer 0548674d30 [Entrypoints] Minor optimization in the orchestrator's final stage determination logic (#275) 
Signed-off-by: Lancer <maruixiang6688@gmai.com>
Co-authored-by: Lancer <maruixiang6688@gmail.com>
 2025-12-11 17:59:00 +08:00
NATURE f995211da0 [Feature] Omni Connector + ray supported (#215) 
Signed-off-by: wzliu <wzliu@connect.hku.hk>
 2025-12-10 20:15:03 +08:00
汪志鹏 71f9f9a9a7 [Bugfix] Remove debug code in AsyncOmni.del to fix resource leak (#260) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2025-12-09 23:23:01 +08:00
WeiQing Chen 9a7f7b7de5 [Bugfix]multi-GPU tp scenarios, devices: "0,1" uses physical IDs instead of logical IDs (#253) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-09 21:04:30 +08:00
Gao Han 45dc189d8d [Bugfix] Support passing vllm cli args to online serving in vLLM-Omni (#206) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2025-12-08 14:59:53 +08:00
Jiangyun Zhu aa689e6af0 [Core] add clean up method for diffusion engine (#219) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-12-08 03:52:07 +00:00
Mengqing Cao 00b1a9a1a0 [Misc] Temporarily support downloading models from ModelScope by snapshot download (#132) 
Signed-off-by: MengqingCao <cmq0113@163.com>
 2025-12-01 17:13:23 +00:00
Roger Wang c852267a1a [Misc] Update default stage config for qwen3-omni (#124) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 20:01:20 -08:00
Alicia a33bbcedea [Fix] Resolve comments & update vLLM-Omni name usages. (#122) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-12-01 08:05:24 +08:00
Alicia 3427fb4774 [Feat] Rename AsyncOmniLLM -> AsyncOmni (#103) 
Signed-off-by: WANG Cong <115451386+congw729@users.noreply.github.com>
 2025-11-30 22:26:33 +08:00
Gao Han bfd3cad5ee [Bugfix] Fix bug of online serving for qwen2.5-omni (#118) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 21:30:17 +08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-30 20:39:06 +08:00
Gao Han 88c00fb271 [Bugfix] Remove mandatory flash-attn dependency and optimzie docs (#113) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 18:20:44 +08:00
Zhou Taichang 228f4e5070 [Bug-fix] Fix Bugs in Qwen3/Qwen2.5 Omni Rebased Support (#114) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-30 18:15:10 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Roger WangandGao Han a5800cd2c2 [Doc] Misc documentation polishing (#98) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Gao Han <gaohan19@huawei.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-11-29 20:42:39 +08:00
Jiangyun Zhu bcc6bd9642 [diffusion] add doc and fix qwen-image (#96) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 18:07:44 +08:00
Syed Muhammad Bin Asif f4633fe381 [Bugfix] Fix removal of old logs when stats are enabled (#84) 
Signed-off-by: syedmba <syedmba7@connect.hku.hk>
 2025-11-28 15:58:16 +08:00
Jiangyun Zhu 4049f356f2 [Diffusion] Qwen image support (#82) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-28 14:22:11 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-28 11:03:52 +08:00
Baoyuan Qi 38f58064eb [fix] Remove redundant parameter passing (#90) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-27 20:26:51 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
tzhouam 26fcae5def Debug and Update doc 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 14:49:45 +08:00
tzhouam bc69d8e931 rename util further rename imports 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:57:15 +08:00
tzhouam 12cc2cd245 rename util 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:55:57 +08:00
tzhouam edae58b137 modify code based on review suggestions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:03:26 +08:00
tzhouam c83d223e91 Squash 7 commits: 36ada0478a 0ef1fefd45 519c140d20 c18fc40f10 5f3f53a39d da385277a2 edefcc244b 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-10 16:42:58 +08:00
tzhouam 8e8783d78a Refactor end-to-end inference script and enhance argument parsing. Added support for loading prompts from a .pt file, introduced new command-line arguments for initialization and output handling, and improved error handling for prompt loading. Removed deprecated files related to previous implementations. 2025-11-10 11:58:47 +08:00
tzhouam c5631ca753 Enhance error handling and logging in pipeline and stage processing. Added detailed logging for prompt loading failures, file writing issues, and stage shutdown processes. Introduced a new logging utility for orchestrator metrics and streamlined stats handling in the PipelinedOmniLLM class. 2025-11-07 18:05:00 +08:00