anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
337 Commits 235 Branches 31 Tags
be81443c8a52f863452253ee6ea1d9f263bd2fb6
Commit Graph
56 Commits
Author SHA1 Message Date
Canlin Guo be81443c8a [Bugfix][NPU] Add _model_forward for ModelRunner (#505) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-28 04:09:34 +08:00
Chenguang ZhengyinpeiqiHongsheng Liu
 		595e7c0032 [Core] Supports stage abstraction in the diffusion model (#391) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 01:59:33 +13:00
Canlin GuoandHongsheng Liu 52d15c4c04 [NPU][Model] Support Qwen3-Omni on NPU (#484) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 00:05:55 +13:00
Peiqi Yin e9e6b3711c [Core] customize process between stages, remove the need of "request_ids" in model forward, rebase to 0.12.0 (#458) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-24 13:26:33 +00:00
Peiqi Yin da7f25e14b [BugFix] Qwen3-Omni / Qwen2.5-Omni thinker batching with audio input (#451) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-24 18:43:00 +08:00
TJianandHongxia Yang 13584fd98e [ROCm] [CI] Add ROCm Dockerfile and AMD CI (vllm v0.12.0) (#280) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
 2025-12-21 11:51:37 +08:00
Zhou TaichangRein YangIsotr0py
 		8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:42:02 +08:00
Gao Han 8722c6374c [Feature] Support output modalities control per request (#298) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2025-12-17 17:19:56 +08:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126) 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
 2025-12-16 15:19:40 +08:00
Lancer 31fedcfa2b [Misc] fix yaml config (#320) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2025-12-16 02:40:30 +00:00
WeiQing Chen 3f699a7b4d [Misc] fix stage config describe and yaml format (#265) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-11 13:34:52 +13:00
NATURE f995211da0 [Feature] Omni Connector + ray supported (#215) 
Signed-off-by: wzliu <wzliu@connect.hku.hk>
 2025-12-10 20:15:03 +08:00
WeiQing Chen 9a7f7b7de5 [Bugfix]multi-GPU tp scenarios, devices: "0,1" uses physical IDs instead of logical IDs (#253) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-09 21:04:30 +08:00
WeiQing Chen 75a84c8127 [Bugfix] fix issue251, qwen3 omni does not support chunked prefill now (#256) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-09 20:27:37 +08:00
rein yang baf1086d55 [CI] Add Qwen3-omni offline UT (#216) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2025-12-08 18:11:39 +08:00
Peiqi Yin bbb5a72871 [BugFix] Fix qwen3omni thinker batching. (#207) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
 2025-12-08 04:44:13 +00:00
Isotr0py 059f45e48d [Diffusion] Refactor diffusion models weights loading (#157) 
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-06 11:12:13 +00:00
Canlin Guo 6079b14040 [NPU][Refactor] Rename Diffusion* to Generation* (#211) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-05 08:23:37 +00:00
Chenguang Zheng a3f5cd55ee [bugfix] qwen2.5 omni does not support chunked prefill now (#193) 
Signed-off-by: ZHENG Chenguang <645327136@qq.com>
 2025-12-05 04:12:17 +00:00
Zhou TaichangandGao Han c70b167f70 [Engine] Remove Diffusion_XX which duplicates with Generation_XX (#163) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-12-05 03:04:23 +00:00
汪志鹏 f3c69df419 [FixBug]online serving fails for high-resolution videos (#198) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2025-12-05 02:29:01 +00:00
Roger Wang c852267a1a [Misc] Update default stage config for qwen3-omni (#124) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 20:01:20 -08:00
Roger Wang 7391db62d1 [Doc] Cleanup non-english comments (#125) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-30 16:45:30 -08:00
Canlin Guo ec9e04027d [Feat] Add NPU Backend support for vLLM-Omni (#89) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-11-30 20:39:06 +08:00
Gao Han 88c00fb271 [Bugfix] Remove mandatory flash-attn dependency and optimzie docs (#113) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-30 18:20:44 +08:00
Roger Wang 22cd976980 [Bugfix] Fix special token nothink naming. (#107) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2025-11-29 16:07:26 -08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
 2025-11-30 04:44:01 +08:00
Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-28 11:03:52 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
 2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64) 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-17 14:44:10 +08:00
Baoyuan Qi bf82e18374 [fix] adapt hidden state device for multi-hardware support (#61) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-16 17:31:04 +08:00
Samit 8cafb4294d [Docs] Setup Documentation System and Re-organize Dependencies (#49) 
Signed-off-by: SamitHuang <285365963@qq.com>
 2025-11-15 21:45:43 +08:00
Gao Han a089dd4c4e Merge pull request #51 from tzhouam/feat/multi-request-stream-new 
[Feature] Multi Request Stream for Sync Mode
 2025-11-13 10:14:27 +08:00
tzhouam edae58b137 modify code based on review suggestions 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2025-11-11 10:03:26 +08:00
齐保元 d7618b1be4 [fix] support loading model from local path 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2025-11-09 04:37:19 +08:00
tzhouam 6a7a27801b Port changes from feat/multi-request-stream (vs main) 2025-11-07 15:48:47 +08:00
Gaohan123 e2b803fbe6 fix a bug 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-05 13:05:12 +08:00
Gaohan123 6e1478222e Dependency refractored to vLLM v0.11.0 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-04 22:06:41 +08:00
Gaohan123 d4a364f052 fix huggingface download problem for spk_dict.pt 
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
 2025-11-01 16:45:34 +08:00
Gaohan123 6cf73c4e80 Remove unused files and fix some bugs 2025-10-25 18:02:11 +08:00
tzhouam d45f8a6be7 debug typo 2025-10-24 17:47:49 +08:00
tzhouam 61b344dc39 Add import for OmniModelsRegistry in model executor initialization 2025-10-24 17:44:31 +08:00
Gaohan123 171f3c1ea7 fix conflicts 2025-10-24 16:49:19 +08:00
Gaohan123 22f36de34e fix error 2025-10-24 16:21:52 +08:00
Gaohan123 eafc7c44c3 fix error for pre-commit 2025-10-24 16:21:50 +08:00
Hongsheng Liu 15dbe97750 Merge pull request #13 from Gaohan123/entrypoint_stage 
[Engine] Add entrypoint class and stage management
 2025-10-24 10:38:16 +08:00
Gaohan123 2c08c04730 fix for reviews of copilot 2025-10-24 10:17:22 +08:00
tzhouam ae3b7504b2 Merge branch 'feat/qwen_2_5_omni_models' of https://github.com/tzhouam/vllm-omni into feat/qwen_2_5_omni_models 2025-10-24 10:02:58 +08:00
tzhouam 52905a0d17 Enhance Qwen2.5-Omni model by defining TALKER_CODEC_EOS_TOKEN_ID and TALKER_CODEC_BOS_TOKEN_ID constants. Update audio generation logic to use these constants for improved clarity and maintainability. Refactor tensor initialization to utilize the talker's hidden size. Update comments for better understanding and consistency. 2025-10-24 10:02:51 +08:00