anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
453 Commits 235 Branches 31 Tags
b4775d52fed7692ea464c8fbcd5689b16977c840
Commit Graph
110 Commits
Author SHA1 Message Date
copilot-swe-agent[bot]andhsliuustc0106 b4775d52fe Extract usage info from text output before modality filtering 
This ensures that usage/billing data is captured even when text output
is filtered out (e.g., for audio-only requests). Previously, audio-only
requests would return zero usage tokens because the text output was
skipped before extracting usage information.

Co-authored-by: hsliuustc0106 <222337142+hsliuustc0106@users.noreply.github.com>
 2026-01-19 04:11:36 +00:00
copilot-swe-agent[bot]andhsliuustc0106 be7d524326 Fix line length linting issue 
Co-authored-by: hsliuustc0106 <222337142+hsliuustc0106@users.noreply.github.com>
 2026-01-19 03:07:31 +00:00
copilot-swe-agent[bot]andhsliuustc0106 c7bdfb2934 Add modalities filtering to non-streaming chat completions 
Co-authored-by: hsliuustc0106 <222337142+hsliuustc0106@users.noreply.github.com>
 2026-01-19 03:04:26 +00:00
Yuhan Liu e213bdd0a4 [BugFix]Remove duplicate error handling for request results (#781) 
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
 2026-01-14 20:08:23 +08:00
Ziming Huang 1444e1fc45 [Feature] Support Qwen3 Omni talker mtp batch inference (#722) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-14 19:07:31 +08:00
JustQJ 9cdf592782 [Misc] Enable tensor_parallel_size argument with online serving cmd (#761) 
Signed-off-by: TangPeng <85704592@qq.com>
 2026-01-13 15:55:25 +08:00
Chenguang Zheng 7e14c943d5 [Misc] Group omni arguments into OmniConfig section (#744) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: ZEHNG Chenguang <645327136@qq.com>
Signed-off-by: ZHENG Chenguang <645327136@qq.com>
 2026-01-12 23:20:20 +08:00
Prajwal Aandzjy0516 779f5987d9 [feature] cpu offloading support for diffusion (#497) 
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
 2026-01-12 10:41:18 +00:00
Sihao LiandHongsheng Liu 2c92b63cce fix:#issue 432 (#517) 
Signed-off-by: GG-li <3226868735@qq.com>
Signed-off-by: Sihao Li <111170255+GG-li@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-12 15:38:47 +08:00
Ziming Huang 978e3ad7d4 [Feature] Support Qwen3 Omni talker cudagraph (#669) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-09 11:45:58 +08:00
Ziming Huang cddcc400d3 [BugFix] Fix stage engine outputs mismatch bug in online batching (#691) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-09 04:01:57 +13:00
Yuanheng Zhao 4c72b67444 [BugFix] Explicitly release file locks during stage worker init (#703) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
 2026-01-08 20:35:00 +08:00
Junhong Liu bb13467adc [Bug fix] fix e2e_total_tokens and e2e_total_time_ms (#648) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-01-08 20:25:04 +08:00
dsinghvi 8c1259383c omni entrypoint support tokenizer arg (#572) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
 2026-01-08 15:44:14 +08:00
Yueqian LinandHongsheng Liu 0cb7ec62da [Model] Add Stable Audio Open support for text-to-audio generation (#331) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-07 17:01:06 +08:00
Chenguang Zheng aaba243832 [BUGFIX] Delete the CUDA context in the stage process. (#661) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: Chenguang Zheng <645327136@qq.com>
 2026-01-06 22:22:04 +08:00
汪志鹏 e7eeb54e08 [BugFix]Fix text_to_image bagel command error (#644) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-01-05 23:22:15 +13:00
Canlin Guoandlishunyang 52afcb6055 [Feature] Support torch profiler across omni stages (#553) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-01-05 17:32:20 +08:00
Ziming Huang f9b8bfd7bb [BugFix] AttributeError: 'AsyncOmniDiffusion' object has no attribute 'abort' (#624) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-01-05 14:53:10 +08:00
Gao HanandHongsheng Liu 6b18d31aaa [Bugfix] Support both list and generator mode & fix unnecessary error log (#599) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-05 09:17:06 +08:00
catcat 7961eb6a02 [Bugfix] Fix on_finalize_request method arguments number mismatch (#583) 
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
 2026-01-04 09:51:46 +08:00
Hongsheng Liu d7062550d7 [Core] omni refactor: optimize stage initialization polling and troubleshooting logs (#575) 
Signed-off-by: hsliu <liuhongsheng4@huawei.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-02 11:31:31 +08:00
Canlin GuoandHongsheng Liu 4f8cb95571 [Metric] Fix the computation of e2e_total_tokens (#519) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-01 00:13:17 +08:00
XU MingshiJiangyun ZhuHongsheng Liu
 		680742bc33 [Diffusion]: Diffusion Ring Attention support (#273) 
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-31 14:26:30 +00:00
Gao Han 9c7de5373a [Feature]Basic version of supporting streaming output (#367) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2026-01-01 03:02:04 +13:00
Ziming HuangandHongsheng Liu ad0224f2e6 [Feature] Support Omni serving abort request (#486) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-31 16:14:03 +08:00
汪志鹏wzliuHongsheng Liu
 		23bf317cc5 [New Model]Bagel model(Diffusion Only) (#319) 
Signed-off-by: wzliu <wzliu@connect.hku.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: wzliu <wzliu@connect.hku.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-31 13:47:35 +08:00
SYLARandHongsheng Liu 380c43eb86 [Feature] Add tqdm in Omni pipeline to align with Vllm (#552) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-31 04:44:11 +08:00
Peiqi YinandHongsheng Liu 2eebd22e32 [BugFix] __del__ exit don't join the spawned process (#555) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-31 03:59:25 +08:00
Gao Han 194fa159a3 [Bugfix]fix proces close problem for omni class (#554) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2025-12-31 08:23:54 +13:00
Zhou TaichangCopilotCanlin GuoPeiqi YinHongsheng LiuSYLARJiangyun ZhudengyunyangSamit
 		f0b8d3db82 [Feature] Control the stage init timeout threshold by --stage-init-timeout (#393) 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Li Shunyang <gmsv2544@nus.edu.sg>
Signed-off-by: Li Shunyang <lishunyang12@163.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: Samit <285365963@qq.com>
 2025-12-30 23:37:10 +08:00
WeiQing ChenandHongsheng Liu 1753a878e0 [Bugfix] t2i online fix cache configuration failure and errors in /v1/images/generation endpoint. (#499) 
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-30 16:50:30 +08:00
Peiqi YinJiangyun ZhuHongsheng Liu
 		ebe37e9f57 [Refactor] let the AsyncOmni class inherit from Omni class (#511) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-29 21:04:17 +08:00
Peiqi YinandHongsheng Liu 60143adc52 [Core] remove deparated code from PR391 (#502) 
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-28 21:11:37 +13:00
Bhanu Prakash VoutharojaHongsheng LiuYueqian Lin
 		2f69748ee4 add openai create speech endpoint (#305) 
Signed-off-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
Signed-off-by: Bhanu Prakash Voutharoja <59905694+Bhanu068@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2025-12-27 13:56:51 +08:00
Ziming HuangandHongsheng Liu d456721d81 [BugFix] Make max_batch_size work in Omni online serving (#487) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-26 21:51:07 +08:00
Chenguang ZhengyinpeiqiHongsheng Liu
 		595e7c0032 [Core] Supports stage abstraction in the diffusion model (#391) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 01:59:33 +13:00
Junhong Liu 51eefbfebc [BugFix] initial sampling_params_list when offline inference (#468) 
Signed-off-by: Junhong <liujunhong11@huawei.com>
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
 2025-12-26 13:54:12 +08:00
dengyunyangandHongsheng Liu a500ca1f28 [Feature] logger refactor (#466) 
Signed-off-by: dengyunyang <584797741@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-26 08:42:06 +08:00
Ziming Huang 1a7dcdf65c [Feature] Support Qwen Omni online batch inference (#438) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2025-12-25 19:38:48 +08:00
Wonjae Leeandwonjae.lee0 341457a121 [Feature] Support standard OpenAI API sampling parameters for thinker stage. (max_tokens...) (#450) 
Signed-off-by: wonjae.lee0 <wonjae.lee0@navercorp.com>
Co-authored-by: wonjae.lee0 <wonjae.lee0@navercorp.com>
 2025-12-24 20:39:01 +08:00
Doug Smith 74c142460d DALL-E compatible image generation endpoint (#292) 
Signed-off-by: dougbtv <dosmith@redhat.com>
 2025-12-23 09:05:37 +08:00
Peiqi Yin 0e878a92d2 [BugFix] raise error when request.stream=True. (#350) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
 2025-12-23 08:52:18 +08:00
dsinghvi 92c20afdc1 [Feature]: Adding traceback for error in background worker (#385) 
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
 2025-12-22 13:21:53 +13:00
TJianandHongxia Yang 13584fd98e [ROCm] [CI] Add ROCm Dockerfile and AMD CI (vllm v0.12.0) (#280) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
Co-authored-by: Hongxia Yang <hongxia.yang@amd.com>
 2025-12-21 11:51:37 +08:00
rein yang c591ea124f [Bugfix] fix GPU VRAM calculation problem (#328) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2025-12-19 19:21:33 +08:00
WeiQing Chen 628c0a9217 [Entrypoints] Support Qwen-Image-Edit 2509 online serving & update doc (#368) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-19 16:50:32 +08:00
Zhou TaichangRein YangIsotr0py
 		8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2025-12-19 16:42:02 +08:00
WeiQing Chen 9464e1479d [Entrypoints] online serving support usp args (#366) 
Signed-off-by: David Chen <530634352@qq.com>
 2025-12-19 11:59:07 +08:00
Gao Han c4cd502d82 [Misc] Fix left comments for PR298 (#346) 
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
 2025-12-17 22:40:20 +08:00