Commit Graph
37 Commits
Author SHA1 Message Date
tzhouam f24273dc03 Update vLLM version to 0.16.0 in quickstart and CUDA installation documentation, and adjust parameters in multimodal generation script for improved performance. Refactor request logging in serving_chat.py for better input tracking.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-13 09:50:11 +00:00
tzhouam 31e9fdbb57 Refactor chat_utils to remove unused multi-modal classes and add async audio extraction from video. Update README to reflect new entry point for audio extraction. Enhance serving_chat to process audio from video URLs when specified.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-13 03:27:16 +00:00
tzhouam 7e9cae56fc Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-13 02:28:25 +00:00
Taichang Zhou a6d633c319 debug
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
2026-02-12 23:46:20 +08:00
tzhouam 59a7602d6e debug
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-12 13:55:08 +00:00
Taichang Zhou 85a7b558ab rebase to v0.16.0
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
2026-02-12 21:30:54 +08:00
tzhouam c751f0d684 Update Dockerfile.ci to install a new version of vllm from a different extra index URL. Refactor import statement in serving_chat.py to use the correct BaseRenderer module path.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-12 06:37:17 +00:00
tzhouamandCursor 50f2b1e854 Merge origin/main into dev/vllm-align
Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-12 03:41:38 +00:00
Junhong Liu 82fd2c3637 Fix yield token metrics and opt metrics record stats (#1292) 2026-02-12 00:52:05 +08:00
tzhouam 279b6b5050 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-11 01:44:54 +00:00
amy-why-3459 ee624d0686 [BugFix] Fixed the issue where ignore_eos was not working. (#1286)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
2026-02-10 16:08:20 +08:00
tzhouam 7e4a604d4d reabse 2025/2/9
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
2026-02-09 03:07:43 +00:00
tzhouam 24b04f58a0 rebase 2026/2/5 2026-02-05 10:52:28 +00:00
Zhou Taichangand汪志鹏 70ef548448 Dev/rebase v0.15.0 (#1159)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
2026-02-03 12:24:45 +08:00
ceanna93andanna df3b347cd9 [Misc] Align error handling with upstream vLLM v0.14.0 (#1122)
Signed-off-by: anna <lee.anna@navercorp.com>
Co-authored-by: anna <lee.anna@navercorp.com>
2026-02-01 21:59:08 +08:00
Pierre LE GUEN 2c294a750b fix: UnboundLocalError for role in streaming audio/image responses (#784)
Signed-off-by: Pierre Le Guen <26087574+PierreLeGuen@users.noreply.github.com>
2026-01-30 16:27:04 +08:00
amy-why-3459andRein Yang dd518973de [BugFix] Fix the accuracy issue of multimodal input. (#1020)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
2026-01-29 19:30:11 +08:00
Zeyu Huang | 黃澤宇 d64bbde561 [Frontend][Model] Support batch request with refined OmniDiffusionReq… (#797)
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
2026-01-28 16:07:42 +08:00
5037af1033 [Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment (#758)
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk>
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Samit <285365963@qq.com>
2026-01-27 10:22:40 +08:00
52bf1e2f40 [Feature]Support async computation and communication across stages by chunks (#727)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2026-01-26 11:21:24 +08:00
Sihao Li 0aa72b952f [bugfix] support text + audio mixed output (#843)
Signed-off-by: GG-li <3226868735@qq.com>
2026-01-21 17:44:17 +08:00
+13 776c3a794e Dev/rebase 0.14.0 (#847)
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: root <root@hk01dgx039.cm.cluster>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: TangPeng <85704592@qq.com>
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Signed-off-by: mxuax <mxuax@connect.ust.hk>
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com>
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
Signed-off-by: root <root@hk01dgx028.cm.cluster>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: JaredforReal <w13431838023@gmail.com>
Co-authored-by: root <root@hk01dgx039.cm.cluster>
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com>
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com>
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com>
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com>
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com>
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com>
Co-authored-by: root <root@hk01dgx028.cm.cluster>
2026-01-21 00:25:25 +08:00
Gao Han 9c7de5373a [Feature]Basic version of supporting streaming output (#367)
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
2026-01-01 03:02:04 +13:00
WeiQing ChenandHongsheng Liu 1753a878e0 [Bugfix] t2i online fix cache configuration failure and errors in /v1/images/generation endpoint. (#499)
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-30 16:50:30 +08:00
2f69748ee4 add openai create speech endpoint (#305)
Signed-off-by: Bhanu068 <voutharoja.bhanu06@gmail.com>
Signed-off-by: Bhanu Prakash Voutharoja <59905694+Bhanu068@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
2025-12-27 13:56:51 +08:00
595e7c0032 [Core] Supports stage abstraction in the diffusion model (#391)
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
2025-12-27 01:59:33 +13:00
Wonjae Leeandwonjae.lee0 341457a121 [Feature] Support standard OpenAI API sampling parameters for thinker stage. (max_tokens...) (#450)
Signed-off-by: wonjae.lee0 <wonjae.lee0@navercorp.com>
Co-authored-by: wonjae.lee0 <wonjae.lee0@navercorp.com>
2025-12-24 20:39:01 +08:00
Peiqi Yin 0e878a92d2 [BugFix] raise error when request.stream=True. (#350)
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
2025-12-23 08:52:18 +08:00
WeiQing Chen 628c0a9217 [Entrypoints] Support Qwen-Image-Edit 2509 online serving & update doc (#368)
Signed-off-by: David Chen <530634352@qq.com>
2025-12-19 16:50:32 +08:00
8939eb835d [Rebase] Rebase to vllm 0.12.0 (#335)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
2025-12-19 16:42:02 +08:00
Chenguang Zhengandyinpe acf66ee2cc [Entrypoints] Support Online Serving for Diffusion-only Models (#259)
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
2025-12-17 17:40:45 +08:00
Gao Han 8722c6374c [Feature] Support output modalities control per request (#298)
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
2025-12-17 17:19:56 +08:00
Cyrus Leung c091a03d82 [Chore] Bump Python version and apply formatting changes for v0.12 release (#126)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
2025-12-16 15:19:40 +08:00
R2-YandRein 0a4e5a8710 [Feature] add support for Qwen3-omni (#55)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein <rein@Reins-MacBook-Air.local>
2025-11-30 04:44:01 +08:00
Gao Han 598b4ddb51 [Feature] support multimodal inputs with multiple requests (#76)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
2025-11-24 16:02:44 +08:00
Jiangyun Zhu 5707fc78d5 [CI] fix ci and format existing code (#71)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
2025-11-18 20:54:26 +08:00
Gao Han c5a89302f1 [Feature] Support online inference (#64)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
2025-11-17 14:44:10 +08:00