tzhouam
63fd163efb
Refactor import statement in OpenAI API server for improved module organization
...
- Updated the import path for AnthropicServingMessages in api_server.py to enhance clarity and maintain consistency with the new module structure.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-01-29 08:23:17 +00:00
tzhouam
478c036a01
[Refactor] Update OpenAI API server and related modules for improved structure and functionality
...
- Refactored import statements in api_server.py to enhance module organization, including changes to the routing and server setup.
- Updated the method for removing routes to ensure compatibility with the new app structure.
- Adjusted the handling of chat completion requests to improve error responses and streamline processing.
- Modified serving modules to align with the new import paths and enhance code clarity.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
2026-01-29 15:28:51 +08:00
Yueqian Lin
9bca8880fe
Fix TTS speaker typo and add supported languages ( #990 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-01-27 23:19:51 +08:00
Yueqian Lin
77ff875170
Support Qwen3 tts online serving ( #968 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
2026-01-27 14:37:21 +08:00
5037af1033
[Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment ( #758 )
...
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk >
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk >
Signed-off-by: dongbo910220 <1275604947@qq.com >
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: dongbo910220 <1275604947@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2026-01-27 10:22:40 +08:00
Samit and Hongsheng Liu
978a290d6e
[Bugfix] Fix diffusion pipeline CFG (guidance_scale parsing failure bug) ( #922 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 05:09:03 +00:00
52bf1e2f40
[Feature]Support async computation and communication across stages by chunks ( #727 )
...
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com >
Signed-off-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: Rein Yang <ruiruyang2@gmail.com >
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-26 11:21:24 +08:00
5d9918fa67
[Misc] Fix error log for the diffusion stage timeout ( #915 )
...
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: Samit <285365963@qq.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
2026-01-23 15:16:40 +08:00
汪志鹏 and wzliu
7f821be57b
Support Bagel Model ( #726 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Co-authored-by: wzliu <wzliu@connect.hku.hk >
2026-01-22 23:09:55 +13:00
Sihao Li
0aa72b952f
[bugfix] support text + audio mixed output ( #843 )
...
Signed-off-by: GG-li <3226868735@qq.com >
2026-01-21 17:44:17 +08:00
Chenguang Zheng
72a8553310
[Bugfix] Remove the duplicate api registration in vllm-omni ( #880 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
2026-01-21 07:00:31 +00:00
+13
776c3a794e
Dev/rebase 0.14.0 ( #847 )
...
Signed-off-by: JaredforReal <w13431838023@gmail.com >
Signed-off-by: root <root@hk01dgx039.cm.cluster >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: TangPeng <85704592@qq.com >
Signed-off-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com >
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com >
Signed-off-by: samithuang <285365963@qq.com >
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: David Chen <530634352@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Signed-off-by: wangyu31577 <wangyu31577@hundsun.com >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: John Liu BUAA <liukecheng97@gmail.com >
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
Signed-off-by: root <root@hk01dgx028.cm.cluster >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Co-authored-by: JaredforReal <w13431838023@gmail.com >
Co-authored-by: root <root@hk01dgx039.cm.cluster >
Co-authored-by: JustQJ <37905360+JustQJ@users.noreply.github.com >
Co-authored-by: XU Mingshi <91017482+mxuax@users.noreply.github.com >
Co-authored-by: Sihyeon Jang <uneedsihyeon@gmail.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: catcat <108673086+iwzbi@users.noreply.github.com >
Co-authored-by: Ziming Huang <hzm414167@alibaba-inc.com >
Co-authored-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com >
Co-authored-by: Samit <285365963@qq.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com >
Co-authored-by: wangyu31577 <wangyu31577@hundsun.com >
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: John Liu BUAA <liukecheng97@gmail.com >
Co-authored-by: D!NE$H <67671800+gDINESH13@users.noreply.github.com >
Co-authored-by: root <root@hk01dgx028.cm.cluster >
2026-01-21 00:25:25 +08:00
Zeyu Huang | 黃澤宇
430c60b9b5
[Bugfix] Diffusion model fails to load when stage config is present ( #860 )
...
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com >
2026-01-20 12:42:26 +08:00
SYLAR
a1c37cc6e3
[Core] Implement Diffusion Profiler Support ( #709 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
2026-01-20 11:35:48 +08:00
D!NE$H
5e7035e183
[Feature]: Add CFG param to online serving ( #824 )
...
Signed-off-by: Dinesh G <G.Dinesh@ibm.com >
Signed-off-by: gDINESH13 <dinesh13g@gmail.com >
2026-01-19 04:23:15 +00:00
Yuhan Liu
e213bdd0a4
[BugFix]Remove duplicate error handling for request results ( #781 )
...
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com >
2026-01-14 20:08:23 +08:00
Ziming Huang
1444e1fc45
[Feature] Support Qwen3 Omni talker mtp batch inference ( #722 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-14 19:07:31 +08:00
JustQJ
9cdf592782
[Misc] Enable tensor_parallel_size argument with online serving cmd ( #761 )
...
Signed-off-by: TangPeng <85704592@qq.com >
2026-01-13 15:55:25 +08:00
Chenguang Zheng
7e14c943d5
[Misc] Group omni arguments into OmniConfig section ( #744 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: ZEHNG Chenguang <645327136@qq.com >
Signed-off-by: ZHENG Chenguang <645327136@qq.com >
2026-01-12 23:20:20 +08:00
Prajwal A and zjy0516
779f5987d9
[feature] cpu offloading support for diffusion ( #497 )
...
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com >
Signed-off-by: Prajwal A <prajwalanagani@gmail.com >
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com >
2026-01-12 10:41:18 +00:00
Sihao Li and Hongsheng Liu
2c92b63cce
fix:#issue 432 ( #517 )
...
Signed-off-by: GG-li <3226868735@qq.com >
Signed-off-by: Sihao Li <111170255+GG-li@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-12 15:38:47 +08:00
Ziming Huang
978e3ad7d4
[Feature] Support Qwen3 Omni talker cudagraph ( #669 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-09 11:45:58 +08:00
Ziming Huang
cddcc400d3
[BugFix] Fix stage engine outputs mismatch bug in online batching ( #691 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-09 04:01:57 +13:00
Yuanheng Zhao
4c72b67444
[BugFix] Explicitly release file locks during stage worker init ( #703 )
...
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com >
2026-01-08 20:35:00 +08:00
Junhong Liu
bb13467adc
[Bug fix] fix e2e_total_tokens and e2e_total_time_ms ( #648 )
...
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
2026-01-08 20:25:04 +08:00
dsinghvi
8c1259383c
omni entrypoint support tokenizer arg ( #572 )
...
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com >
2026-01-08 15:44:14 +08:00
Yueqian Lin and Hongsheng Liu
0cb7ec62da
[Model] Add Stable Audio Open support for text-to-audio generation ( #331 )
...
Signed-off-by: linyueqian <linyueqian@outlook.com >
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-07 17:01:06 +08:00
Chenguang Zheng
aaba243832
[BUGFIX] Delete the CUDA context in the stage process. ( #661 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: Chenguang Zheng <645327136@qq.com >
2026-01-06 22:22:04 +08:00
汪志鹏
e7eeb54e08
[BugFix]Fix text_to_image bagel command error ( #644 )
...
Signed-off-by: princepride <wangzhipeng628@gmail.com >
2026-01-05 23:22:15 +13:00
Canlin Guo and lishunyang
52afcb6055
[Feature] Support torch profiler across omni stages ( #553 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: lishunyang <lishunyang12@163.com >
Co-authored-by: lishunyang <lishunyang12@163.com >
2026-01-05 17:32:20 +08:00
Ziming Huang
f9b8bfd7bb
[BugFix] AttributeError: 'AsyncOmniDiffusion' object has no attribute 'abort' ( #624 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
2026-01-05 14:53:10 +08:00
Gao Han and Hongsheng Liu
6b18d31aaa
[Bugfix] Support both list and generator mode & fix unnecessary error log ( #599 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-05 09:17:06 +08:00
catcat
7961eb6a02
[Bugfix] Fix on_finalize_request method arguments number mismatch ( #583 )
...
Signed-off-by: iwzbi <wzbi@zju.edu.cn >
2026-01-04 09:51:46 +08:00
Hongsheng Liu
d7062550d7
[Core] omni refactor: optimize stage initialization polling and troubleshooting logs ( #575 )
...
Signed-off-by: hsliu <liuhongsheng4@huawei.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-02 11:31:31 +08:00
Canlin Guo and Hongsheng Liu
4f8cb95571
[Metric] Fix the computation of e2e_total_tokens ( #519 )
...
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2026-01-01 00:13:17 +08:00
680742bc33
[Diffusion]: Diffusion Ring Attention support ( #273 )
...
Signed-off-by: mxuax <mxuax@connect.ust.hk >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-31 14:26:30 +00:00
Gao Han
9c7de5373a
[Feature]Basic version of supporting streaming output ( #367 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2026-01-01 03:02:04 +13:00
Ziming Huang and Hongsheng Liu
ad0224f2e6
[Feature] Support Omni serving abort request ( #486 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-31 16:14:03 +08:00
23bf317cc5
[New Model]Bagel model(Diffusion Only) ( #319 )
...
Signed-off-by: wzliu <wzliu@connect.hku.hk >
Signed-off-by: princepride <wangzhipeng628@gmail.com >
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com >
Co-authored-by: wzliu <wzliu@connect.hku.hk >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-31 13:47:35 +08:00
SYLAR and Hongsheng Liu
380c43eb86
[Feature] Add tqdm in Omni pipeline to align with Vllm ( #552 )
...
Signed-off-by: lishunyang <lishunyang12@163.com >
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-31 04:44:11 +08:00
Peiqi Yin and Hongsheng Liu
2eebd22e32
[BugFix] __del__ exit don't join the spawned process ( #555 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-31 03:59:25 +08:00
Gao Han
194fa159a3
[Bugfix]fix proces close problem for omni class ( #554 )
...
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk >
2025-12-31 08:23:54 +13:00
f0b8d3db82
[Feature] Control the stage init timeout threshold by --stage-init-timeout ( #393 )
...
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk >
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk >
Signed-off-by: tzhouam <tzhouam@connect.ust.hk >
Signed-off-by: gcanlin <canlinguosdu@gmail.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Li Shunyang <gmsv2544@nus.edu.sg >
Signed-off-by: Li Shunyang <lishunyang12@163.com >
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Signed-off-by: dengyunyang <584797741@qq.com >
Signed-off-by: Samit <285365963@qq.com >
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com >
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com >
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: dengyunyang <584797741@qq.com >
Co-authored-by: Samit <285365963@qq.com >
2025-12-30 23:37:10 +08:00
WeiQing Chen and Hongsheng Liu
1753a878e0
[Bugfix] t2i online fix cache configuration failure and errors in /v1/images/generation endpoint. ( #499 )
...
Signed-off-by: David Chen <530634352@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-30 16:50:30 +08:00
ebe37e9f57
[Refactor] let the AsyncOmni class inherit from Omni class ( #511 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com >
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-29 21:04:17 +08:00
Peiqi Yin and Hongsheng Liu
60143adc52
[Core] remove deparated code from PR391 ( #502 )
...
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-28 21:11:37 +13:00
2f69748ee4
add openai create speech endpoint ( #305 )
...
Signed-off-by: Bhanu068 <voutharoja.bhanu06@gmail.com >
Signed-off-by: Bhanu Prakash Voutharoja <59905694+Bhanu068@users.noreply.github.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com >
2025-12-27 13:56:51 +08:00
Ziming Huang and Hongsheng Liu
d456721d81
[BugFix] Make max_batch_size work in Omni online serving ( #487 )
...
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-26 21:51:07 +08:00
595e7c0032
[Core] Supports stage abstraction in the diffusion model ( #391 )
...
Signed-off-by: Chenguang ZHENG <645327136@qq.com >
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com >
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com >
2025-12-27 01:59:33 +13:00
Junhong Liu
51eefbfebc
[BugFix] initial sampling_params_list when offline inference ( #468 )
...
Signed-off-by: Junhong <liujunhong11@huawei.com >
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com >
Signed-off-by: Junhong Liu <ljh_lbj@163.com >
2025-12-26 13:54:12 +08:00