tzhouam
|
004c60a031
|
Refactor shutdown and cleanup logic in OmniRunner and OmniStage to improve process termination and resource management. The changes ensure that stage workers are stopped before closing queues, preventing potential orphaned subprocesses. Additionally, enhanced error handling during process termination provides clearer logging for failed shutdown attempts. This update aims to increase the reliability and stability of the shutdown process across the application.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
|
2026-03-17 09:33:28 +00:00 |
|
Lancer
|
af012babf8
|
[Bugfix] Fix cpu offload and quantization compatibility (#1473)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
|
2026-03-14 18:55:30 +00:00 |
|
汪志鹏
|
e0b71056ba
|
[BugFix]: Fix bagel online inference bug (#1804)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
|
2026-03-13 08:58:44 +00:00 |
|
Yueqian Lin
|
e1efcb7560
|
Fix duplicate get_supported_tasks definition in async_omni.py (#1825)
Signed-off-by: linyueqian <linyueqian@outlook.com>
|
2026-03-11 19:43:37 -04:00 |
|
Yueqian Lin
|
a6b5519616
|
[Bugfix] Report supported tasks for omni models to skip unnecessary chat init (#1645)
Signed-off-by: linyueqian <linyueqian@outlook.com>
|
2026-03-12 01:06:20 +08:00 |
|
Ting FU
|
6ac85d4607
|
[Feat] Enable expert parallel for diffusion MoE layers (#1323)
Signed-off-by: Semmer2 <semmer@live.cn>
|
2026-03-11 10:45:36 +08:00 |
|
Samit
|
155856ff30
|
[Perf] Reduce IPC overhead for single-stage diffusion serving for Wan2.2 (#1715)
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
|
2026-03-10 00:03:51 +13:00 |
|
  
|
89b527e533
|
Dev/rebase v0170 (#1639)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
|
2026-03-09 17:29:38 +08:00 |
|
 knlnguyen1802andSamit
|
92e1a54419
|
[RL] Support collective rpc api to entrypoint && Support custom input output (#1646)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
|
2026-03-09 10:12:43 +08:00 |
|
Didan Deng
|
30191ed112
|
[Feature]: Add vae-patch-parallel CLI argument in online serving (#1716)
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
|
2026-03-07 01:45:02 +08:00 |
|
WeiQing Chen
|
83fe7d005b
|
[Model] support LTX-2 text-to-video image-to-video (#841)
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
|
2026-03-05 23:31:48 +08:00 |
|
Samit
|
22a51a7cdb
|
[Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading (#1504)
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
|
2026-03-02 14:53:25 +00:00 |
|
 Canlin GuoandHongsheng Liu
|
f53152d340
|
[Feature] Support HSDP for diffusion models (#1339)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-02-26 21:31:10 +08:00 |
|
 
|
03c01e0415
|
[Qwen3TTS][Feat] Streaming output (#1438)
Signed-off-by: pablo <pablo@agigo.ai>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: pablo <pablo@agigo.ai>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-02-26 21:02:35 +08:00 |
|
 
|
36b8f80266
|
[Feature] Support Stage Based Deployment CLI (#939)
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
2026-02-24 16:55:23 +08:00 |
|
 Sy03andHongsheng Liu
|
efbe4116ff
|
[Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161)
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-02-21 07:36:44 +08:00 |
|
knlnguyen1802
|
9994594782
|
[Misc] Support WorkerWrapperBase and CustomPipeline for Diffusion Worker (#764)
Signed-off-by: knlnguyen1802 <knlnguyen1802@gmail.com>
|
2026-02-12 19:23:51 +08:00 |
|
 Yuanheng ZhaoandSamit
|
8bf849c399
|
Refactor CPU Offloading Backend Pattern (#1223)
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
|
2026-02-12 17:29:55 +08:00 |
|
Junhong Liu
|
82fd2c3637
|
Fix yield token metrics and opt metrics record stats (#1292)
|
2026-02-12 00:52:05 +08:00 |
|
 Junhong LiuandHongsheng Liu
|
9af6fb9a1f
|
[Feature] Opt metrics structure (#891)
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-02-09 19:17:32 +08:00 |
|
 Junhong LiuandHongsheng Liu
|
959b3ca8c8
|
[Bugfix] E2E metric incorrect qwen3-omni with async chunk feature (#1018)
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-02-04 15:27:30 +08:00 |
|
 Zhou Taichangand汪志鹏
|
70ef548448
|
Dev/rebase v0.15.0 (#1159)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
|
2026-02-03 12:24:45 +08:00 |
|
 
|
2923d5a91e
|
[Feature] Support DiT Layerwise (Blockwise) CPU Offloading (#858)
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
2026-01-30 17:35:09 +08:00 |
|
 amy-why-3459andRein Yang
|
dd518973de
|
[BugFix] Fix the accuracy issue of multimodal input. (#1020)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
|
2026-01-29 19:30:11 +08:00 |
|
Samit
|
c0beebbfe3
|
[Feature] Support cache-dit for Wan 2.2 inference (#1021)
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
|
2026-01-28 20:07:50 +08:00 |
|
Zeyu Huang | 黃澤宇
|
d64bbde561
|
[Frontend][Model] Support batch request with refined OmniDiffusionReq… (#797)
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
|
2026-01-28 16:07:42 +08:00 |
|
 
|
5037af1033
|
[Feature] Diffusion LoRA Adapter Support (PEFT compatible) for vLLM alignment (#758)
Signed-off-by: AndyZhou952 <jzhoubc@connect.ust.hk>
Signed-off-by: Andy Zhou <jzhoubc@connect.ust.hk>
Signed-off-by: dongbo910220 <1275604947@qq.com>
Signed-off-by: Andy Zhou <46011930+AndyZhou952@users.noreply.github.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: dongbo910220 <1275604947@qq.com>
Co-authored-by: Samit <285365963@qq.com>
|
2026-01-27 10:22:40 +08:00 |
|
  
|
52bf1e2f40
|
[Feature]Support async computation and communication across stages by chunks (#727)
Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: CHEN <116010019@link.cuhk.edu.cn>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2026-01-26 11:21:24 +08:00 |
|
D!NE$H
|
5e7035e183
|
[Feature]: Add CFG param to online serving (#824)
Signed-off-by: Dinesh G <G.Dinesh@ibm.com>
Signed-off-by: gDINESH13 <dinesh13g@gmail.com>
|
2026-01-19 04:23:15 +00:00 |
|
Yuhan Liu
|
e213bdd0a4
|
[BugFix]Remove duplicate error handling for request results (#781)
Signed-off-by: Yuhan Liu <30294295+liuyuhanalex@users.noreply.github.com>
|
2026-01-14 20:08:23 +08:00 |
|
JustQJ
|
9cdf592782
|
[Misc] Enable tensor_parallel_size argument with online serving cmd (#761)
Signed-off-by: TangPeng <85704592@qq.com>
|
2026-01-13 15:55:25 +08:00 |
|
 Prajwal Aandzjy0516
|
779f5987d9
|
[feature] cpu offloading support for diffusion (#497)
Signed-off-by: zjy0516 <riverclouds.zhu@qq.com>
Signed-off-by: Prajwal A <prajwalanagani@gmail.com>
Co-authored-by: zjy0516 <riverclouds.zhu@qq.com>
|
2026-01-12 10:41:18 +00:00 |
|
Ziming Huang
|
cddcc400d3
|
[BugFix] Fix stage engine outputs mismatch bug in online batching (#691)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
|
2026-01-09 04:01:57 +13:00 |
|
Junhong Liu
|
bb13467adc
|
[Bug fix] fix e2e_total_tokens and e2e_total_time_ms (#648)
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
|
2026-01-08 20:25:04 +08:00 |
|
 Canlin Guoandlishunyang
|
52afcb6055
|
[Feature] Support torch profiler across omni stages (#553)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
|
2026-01-05 17:32:20 +08:00 |
|
catcat
|
7961eb6a02
|
[Bugfix] Fix on_finalize_request method arguments number mismatch (#583)
Signed-off-by: iwzbi <wzbi@zju.edu.cn>
|
2026-01-04 09:51:46 +08:00 |
|
Gao Han
|
9c7de5373a
|
[Feature]Basic version of supporting streaming output (#367)
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
|
2026-01-01 03:02:04 +13:00 |
|
 Ziming HuangandHongsheng Liu
|
ad0224f2e6
|
[Feature] Support Omni serving abort request (#486)
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2025-12-31 16:14:03 +08:00 |
|
 Peiqi YinandHongsheng Liu
|
2eebd22e32
|
[BugFix] __del__ exit don't join the spawned process (#555)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2025-12-31 03:59:25 +08:00 |
|
       
|
f0b8d3db82
|
[Feature] Control the stage init timeout threshold by --stage-init-timeout (#393)
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Li Shunyang <gmsv2544@nus.edu.sg>
Signed-off-by: Li Shunyang <lishunyang12@163.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: dengyunyang <584797741@qq.com>
Co-authored-by: Samit <285365963@qq.com>
|
2025-12-30 23:37:10 +08:00 |
|
 WeiQing ChenandHongsheng Liu
|
1753a878e0
|
[Bugfix] t2i online fix cache configuration failure and errors in /v1/images/generation endpoint. (#499)
Signed-off-by: David Chen <530634352@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2025-12-30 16:50:30 +08:00 |
|
 
|
ebe37e9f57
|
[Refactor] let the AsyncOmni class inherit from Omni class (#511)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2025-12-29 21:04:17 +08:00 |
|
 Peiqi YinandHongsheng Liu
|
60143adc52
|
[Core] remove deparated code from PR391 (#502)
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2025-12-28 21:11:37 +13:00 |
|
 
|
595e7c0032
|
[Core] Supports stage abstraction in the diffusion model (#391)
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: yinpeiqi <yinpeiqi809@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2025-12-27 01:59:33 +13:00 |
|
 dengyunyangandHongsheng Liu
|
a500ca1f28
|
[Feature] logger refactor (#466)
Signed-off-by: dengyunyang <584797741@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
|
2025-12-26 08:42:06 +08:00 |
|
 
|
8939eb835d
|
[Rebase] Rebase to vllm 0.12.0 (#335)
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Rein Yang <ruiruyang2@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
|
2025-12-19 16:42:02 +08:00 |
|
Gao Han
|
8722c6374c
|
[Feature] Support output modalities control per request (#298)
Signed-off-by: Gaohan123 <hgaoaf@connect.ust.hk>
|
2025-12-17 17:19:56 +08:00 |
|
TJian
|
f785a5d401
|
[Bugfix] [Example] Add fallback to worker_backend and ray_address in AsyncOmni to fix gradio demos (#337)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
|
2025-12-17 01:37:58 +00:00 |
|
dengyunyang
|
bd53347615
|
[Feature] Send response with request id (#301)
Signed-off-by: dengyunyang <584797741@qq.com>
|
2025-12-16 15:41:42 +00:00 |
|
Cyrus Leung
|
c091a03d82
|
[Chore] Bump Python version and apply formatting changes for v0.12 release (#126)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
|
2025-12-16 15:19:40 +08:00 |
|