anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
980 Commits 235 Branches 31 Tags
b8f736c5a9b18eea9b1bfdb982397d8abb2363b3
Commit Graph
980 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam b8f736c5a9 Remove obsolete image and YAML configuration files for Qwen2.5 and Qwen3 models from the e2e stage configurations. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-20 02:11:18 +00:00
tzhouam 0d23f68997 revert changes in qwen 3 tts 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-20 02:11:10 +00:00
tzhouam 3c52432100 Merge branch 'main' into dev/vllm-align 2026-03-20 01:38:43 +00:00
Yuanheng Zhao 6ac7ba692d [Feat] Support T5 Tensor Parallelism (#1881) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-03-20 08:49:01 +08:00
Chen-Yo Sun e9fe78e28c [Model] Add Voxtral TTS model (#1803) 
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai>
 2026-03-19 16:02:11 -04:00
JuboYuAliciaWeiQing Chen
 		b766c4702c Int8 Quantization Support for DiT (Z-Image & Qwen-Image) (#1470) 
Signed-off-by: juboyu <767868009@qq.com>
Signed-off-by: JuboYu <767868009@qq.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-19 22:58:29 +08:00
TJian 0239057cba [CI] [ROCm] Setup test-ready.yml and test-merge.yml (#2017) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-03-19 14:26:31 +00:00
Hui.andAlvisGong 99cc76690d [Bugfix]Fix bug of online server can not return mutli images (#2007) 
Signed-off-by: Hui <1779066624@qq.com>
Co-authored-by: AlvisGong <gwly0401@163.com>
 2026-03-19 03:57:04 -07:00
tzhouam 367ce6328c Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align 2026-03-19 09:38:17 +00:00
rein yang 60a8c01a96 [Bugfix] revert PR#1758 which introduced the accuracy problem of qwen3-omni (#2009) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-03-19 17:34:01 +08:00
Ziming Huang 89fff09628 [Fix] Fix slow hasattr in CUDAGraphWrapper.__getattr__ (#1982) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-03-19 17:10:08 +08:00
Chendi.XueandHongsheng Liu ba5a66b15b [XPU] update bagel modeling to remove cuda hardcode, add xpu stage_config (#1931) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-19 16:55:16 +08:00
rongfu.leng f9268822cd [Bugfix] fix helios video generate use cpu device (#1915) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-19 16:51:45 +08:00
Dnoob 48b3969717 [Bugfix] Fix Ovis Image crash when guidance_scale is set without negative_prompt (#1956) 
Signed-off-by: Dnoob <dxpouo@gmail.com>
 2026-03-19 16:51:02 +08:00
tzhouam 37e055ed9e Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align 2026-03-19 08:49:35 +00:00
tzhouam f83df010ef Remove redundant assignment of rope_theta in Qwen3OmniMoeTalker initialization for cleaner configuration setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 08:35:56 +00:00
Kevin H. Luu b1ceaa7b34 Add HF token to H100 jobs (#2008) 
Signed-off-by: khluu <khluu000@gmail.com>
 2026-03-19 01:25:14 -07:00
SamitandHongsheng Liu 5699fe7a96 [Bugfix] Fix config misalignment between offline and online diffusion inference (Wan2.2, Qwen-Image series) (#1979) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-19 16:00:26 +08:00
汪志鹏 81793f1918 [CI] Split BAGEL tests into dummy/real weight tiers (L2/L3) (#1998) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-19 15:32:48 +08:00
Dom Brown 31c8cb22cf [Perf] [Qwen3-TTS] Keep audio_codes and last_talker_hidden on GPU to eliminate per-step sync stalls (#1985) 
Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com>
 2026-03-19 13:50:26 +08:00
tzhouam 22b6703773 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-19 05:16:40 +00:00
tzhouam 034d579d77 Update vLLM precompiled wheel version and refactor various components for improved functionality and error handling. Changes include modifying the get_name method in DiffusionQuantizationConfig to use class-level attributes, enhancing metadata handling in StageEngineCoreClient, and adjusting the AsyncOmni class constructor for better parameter management. Additionally, improved error handling in omni_snapshot_download and optimized initial chunk size computation in talker2code2wav_async_chunk for dynamic load adaptation. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 05:14:34 +00:00
dengyunyangbjf-frzDidan Deng
 		fde4808723 [Performance] diffusion timing (#1757) 
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-19 12:26:12 +08:00
NumberWan ae92ef5f00 [Test] L4 complete diffusion feature test for Bagel models (#1938) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-03-19 11:53:20 +08:00
tzhouam 4f75cccacc Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-19 03:11:40 +00:00
Joshna-Medisettyandgcanlin ca6c7ad2be Fix OmniGen2 transformer config loading for HF models (#1934) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-19 10:21:54 +08:00
tzhouam 342a209924 Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 02:06:26 +00:00
Chendi.XueandHongsheng Liu e1ac1826cd [CI]init intel ci dispatch in buildkite folder (#1721) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-19 09:50:31 +08:00
TJian c85acb1773 [CI] [ROCm] Bugfix device environment issue (#1984) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-03-19 05:11:28 +08:00
Juan Pablo ZuluagapabloHongsheng Liu
 		fc5e8f825e [Optim][Qwen3TTS] big boost model throughput+latency high concurrency (#1852) 
Signed-off-by: pablo <pablo@agigo.ai>
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
Co-authored-by: pablo <pablo@agigo.ai>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-18 12:22:05 -04:00
Taichang Zhou dc6f2fb81f Enhance cu_seqlens computation in Qwen3Omni_VisionTransformer by switching to tensor operations for improved performance and compatibility. Added error handling to fallback on a vectorized implementation if necessary. This update aligns with upstream changes and optimizes the handling of sequence lengths during attention execution. 2026-03-18 23:27:55 +08:00
Taichang Zhou 7a68bd37b7 Refactor cu_seqlens handling in Qwen3Omni_VisionTransformer to align with upstream vLLM changes. The sequence length helper now retains cu_seqlens in numpy format and converts it for attention execution, improving compatibility and performance. 2026-03-18 23:23:41 +08:00
SYLAR 53ebdd0b1d [Bug]: fix CUDA OOM during diffusion post-processing (#1670) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-03-18 14:59:58 +00:00
Alicia 4a44f6ee5c [CI] Nightly Benchmark - Add an HTML generator, Update the EXCEL generator. (#1831) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-18 20:13:57 +08:00
汪志鹏 61e170c69a [Feature]: Remove some useless hf_overrides in yaml (#1898) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-18 19:46:44 +08:00
Y. FisherandCanlin Guo f52b51533a [Feature] Add cache-dit support for HunyuanImage3 (#1848) 
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-03-18 10:18:27 +00:00
Canlin Guo d636c32ff9 [Bugfix] Set PREEMPTED status when moving requests from running to waiting queue (#1893) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-18 17:56:13 +08:00
Chenguang ZhengyinpePeiqi YinwuhangPeiqi Yinlinyueqianprincepridelishunyang
 		3158e9d28f [Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring (#1908) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com>
Co-authored-by: wuhang <whlbx@hotmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-03-18 17:33:53 +08:00
WeiQing Chenanddavid6666666 6a66a1fe36 [skip ci][Docs] Update WeChat QR code (fix filename case) (#1976) 
Signed-off-by: david6666666 <david6666666@users.noreply.github.com>
Co-authored-by: david6666666 <david6666666@users.noreply.github.com>
 2026-03-18 15:17:27 +08:00
WeiQing Chenanddavid6666666 1a5d25a0ba [Docs] Update WeChat QR code for community support (#1974) 
Signed-off-by: david6666666 <david6666666@users.noreply.github.com>
Co-authored-by: david6666666 <david6666666@users.noreply.github.com>
 2026-03-18 13:55:51 +08:00
Yueqian Lin fe786f1a19 Fix Base voice clone streaming quality and stop-token crash (#1945) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-18 06:54:12 +08:00
Taichang Zhou 8a31e7865d Increase timeout for Diffusion Model Test and Diffusion Images API LoRA E2E in Buildkite configurations to 30 minutes. Also, introduce a new timeout constant for diffusion initialization in relevant test files to enhance stability during testing. Update OmniServer initialization to utilize the new timeout settings. 2026-03-17 23:43:46 +08:00
tzhouam 004c60a031 Refactor shutdown and cleanup logic in OmniRunner and OmniStage to improve process termination and resource management. The changes ensure that stage workers are stopped before closing queues, preventing potential orphaned subprocesses. Additionally, enhanced error handling during process termination provides clearer logging for failed shutdown attempts. This update aims to increase the reliability and stability of the shutdown process across the application. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 09:33:28 +00:00
tzhouam cc4ade4ba7 Refactor shutdown logic in OmniLLM and stage worker to ensure proper engine closure. The shutdown method now checks for the presence of engine_core before invoking shutdown, enhancing resource management. Additionally, the stage worker now explicitly calls stage_engine.close() upon receiving a shutdown signal. This improves the overall stability and reliability of the shutdown process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 08:19:56 +00:00
tzhouam 35b3dd638a Refactor Dockerfile.ci to install vLLM from a dynamically retrieved precompiled wheel URL, improving the installation process by ensuring compatibility with PEP 440 local version identifiers. This change enhances the reliability of vLLM installations. Signed-off-by: tzhouam <tzhouam@connect.ust.hk> 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 06:08:59 +00:00
tzhouam 83effe39ac Update Dockerfile.ci to include an additional extra index URL for pip installation, enhancing package resolution for vLLM. This change ensures compatibility with both precompiled wheels and packages from PyPI. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 04:07:50 +00:00
tzhouam b7a36d16d4 Merge branch 'dev/vllm-align' of https://github.com/vllm-project/vllm-omni into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 03:54:31 +00:00
tzhouam 722ad6bbae rebase: align vllm-omni with vLLM 0a0a1a198be8 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 03:26:42 +00:00
Zhou Taichang ac937b82e6 Refactor Dockerfile to install vLLM from source 
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-17 11:08:45 +08:00
tzhouam 0eccb77c4f Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-17 02:48:50 +00:00