anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
980 Commits 235 Branches 31 Tags
b8f736c5a9b18eea9b1bfdb982397d8abb2363b3
Commit Graph
627 Commits
Author SHA1 Message Date
tzhouam 0d23f68997 revert changes in qwen 3 tts 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-20 02:11:10 +00:00
tzhouam 3c52432100 Merge branch 'main' into dev/vllm-align 2026-03-20 01:38:43 +00:00
Yuanheng Zhao 6ac7ba692d [Feat] Support T5 Tensor Parallelism (#1881) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-03-20 08:49:01 +08:00
Chen-Yo Sun e9fe78e28c [Model] Add Voxtral TTS model (#1803) 
Signed-off-by: Chen-Yo Sun <chenyo.sun@mistral.ai>
 2026-03-19 16:02:11 -04:00
JuboYuAliciaWeiQing Chen
 		b766c4702c Int8 Quantization Support for DiT (Z-Image & Qwen-Image) (#1470) 
Signed-off-by: juboyu <767868009@qq.com>
Signed-off-by: JuboYu <767868009@qq.com>
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-19 22:58:29 +08:00
Hui.andAlvisGong 99cc76690d [Bugfix]Fix bug of online server can not return mutli images (#2007) 
Signed-off-by: Hui <1779066624@qq.com>
Co-authored-by: AlvisGong <gwly0401@163.com>
 2026-03-19 03:57:04 -07:00
tzhouam 367ce6328c Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align 2026-03-19 09:38:17 +00:00
rein yang 60a8c01a96 [Bugfix] revert PR#1758 which introduced the accuracy problem of qwen3-omni (#2009) 
Signed-off-by: Rein Yang <ruiruyang2@gmail.com>
 2026-03-19 17:34:01 +08:00
Ziming Huang 89fff09628 [Fix] Fix slow hasattr in CUDAGraphWrapper.__getattr__ (#1982) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-03-19 17:10:08 +08:00
Chendi.XueandHongsheng Liu ba5a66b15b [XPU] update bagel modeling to remove cuda hardcode, add xpu stage_config (#1931) 
Signed-off-by: Chendi Xue <chendi.xue@intel.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-19 16:55:16 +08:00
rongfu.leng f9268822cd [Bugfix] fix helios video generate use cpu device (#1915) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-19 16:51:45 +08:00
Dnoob 48b3969717 [Bugfix] Fix Ovis Image crash when guidance_scale is set without negative_prompt (#1956) 
Signed-off-by: Dnoob <dxpouo@gmail.com>
 2026-03-19 16:51:02 +08:00
tzhouam 37e055ed9e Merge remote-tracking branch 'remotes/origin/main' into dev/vllm-align 2026-03-19 08:49:35 +00:00
tzhouam f83df010ef Remove redundant assignment of rope_theta in Qwen3OmniMoeTalker initialization for cleaner configuration setup. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 08:35:56 +00:00
SamitandHongsheng Liu 5699fe7a96 [Bugfix] Fix config misalignment between offline and online diffusion inference (Wan2.2, Qwen-Image series) (#1979) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-19 16:00:26 +08:00
Dom Brown 31c8cb22cf [Perf] [Qwen3-TTS] Keep audio_codes and last_talker_hidden on GPU to eliminate per-step sync stalls (#1985) 
Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com>
 2026-03-19 13:50:26 +08:00
tzhouam 22b6703773 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-19 05:16:40 +00:00
tzhouam 034d579d77 Update vLLM precompiled wheel version and refactor various components for improved functionality and error handling. Changes include modifying the get_name method in DiffusionQuantizationConfig to use class-level attributes, enhancing metadata handling in StageEngineCoreClient, and adjusting the AsyncOmni class constructor for better parameter management. Additionally, improved error handling in omni_snapshot_download and optimized initial chunk size computation in talker2code2wav_async_chunk for dynamic load adaptation. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 05:14:34 +00:00
dengyunyangbjf-frzDidan Deng
 		fde4808723 [Performance] diffusion timing (#1757) 
Signed-off-by: dengyunyang <584797741@qq.com>
Signed-off-by: bjf-frz <frz123db@gmail.com>
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
Co-authored-by: bjf-frz <frz123db@gmail.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-19 12:26:12 +08:00
tzhouam 4f75cccacc Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-19 03:11:40 +00:00
Joshna-Medisettyandgcanlin ca6c7ad2be Fix OmniGen2 transformer config loading for HF models (#1934) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-19 10:21:54 +08:00
tzhouam 342a209924 Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-19 02:06:26 +00:00
TJian c85acb1773 [CI] [ROCm] Bugfix device environment issue (#1984) 
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
 2026-03-19 05:11:28 +08:00
Juan Pablo ZuluagapabloHongsheng Liu
 		fc5e8f825e [Optim][Qwen3TTS] big boost model throughput+latency high concurrency (#1852) 
Signed-off-by: pablo <pablo@agigo.ai>
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
Co-authored-by: pablo <pablo@agigo.ai>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-18 12:22:05 -04:00
Taichang Zhou dc6f2fb81f Enhance cu_seqlens computation in Qwen3Omni_VisionTransformer by switching to tensor operations for improved performance and compatibility. Added error handling to fallback on a vectorized implementation if necessary. This update aligns with upstream changes and optimizes the handling of sequence lengths during attention execution. 2026-03-18 23:27:55 +08:00
Taichang Zhou 7a68bd37b7 Refactor cu_seqlens handling in Qwen3Omni_VisionTransformer to align with upstream vLLM changes. The sequence length helper now retains cu_seqlens in numpy format and converts it for attention execution, improving compatibility and performance. 2026-03-18 23:23:41 +08:00
SYLAR 53ebdd0b1d [Bug]: fix CUDA OOM during diffusion post-processing (#1670) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-03-18 14:59:58 +00:00
汪志鹏 61e170c69a [Feature]: Remove some useless hf_overrides in yaml (#1898) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-18 19:46:44 +08:00
Y. FisherandCanlin Guo f52b51533a [Feature] Add cache-dit support for HunyuanImage3 (#1848) 
Signed-off-by: KexiongYu <yukexiong1@huawei.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
 2026-03-18 10:18:27 +00:00
Canlin Guo d636c32ff9 [Bugfix] Set PREEMPTED status when moving requests from running to waiting queue (#1893) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-18 17:56:13 +08:00
Chenguang ZhengyinpePeiqi YinwuhangPeiqi Yinlinyueqianprincepridelishunyang
 		3158e9d28f [Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring (#1908) 
Signed-off-by: yinpe <11810305@mail.sustech.edu.cn>
Signed-off-by: Peiqi Yin <yinpeiqi809@gmail.com>
Signed-off-by: yinpeiqi <yinpeiqi809@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
Signed-off-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
Co-authored-by: Peiqi Yin <yinpeiqi809@gmail.com>
Co-authored-by: wuhang <whlbx@hotmail.com>
Co-authored-by: Peiqi Yin <60515999+yinpeiqi@users.noreply.github.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-03-18 17:33:53 +08:00
Yueqian Lin fe786f1a19 Fix Base voice clone streaming quality and stop-token crash (#1945) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-18 06:54:12 +08:00
Taichang Zhou 8a31e7865d Increase timeout for Diffusion Model Test and Diffusion Images API LoRA E2E in Buildkite configurations to 30 minutes. Also, introduce a new timeout constant for diffusion initialization in relevant test files to enhance stability during testing. Update OmniServer initialization to utilize the new timeout settings. 2026-03-17 23:43:46 +08:00
tzhouam 004c60a031 Refactor shutdown and cleanup logic in OmniRunner and OmniStage to improve process termination and resource management. The changes ensure that stage workers are stopped before closing queues, preventing potential orphaned subprocesses. Additionally, enhanced error handling during process termination provides clearer logging for failed shutdown attempts. This update aims to increase the reliability and stability of the shutdown process across the application. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 09:33:28 +00:00
tzhouam cc4ade4ba7 Refactor shutdown logic in OmniLLM and stage worker to ensure proper engine closure. The shutdown method now checks for the presence of engine_core before invoking shutdown, enhancing resource management. Additionally, the stage worker now explicitly calls stage_engine.close() upon receiving a shutdown signal. This improves the overall stability and reliability of the shutdown process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 08:19:56 +00:00
tzhouam 722ad6bbae rebase: align vllm-omni with vLLM 0a0a1a198be8 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 03:26:42 +00:00
tzhouam 0eccb77c4f Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-17 02:48:50 +00:00
Juan Pablo Zuluaga cf328eb8e8 [Bug][Qwen3TTS][Streaming] remove dynamic initial chunk and only compute on initial request (#1930) 
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
 2026-03-16 15:23:09 -04:00
rongfu.leng 4036cd547c [Misc] removed qwen3_tts.py as it is out-dated (#1926) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-16 13:09:04 -04:00
tzhouam de6bcc734c Update Dockerfile.ci to use a new vllm wheel URL and refactor MooncakeConnector import logic in monkey_patch.py for improved compatibility. Adjust OpenAIServingChat to call the harmony request method from the correct context. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 08:33:28 +00:00
tzhouam a26e568dfc Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 07:30:43 +00:00
Canlin Guo 9718a9e369 [NPU] Upgrade to v0.17.0 (#1890) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-16 14:42:59 +08:00
tzhouam 9c22e3afb0 Refactor import statement for OpenAIServingEmbedding in api_server.py to use aliasing for clarity 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 06:13:07 +00:00
Jinheng 88caaf182b [Feature] Split #1303 Part 1: PD disaggregation scaffolding (#1863) 
Signed-off-by: Jinheng Li <ahengljh@gmail.com>
 2026-03-16 14:05:19 +08:00
tzhouam aae101f495 Refactor import statement for OpenAIServingEmbedding in api_server.py 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 03:39:51 +00:00
tzhouam 3a508d7b7a rebase: align vllm-omni with vLLM e9163b536e72 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 02:32:39 +00:00
tzhouam f137a1e3ad Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 01:49:46 +00:00
Lancer 9aae75c3ee [Feat] support HSDP for Flux family (#1900) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-15 22:48:12 +08:00
Lancer 2472c360aa [Feat] support SP for FLUX.2-klein (#1250) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-15 12:18:12 +08:00
Lancer af012babf8 [Bugfix] Fix cpu offload and quantization compatibility (#1473) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-14 18:55:30 +00:00