anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
941 Commits 235 Branches 31 Tags
dc6f2fb81f7e55a53727360bfbf1a3801bf6b5e3
Commit Graph
941 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Taichang Zhou dc6f2fb81f Enhance cu_seqlens computation in Qwen3Omni_VisionTransformer by switching to tensor operations for improved performance and compatibility. Added error handling to fallback on a vectorized implementation if necessary. This update aligns with upstream changes and optimizes the handling of sequence lengths during attention execution. 2026-03-18 23:27:55 +08:00
Taichang Zhou 7a68bd37b7 Refactor cu_seqlens handling in Qwen3Omni_VisionTransformer to align with upstream vLLM changes. The sequence length helper now retains cu_seqlens in numpy format and converts it for attention execution, improving compatibility and performance. 2026-03-18 23:23:41 +08:00
Taichang Zhou 8a31e7865d Increase timeout for Diffusion Model Test and Diffusion Images API LoRA E2E in Buildkite configurations to 30 minutes. Also, introduce a new timeout constant for diffusion initialization in relevant test files to enhance stability during testing. Update OmniServer initialization to utilize the new timeout settings. 2026-03-17 23:43:46 +08:00
tzhouam 004c60a031 Refactor shutdown and cleanup logic in OmniRunner and OmniStage to improve process termination and resource management. The changes ensure that stage workers are stopped before closing queues, preventing potential orphaned subprocesses. Additionally, enhanced error handling during process termination provides clearer logging for failed shutdown attempts. This update aims to increase the reliability and stability of the shutdown process across the application. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 09:33:28 +00:00
tzhouam cc4ade4ba7 Refactor shutdown logic in OmniLLM and stage worker to ensure proper engine closure. The shutdown method now checks for the presence of engine_core before invoking shutdown, enhancing resource management. Additionally, the stage worker now explicitly calls stage_engine.close() upon receiving a shutdown signal. This improves the overall stability and reliability of the shutdown process. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 08:19:56 +00:00
tzhouam 35b3dd638a Refactor Dockerfile.ci to install vLLM from a dynamically retrieved precompiled wheel URL, improving the installation process by ensuring compatibility with PEP 440 local version identifiers. This change enhances the reliability of vLLM installations. Signed-off-by: tzhouam <tzhouam@connect.ust.hk> 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 06:08:59 +00:00
tzhouam 83effe39ac Update Dockerfile.ci to include an additional extra index URL for pip installation, enhancing package resolution for vLLM. This change ensures compatibility with both precompiled wheels and packages from PyPI. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 04:07:50 +00:00
tzhouam b7a36d16d4 Merge branch 'dev/vllm-align' of https://github.com/vllm-project/vllm-omni into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 03:54:31 +00:00
tzhouam 722ad6bbae rebase: align vllm-omni with vLLM 0a0a1a198be8 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 03:26:42 +00:00
Zhou Taichang ac937b82e6 Refactor Dockerfile to install vLLM from source 
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-17 11:08:45 +08:00
tzhouam 0eccb77c4f Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-17 02:48:50 +00:00
tzhouam b0e36f9f5d Update Dockerfile.ci to force reinstall vLLM from a precompiled wheel, ensuring the latest version is used. This change enhances the installation process by guaranteeing that any existing installations are replaced. Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk> 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 02:04:51 +00:00
tzhouam 83b05ce3e4 Refactor Dockerfile.ci to install vLLM using a precompiled wheel from a specified commit, replacing the previous source installation method. This update simplifies the installation process and ensures correct dependency management. Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk> 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 01:35:12 +00:00
Juan Pablo Zuluaga cf328eb8e8 [Bug][Qwen3TTS][Streaming] remove dynamic initial chunk and only compute on initial request (#1930) 
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
 2026-03-16 15:23:09 -04:00
rongfu.leng 4036cd547c [Misc] removed qwen3_tts.py as it is out-dated (#1926) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-16 13:09:04 -04:00
Zhou Taichang 22a2bd3406 Upgrade flashinfer-cubin and flashinfer-jit-cache versions 
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-16 19:31:03 +08:00
tzhouam 8e81e60d32 Refactor Dockerfile.ci to install vLLM from source using precompiled artifacts at a specified commit. This change replaces the previous wheel installation method and ensures the correct versioning of dependencies. Signed-off-by: tzhouam <tzhouam@connect.ust.hk> 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 10:01:18 +00:00
tzhouam 57ac7eb155 rebase: align vllm-omni with vLLM 2754231ba3a7 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 09:28:56 +00:00
tzhouam de6bcc734c Update Dockerfile.ci to use a new vllm wheel URL and refactor MooncakeConnector import logic in monkey_patch.py for improved compatibility. Adjust OpenAIServingChat to call the harmony request method from the correct context. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 08:33:28 +00:00
tzhouam a26e568dfc Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 07:30:43 +00:00
Canlin Guo 9718a9e369 [NPU] Upgrade to v0.17.0 (#1890) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-16 14:42:59 +08:00
tzhouam 9c22e3afb0 Refactor import statement for OpenAIServingEmbedding in api_server.py to use aliasing for clarity 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 06:13:07 +00:00
Jinheng 88caaf182b [Feature] Split #1303 Part 1: PD disaggregation scaffolding (#1863) 
Signed-off-by: Jinheng Li <ahengljh@gmail.com>
 2026-03-16 14:05:19 +08:00
Roger Wang 6a5fa588c6 Update latest news section in README.md (#1909) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-03-15 22:50:42 -07:00
Roger WangCanlin GuoIsotr0pyWeiQing ChenZhou TaichangSamitJiangyun ZhuYueqian Lin汪志鹏
 		1deb4fdf92 Add Governance section (#1889) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-15 22:47:42 -07:00
tzhouam aae101f495 Refactor import statement for OpenAIServingEmbedding in api_server.py 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 03:39:51 +00:00
tzhouam 3a508d7b7a rebase: align vllm-omni with vLLM e9163b536e72 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 02:32:39 +00:00
tzhouam f137a1e3ad Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 01:49:46 +00:00
Lancer 9aae75c3ee [Feat] support HSDP for Flux family (#1900) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-15 22:48:12 +08:00
汪志鹏 284575ab82 [CI]: Add/Fix bagel e2e online/offline test (#1895) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-15 18:27:07 +08:00
Lancer 2472c360aa [Feat] support SP for FLUX.2-klein (#1250) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-15 12:18:12 +08:00
Lancer af012babf8 [Bugfix] Fix cpu offload and quantization compatibility (#1473) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-14 18:55:30 +00:00
Wu JIAZHEN 4de0c8d41b [skip CI][Docs][Benchmark]: clarify vbench parameter behavior and add t2v example (#1497) 
Signed-off-by: asuka <1311722138@qq.com>
 2026-03-14 23:31:18 +08:00
Gao Han 365dfd5f8e [CI pipeline] Re-enable Diffusion Tensor Parallelism Test in pipeline (#1892) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-03-14 22:56:43 +08:00
SYLARGao HanHongsheng Liu
 		5377619c0b fix: propagate parallel_config through create_default_diffusion (#1878) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-14 17:07:13 +08:00
zhou zhuoxinprinceprideDing Zuhao
 		dc72aede40 [CI] add multimodal processing correctness tests for Omni models (#1445) 
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Ding Zuhao <e1583181@u.nus.edu>
 2026-03-14 14:15:25 +08:00
Daniel Huang c107f0b336 Buildkite hardware ci xpu test (#1340) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-14 11:24:05 +08:00
Alex Brooks e17130090c [BugFix] Fix Max Rank Handling in LoRA (#1397) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-03-14 02:42:54 +00:00
tzhouam 97899c6c98 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-13 19:03:57 +00:00
Yueqian Lin cebf78ba3e [Bugfix] Restore voice upload API and profiler endpoints reverted by #1719 (#1879) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-14 04:37:08 +13:00
dengyunyang 3e8cb6c631 [Diffusion] [Model] Dreamid omni (#1855) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-03-13 21:53:06 +08:00
Ian EavesSamitGao Han
 		92f771d455 [Frontend] Rewrite video API for async job lifecycle (#1665) 
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-03-13 17:11:06 +08:00
汪志鹏 e0b71056ba [BugFix]: Fix bagel online inference bug (#1804) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-13 08:58:44 +00:00
Gao Han 26b3ddee47 [CI failed] Disable Diffusion Tensor Parallelism Test (#1876) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-03-13 16:56:41 +08:00
Lancer 53a3df7309 [Bugfix] fix layer-wise offload incompatible with cache-dit (#1786) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-13 15:28:13 +08:00
SYLAR 75056b2db2 [Bugfix] Fix SD3.5-medium attn2 uninitialized weights (#1659) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-03-13 14:58:17 +08:00
Gao Han 717dbee5a6 [CI failed] Disable test for zimage tensor parallelism (#1870) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-03-13 14:52:48 +08:00
tzhouam d3db476f2d Merge branch 'main' into dev/vllm-align 2026-03-13 06:51:22 +00:00
tzhouam 62c8695dfd Update GPUGenerationModelRunner to clarify KV connector metadata handling during draft model runs 
- Changed variable name to better reflect its purpose in managing KV connector metadata.
- Updated comments for improved clarity regarding the handling of speculative configurations.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 03:16:33 +00:00
Alex BrooksandDidan Deng c8cf8a799d [Refactor] Use SP Plan for LongCat Sequence Parallelism (#1772) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-13 11:13:34 +08:00