anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
936 Commits 235 Branches 31 Tags
35b3dd638abe7d64fe1116e2c2ad293cc401a746
Commit Graph
936 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam 35b3dd638a Refactor Dockerfile.ci to install vLLM from a dynamically retrieved precompiled wheel URL, improving the installation process by ensuring compatibility with PEP 440 local version identifiers. This change enhances the reliability of vLLM installations. Signed-off-by: tzhouam <tzhouam@connect.ust.hk> 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 06:08:59 +00:00
tzhouam 83effe39ac Update Dockerfile.ci to include an additional extra index URL for pip installation, enhancing package resolution for vLLM. This change ensures compatibility with both precompiled wheels and packages from PyPI. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 04:07:50 +00:00
tzhouam b7a36d16d4 Merge branch 'dev/vllm-align' of https://github.com/vllm-project/vllm-omni into dev/vllm-align 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 03:54:31 +00:00
tzhouam 722ad6bbae rebase: align vllm-omni with vLLM 0a0a1a198be8 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 03:26:42 +00:00
Zhou Taichang ac937b82e6 Refactor Dockerfile to install vLLM from source 
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-17 11:08:45 +08:00
tzhouam 0eccb77c4f Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-17 02:48:50 +00:00
tzhouam b0e36f9f5d Update Dockerfile.ci to force reinstall vLLM from a precompiled wheel, ensuring the latest version is used. This change enhances the installation process by guaranteeing that any existing installations are replaced. Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk> 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 02:04:51 +00:00
tzhouam 83b05ce3e4 Refactor Dockerfile.ci to install vLLM using a precompiled wheel from a specified commit, replacing the previous source installation method. This update simplifies the installation process and ensures correct dependency management. Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk> 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-17 01:35:12 +00:00
Juan Pablo Zuluaga cf328eb8e8 [Bug][Qwen3TTS][Streaming] remove dynamic initial chunk and only compute on initial request (#1930) 
Signed-off-by: JuanPZuluaga <juanz9312@gmail.com>
 2026-03-16 15:23:09 -04:00
rongfu.leng 4036cd547c [Misc] removed qwen3_tts.py as it is out-dated (#1926) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-16 13:09:04 -04:00
Zhou Taichang 22a2bd3406 Upgrade flashinfer-cubin and flashinfer-jit-cache versions 
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-16 19:31:03 +08:00
tzhouam 8e81e60d32 Refactor Dockerfile.ci to install vLLM from source using precompiled artifacts at a specified commit. This change replaces the previous wheel installation method and ensures the correct versioning of dependencies. Signed-off-by: tzhouam <tzhouam@connect.ust.hk> 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 10:01:18 +00:00
tzhouam 57ac7eb155 rebase: align vllm-omni with vLLM 2754231ba3a7 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 09:28:56 +00:00
tzhouam de6bcc734c Update Dockerfile.ci to use a new vllm wheel URL and refactor MooncakeConnector import logic in monkey_patch.py for improved compatibility. Adjust OpenAIServingChat to call the harmony request method from the correct context. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 08:33:28 +00:00
tzhouam a26e568dfc Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 07:30:43 +00:00
Canlin Guo 9718a9e369 [NPU] Upgrade to v0.17.0 (#1890) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-16 14:42:59 +08:00
tzhouam 9c22e3afb0 Refactor import statement for OpenAIServingEmbedding in api_server.py to use aliasing for clarity 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 06:13:07 +00:00
Jinheng 88caaf182b [Feature] Split #1303 Part 1: PD disaggregation scaffolding (#1863) 
Signed-off-by: Jinheng Li <ahengljh@gmail.com>
 2026-03-16 14:05:19 +08:00
Roger Wang 6a5fa588c6 Update latest news section in README.md (#1909) 
Signed-off-by: Roger Wang <hey@rogerw.io>
 2026-03-15 22:50:42 -07:00
Roger WangCanlin GuoIsotr0pyWeiQing ChenZhou TaichangSamitJiangyun ZhuYueqian Lin汪志鹏
 		1deb4fdf92 Add Governance section (#1889) 
Signed-off-by: Roger Wang <hey@rogerw.io>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-15 22:47:42 -07:00
tzhouam aae101f495 Refactor import statement for OpenAIServingEmbedding in api_server.py 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 03:39:51 +00:00
tzhouam 3a508d7b7a rebase: align vllm-omni with vLLM e9163b536e72 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-16 02:32:39 +00:00
tzhouam f137a1e3ad Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-16 01:49:46 +00:00
Lancer 9aae75c3ee [Feat] support HSDP for Flux family (#1900) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-15 22:48:12 +08:00
汪志鹏 284575ab82 [CI]: Add/Fix bagel e2e online/offline test (#1895) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-15 18:27:07 +08:00
Lancer 2472c360aa [Feat] support SP for FLUX.2-klein (#1250) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-15 12:18:12 +08:00
Lancer af012babf8 [Bugfix] Fix cpu offload and quantization compatibility (#1473) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-14 18:55:30 +00:00
Wu JIAZHEN 4de0c8d41b [skip CI][Docs][Benchmark]: clarify vbench parameter behavior and add t2v example (#1497) 
Signed-off-by: asuka <1311722138@qq.com>
 2026-03-14 23:31:18 +08:00
Gao Han 365dfd5f8e [CI pipeline] Re-enable Diffusion Tensor Parallelism Test in pipeline (#1892) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-03-14 22:56:43 +08:00
SYLARGao HanHongsheng Liu
 		5377619c0b fix: propagate parallel_config through create_default_diffusion (#1878) 
Signed-off-by: lishunyang <lishunyang12@163.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-14 17:07:13 +08:00
zhou zhuoxinprinceprideDing Zuhao
 		dc72aede40 [CI] add multimodal processing correctness tests for Omni models (#1445) 
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: Ding Zuhao <e1583181@u.nus.edu>
 2026-03-14 14:15:25 +08:00
Daniel Huang c107f0b336 Buildkite hardware ci xpu test (#1340) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-14 11:24:05 +08:00
Alex Brooks e17130090c [BugFix] Fix Max Rank Handling in LoRA (#1397) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-03-14 02:42:54 +00:00
tzhouam 97899c6c98 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-03-13 19:03:57 +00:00
Yueqian Lin cebf78ba3e [Bugfix] Restore voice upload API and profiler endpoints reverted by #1719 (#1879) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-14 04:37:08 +13:00
dengyunyang 3e8cb6c631 [Diffusion] [Model] Dreamid omni (#1855) 
Signed-off-by: dengyunyang <584797741@qq.com>
 2026-03-13 21:53:06 +08:00
Ian EavesSamitGao Han
 		92f771d455 [Frontend] Rewrite video API for async job lifecycle (#1665) 
Signed-off-by: Ian Eaves <ian.k.eaves@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-03-13 17:11:06 +08:00
汪志鹏 e0b71056ba [BugFix]: Fix bagel online inference bug (#1804) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-03-13 08:58:44 +00:00
Gao Han 26b3ddee47 [CI failed] Disable Diffusion Tensor Parallelism Test (#1876) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-03-13 16:56:41 +08:00
Lancer 53a3df7309 [Bugfix] fix layer-wise offload incompatible with cache-dit (#1786) 
Signed-off-by: Lancer <maruixiang6688@gmail.com>
 2026-03-13 15:28:13 +08:00
SYLAR 75056b2db2 [Bugfix] Fix SD3.5-medium attn2 uninitialized weights (#1659) 
Signed-off-by: lishunyang <lishunyang12@163.com>
 2026-03-13 14:58:17 +08:00
Gao Han 717dbee5a6 [CI failed] Disable test for zimage tensor parallelism (#1870) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-03-13 14:52:48 +08:00
tzhouam d3db476f2d Merge branch 'main' into dev/vllm-align 2026-03-13 06:51:22 +00:00
tzhouam 62c8695dfd Update GPUGenerationModelRunner to clarify KV connector metadata handling during draft model runs 
- Changed variable name to better reflect its purpose in managing KV connector metadata.
- Updated comments for improved clarity regarding the handling of speculative configurations.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 03:16:33 +00:00
Alex BrooksandDidan Deng c8cf8a799d [Refactor] Use SP Plan for LongCat Sequence Parallelism (#1772) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-13 11:13:34 +08:00
dsinghviandYueqian Lin 0540af327e [BugFix] Readme and example runner file for cosyvoice3 missed in refactoring (#1685) 
Signed-off-by: dsinghvi <divyanshsinghvi@gmail.com>
Signed-off-by: Divyansh Singhvi <divyanshsinghvi@gmail.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-13 10:34:29 +08:00
tzhouam d911475b22 Fix output queue handling in OmniStage to return None if queue is closed during shutdown 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:59:04 +00:00
tzhouam e17fb0e97e Refactor OpenAI serving components and enhance multimodal handling 
- Updated import for OpenAIServingEmbedding in api_server.py.
- Enhanced omni_init_app_state to initialize renderer for engine_client.
- Added handle_oov_mm_token parameter to multiple model classes for better multimodal token handling.
- Improved comments for clarity in various model files.
- Adjusted GPUARModelRunner to ensure proper handling of late interaction runner attributes.

Signed-off-by: [Your Name] <[Your Email]>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-13 01:55:17 +00:00
NATURE 54a999d031 [skip CI][Docs] add connector design document (#1737) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-03-13 09:23:32 +08:00
Yueqian Lin 366b3368ea Add Fish Speech S2 Pro support with online serving and voice cloning (#1798) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-12 16:20:52 -04:00