anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
955 Commits 235 Branches 31 Tags
e6d4cdd71a46ab97f5682d0cff6cd3e76db73400
Commit Graph
955 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
tzhouam e6d4cdd71a Refactor model mode initialization and enhance error handling in AsyncOmni. Update worker type retrieval in omni_stage and add audio token handling in Qwen3OmniMoe classes for improved robustness. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-09 06:17:08 +00:00
Hongsheng Liu 9cdbcec44b Merge branch 'main' into dev/rebase-v0170 2026-03-08 09:18:57 +08:00
Alicia b255654a76 [CI] Add release-pipeline.yaml. (#1726) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-08 12:26:04 +13:00
Taichang Zhou c8b695f181 smaller talker gpu util to avoid oom 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-08 01:48:42 +08:00
Canlin Guo 074af30a9b Revert "[Profile] Adding metrics for Diffusion/DiT Single diffusion Pipeline (#668)" (#1724) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-08 00:11:47 +13:00
Taichang Zhou f405240513 Change pytest fixture scope from "module" to "class" for omni_server and omni_server_no_async_chunk in test_qwen3_tts.py to improve test isolation and resource management. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 14:50:59 +08:00
Taichang Zhou c630cf39c1 Enhance Dockerfile.ci by adding system dependencies installation and copying project files into the container. This improves the build process for the CI environment. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 14:02:48 +08:00
Taichang Zhou 3facc89dd1 Update base image in Dockerfile.rocm to v0.17.0 for improved compatibility and features. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:59:23 +08:00
Taichang Zhou 68bad871bc Fix model detection logic in is_diffusion_model function to ensure config_dict is checked for None before accessing its fields. This improves robustness against missing model_index.json files. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:57:44 +08:00
Taichang Zhou f945d2a30d Refactor Dockerfile.ci to streamline installation process and update vllm-omni installation method. Removed commented-out installation commands and ensured Python symlink is created. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:57:23 +08:00
Taichang Zhou 315f9401db Refactor imports in metrics.py and patch.py to use TokenizerLike from vllm.tokenizers instead of PreTrainedTokenizerBase from transformers. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:33:10 +08:00
Taichang Zhou fd4607f215 rebase 2026/03/07 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:30:58 +08:00
Taichang Zhou 0ecfff0694 comment out extra installation 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:29:12 +08:00
Taichang Zhou 4a43c6b017 Merge branch 'dev/rebase-v0170' of https://github.com/vllm-project/vllm-omni into dev/rebase-v0170 2026-03-07 13:26:46 +08:00
Taichang Zhou 0c2464db20 revert unnecessary changes 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:26:32 +08:00
Zhou TaichangandCyrus Leung 5f1e41970c Update vllm_omni/benchmarks/patch/patch.py 
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-07 13:23:47 +08:00
Zhou TaichangandCyrus Leung 4ead1c8913 Update vllm_omni/benchmarks/metrics/metrics.py 
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-03-07 13:23:38 +08:00
Taichang Zhou 2187db0e36 Remove obsolete RFC documents for automated rebase agent, output processor refactor, model runner clearance, long function decomposition, and model runner V2 integration. These files are no longer relevant to the current project direction. 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:22:39 +08:00
Taichang Zhou 5511bc14c7 Merge origin/main into dev/rebase-v0170 
- Resolved conflict in vllm_omni/engine/output_processor.py: use tracing_enabled for vLLM v0.17.0 API
- Resolved conflict in vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: keep correct _CodePredictorDecoderLayer.forward()
- Update Dockerfile.ci vLLM wheel commit to b31e9326a (releases/v0.17.0)

Made-with: Cursor

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:18:33 +08:00
Taichang Zhou 3a6b2e27be [Update] Bump VLLM base tag to v0.17.0 in CI Dockerfile 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:14:10 +08:00
Didan Deng 30191ed112 [Feature]: Add vae-patch-parallel CLI argument in online serving (#1716) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-07 01:45:02 +08:00
erfgss b7fcc9d07e [Profile] Adding metrics for Diffusion/DiT Single diffusion Pipeline (#668) 
Signed-off-by: Chen Yang <2082464740@qq.com>
Signed-off-by: erfgss <97771661+erfgss@users.noreply.github.com>
 2026-03-06 15:39:19 +08:00
Yueqian Lin 411fe05bd4 [skip CI][Docs] Add TTS model developer guide (#1693) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-06 14:49:19 +08:00
JohnJanandHongsheng Liu 92adcdeab4 [feature]: support flux2.klein cache_dit (#1209) 
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-03-06 14:46:57 +08:00
Canlin Guo 846167b237 [1/N][Refactor] Clean up dead code in output processor (#1579) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-06 14:45:53 +08:00
Yueqian Lin 3bc89de0c0 Add non-async chunk support for Qwen3-TTS (#1678) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-06 14:45:32 +08:00
Baoyuan Qi 32fd084ae2 [MiMo-Audio] Bugfix tp lg than 1 (#1688) 
Signed-off-by: 齐保元 <qibaoyuan@xiaomi.com>
 2026-03-06 14:45:08 +08:00
Sy03 90ec499bd5 [Refactor][Perf] Qwen3-TTS: re-prefill Code Predictor with torch.compile + enable Code2Wav decoder CUDA Graph (#1617) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-03-06 18:21:55 +13:00
Du BinlinyueqianYueqian Lin
 		fd51841230 perf: replace per-element .item() GPU syncs with batch .tolist() in TTS code predictor (#1614) 
Signed-off-by: dubin555 <dubin555@gmail.com>
Signed-off-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: linyueqian <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 12:22:01 +08:00
tzhouam 8c41492307 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-06 03:02:50 +00:00
tzhouam 44ac28c386 trigger RTD rebuild 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-06 03:02:07 +00:00
tzhouam 1ccffce2bc rebase to updated commit 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-06 02:32:26 +00:00
Ding Zuhaoand汪志鹏 128e6f4b97 [Feature][Bagel] Add CFG parallel mode (#1578) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-03-06 09:55:54 +08:00
Yueqian Lin 8536dce759 [Cleanup] Move cosyvoice3 tests to model subdirectory (#1666) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-05 15:28:22 -05:00
Jack Leegoogle-labs-jules[bot]JackLeeHal
 		c0b714a8ac [Feature] Support flexible task_type configuration for Qwen3-TTS models (#1197) 
Signed-off-by: Jack Lee <695697442@qq.com>
Signed-off-by: jackleehal <695697442@qq.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com>
 2026-03-05 13:39:09 -05:00
zhanqiuhuandYueqian Lin e643aaec15 [Bugfix] Fix all-silence TTS output: use float32 for speech tokenizer decoder (#1664) 
Signed-off-by: Zhanqiu Hu <zh338@cornell.edu>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
 2026-03-06 00:04:26 +08:00
Canlin Guo d5ddc190f9 [UX] Add progress bar for diffusion models (#1652) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-05 23:56:14 +08:00
wangyu ea317e4552 [Doc] Add the test guide document. [skip ci] (#1376) 
Signed-off-by: yenuo26 <410167048@qq.com>
 2026-03-05 23:41:36 +08:00
Lidang Jiang 28c2200b47 [BugFix] Return proper HTTP status for ErrorResponse in create_speech (#1687) 
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
 2026-03-05 23:38:20 +08:00
WeiQing Chen 83fe7d005b [Model] support LTX-2 text-to-video image-to-video (#841) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-05 23:31:48 +08:00
Juan Pablo Zuluagaandpablo 6a45efb287 [Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-05 23:14:41 +08:00
Ziming Huang 070ea0ddc3 [Optimize][Qwen3-Omni] Reduce inter-packet latency in async chunk (#1656) 
Signed-off-by: ZeldaHuang <hzm414167@alibaba-inc.com>
 2026-03-05 22:26:30 +08:00
Alicia e674d60008 [CI] Remove high concurrency tests before issue #1374 fixed. (#1683) 
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-03-05 17:46:37 +08:00
rongfu.leng 562136d390 [Misc] remove logits_processor_pattern this field, because vllm have … (#1675) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-05 09:36:19 +00:00
Alex Brooks fa944bd8e8 [BugFix] Fix LongCat Sequence Parallelism / Small Cleanup (#1631) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-03-05 17:31:00 +08:00
tzhouam 5be08baed3 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-05 08:53:26 +00:00
Daniel Huang 24c80070d6 Replace hard-coded cuda generator with current_omni_platform.device_type (#1677) 
Signed-off-by: Daniel Huang <daniel1.huang@intel.com>
 2026-03-05 06:53:30 +00:00
zhou zhuoxin 58eb1c68db [FP8 Quantization] Add FP8 quantization support for Flux transformer (#1640) 
Signed-off-by: zhou zhuoxin <zhouzhuoxin1508@outlook.com>
 2026-03-05 11:08:34 +08:00
tzhouam 395eab8984 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-05 02:29:14 +00:00
jiangmengyu18Hongsheng LiuSYLAR
 		5493e75877 [NPU] [Features] [Bugfix] Support mindiesd adaln (#1537) 
Signed-off-by: jiangmengyu18 <451528648@qq.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: SYLAR <125541396+lishunyang12@users.noreply.github.com>
 2026-03-05 09:01:19 +08:00