anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
955 Commits 235 Branches 31 Tags
e6d4cdd71a46ab97f5682d0cff6cd3e76db73400
Commit Graph
231 Commits
Author SHA1 Message Date
tzhouam e6d4cdd71a Refactor model mode initialization and enhance error handling in AsyncOmni. Update worker type retrieval in omni_stage and add audio token handling in Qwen3OmniMoe classes for improved robustness. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-09 06:17:08 +00:00
Hongsheng Liu 9cdbcec44b Merge branch 'main' into dev/rebase-v0170 2026-03-08 09:18:57 +08:00
Canlin Guo 074af30a9b Revert "[Profile] Adding metrics for Diffusion/DiT Single diffusion Pipeline (#668)" (#1724) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-08 00:11:47 +13:00
Taichang Zhou fd4607f215 rebase 2026/03/07 
Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:30:58 +08:00
Taichang Zhou 5511bc14c7 Merge origin/main into dev/rebase-v0170 
- Resolved conflict in vllm_omni/engine/output_processor.py: use tracing_enabled for vLLM v0.17.0 API
- Resolved conflict in vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: keep correct _CodePredictorDecoderLayer.forward()
- Update Dockerfile.ci vLLM wheel commit to b31e9326a (releases/v0.17.0)

Made-with: Cursor

Signed-off-by: Taichang Zhou <tzhouam@connect.ust.hk>
 2026-03-07 13:18:33 +08:00
Didan Deng 30191ed112 [Feature]: Add vae-patch-parallel CLI argument in online serving (#1716) 
Signed-off-by: Didan Deng <33117903+wtomin@users.noreply.github.com>
 2026-03-07 01:45:02 +08:00
erfgss b7fcc9d07e [Profile] Adding metrics for Diffusion/DiT Single diffusion Pipeline (#668) 
Signed-off-by: Chen Yang <2082464740@qq.com>
Signed-off-by: erfgss <97771661+erfgss@users.noreply.github.com>
 2026-03-06 15:39:19 +08:00
Canlin Guo 846167b237 [1/N][Refactor] Clean up dead code in output processor (#1579) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-06 14:45:53 +08:00
tzhouam 8c41492307 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-06 03:02:50 +00:00
Jack Leegoogle-labs-jules[bot]JackLeeHal
 		c0b714a8ac [Feature] Support flexible task_type configuration for Qwen3-TTS models (#1197) 
Signed-off-by: Jack Lee <695697442@qq.com>
Signed-off-by: jackleehal <695697442@qq.com>
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: JackLeeHal <15664755+JackLeeHal@users.noreply.github.com>
 2026-03-05 13:39:09 -05:00
Lidang Jiang 28c2200b47 [BugFix] Return proper HTTP status for ErrorResponse in create_speech (#1687) 
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
 2026-03-05 23:38:20 +08:00
WeiQing Chen 83fe7d005b [Model] support LTX-2 text-to-video image-to-video (#841) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
 2026-03-05 23:31:48 +08:00
Juan Pablo Zuluagaandpablo 6a45efb287 [Feat][Qwen3TTS] reduce TTFA with flexible initial phase (#1583) 
Signed-off-by: pablo <pablo@agigo.ai>
Co-authored-by: pablo <pablo@agigo.ai>
 2026-03-05 23:14:41 +08:00
tzhouam 395eab8984 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0169 2026-03-05 02:29:14 +00:00
Alex BrooksandCanlin Guo 11a5842bd1 [Bugfix] Fix Dtype Parsing (#1391) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
Co-authored-by: Canlin Guo <961750412@qq.com>
 2026-03-04 22:49:28 +08:00
tzhouam 36a507e071 Merge remote-tracking branch 'remotes/origin/main' into dev/rebase-v0170 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-04 08:26:56 +00:00
Yueqian Lin 35bd3ff7d2 [Bugfix] Add TTS request validation to prevent engine crashes (#1641) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-03-04 14:27:02 +08:00
tzhouam 2398a7c1d7 Merge origin/main into dev/rebase-v0170 
Resolve conflict in omni_generation_scheduler.py: keep both
num_cached_tokens tracking and additional_information caching.

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 14:31:05 +00:00
Yuanheng Zhao b13761ed26 [Bugfix] Fix filepath resolution for model with subdir and GLM-Image generation (#1609) 
Signed-off-by: yuanheng <jonathan.zhaoyh@gmail.com>
Signed-off-by: Yuanheng Zhao <jonathan.zhaoyh@gmail.com>
 2026-03-03 23:40:53 +13:00
tzhouam 2da9d28f1b Enhance reproducibility in Ulysses attention tests by replacing current_omni_platform.seed_everything with torch's manual_seed and adding support for CUDA devices. Introduce get_supported_tasks method in AsyncOmni class to dynamically determine available tasks based on stage capabilities. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 09:45:51 +00:00
tzhouam 98926707f5 rebase 2026/3/3 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 03:50:34 +00:00
tzhouam 3e539fcb6a Merge remote-tracking branch 'origin/main' into dev/vllm-align 
Made-with: Cursor

# Conflicts:
#	vllm_omni/entrypoints/openai/serving_chat.py
#	vllm_omni/worker/gpu_model_runner.py

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-03 02:57:27 +00:00
Samit 22a51a7cdb [Feature][Wan2.2] Speed up diffusion model startup by multi-thread weight loading (#1504) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-03-02 14:53:25 +00:00
rongfu.leng a4b2e48718 [Bugfix] Import InputPreprocessor into Renderer (#1566) 
Signed-off-by: rongfu.leng <lenronfu@gmail.com>
 2026-03-02 21:54:15 +08:00
汪志鹏andCursor e37a89ffcb [Feature]: Support cfg kv-cache transfer in multi-stage (#1422) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-03-02 05:06:28 +00:00
tzhouam 26744bcb35 Merge origin/main into dev/vllm-align 
Resolved conflicts:
- vllm_omni/inputs/preprocess.py: took origin/main's elif mm_processor_kwargs branch, fixed undefined mm_uuids reference
- vllm_omni/model_executor/models/qwen3_tts/qwen3_tts_code_predictor_vllm.py: took origin/main's NPU support

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-03-02 02:10:32 +00:00
Ting FU b0156d8e1c [BugFix] Fix unexpected crash when init OmniDiffusion (#1562) 
Signed-off-by: Semmer2 <semmer@live.cn>
 2026-02-28 14:21:49 +08:00
Yueqian Lin 37775ffe60 fix(qwen3-tts): fix Base ICL voice clone producing corrupted audio (#1554) 
Signed-off-by: linyueqian <linyueqian@outlook.com>
 2026-02-28 11:49:47 +08:00
Canlin GuoHongsheng LiuNicolò Lucchesi
 		eaef3a31a4 [Profiler] Support online profiling (#1136) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-02-27 14:41:20 +08:00
WeiQing ChenIsotr0pyIsotr0pyHongsheng Liu
 		0b884ebc6c [Feature]: Native GGUF Quantization Support for DiT (#1285) 
Signed-off-by: David Chen <530634352@qq.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Signed-off-by: WeiQing Chen <40507679+david6666666@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 11:39:14 +08:00
Jared Wen汪志鹏Hongsheng Liu
 		4146095623 [Perf] GLM Image (#920) 
Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-27 10:29:14 +08:00
wuhang 4bee33744f [Bugfix] Use uds for zmq address if not set --stage-id (#1522) 
Signed-off-by: wuhang <wuhang6@huawei.com>
 2026-02-26 21:44:07 +08:00
Canlin GuoandHongsheng Liu f53152d340 [Feature] Support HSDP for diffusion models (#1339) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:31:10 +08:00
Juan Pablo ZuluagapabloHongsheng Liu
 		03c01e0415 [Qwen3TTS][Feat] Streaming output (#1438) 
Signed-off-by: pablo <pablo@agigo.ai>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: pablo <pablo@agigo.ai>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:02:35 +08:00
Zeyu Huang | 黃澤宇andGao Han c68be4dc4b [Chore] remove unused logger in omni_diffusion (#531) (#1509) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2026-02-26 20:30:35 +08:00
Alex Brooks bc26de834e [Bugfix] Fix LoRA Scaling on Active Adapters (#1421) 
Signed-off-by: Alex Brooks <albrooks@redhat.com>
 2026-02-26 17:22:35 +08:00
tzhouam 8d0c46487e Merge origin/main into dev/vllm-align 
Resolve conflicts:
- docker/Dockerfile.ci: Keep wheel install section, update hash to 6042e66cd5
- vllm_omni/platforms/npu/worker/npu_model_runner.py: Accept main's refactoring

Made-with: Cursor

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-26 08:19:53 +00:00
Yanick Schraner 97f4c5b1e7 [Bugfix] Add TTS configuration options (#1177) 
Signed-off-by: Yanick Schraner <yanick.schraner@bs.ch>
 2026-02-26 00:14:44 +08:00
Dong WandHongsheng Liu 1eebb374f1 [Model]Add new nextstep_1(Diffusion) model(only T2I) (#612) 
Signed-off-by: Dong Wang <dongw2019@gmail.com>
Signed-off-by: sniper35 <dongw2019@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 00:09:11 +08:00
Junhong Liu 43939df2a5 [Bugfix] Robust Audio Data Handling in _create_audio_choice (#1222) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
 2026-02-25 22:59:49 +08:00
tzhouam a252918008 [Update] Upgrade vLLM wheel version and enhance metrics handling 
- Updated the vLLM wheel to commit 24650715105a26b41c88f4164777b5e1a0f3200b in Dockerfile.ci.
- Modified the `calculate_metrics` function to handle `tokenizer` as optional (can be None) in metrics.py.
- Adjusted the `benchmark` function to accept `tokenizer` as optional in patch.py.
- Enhanced condition checks in OmniARScheduler to ensure valid token generation in omni_ar_scheduler.py.
- Updated input processing to include supported tasks in input_processor.py.
- Cleaned up unnecessary code in output_processor.py.
- Added support for SSL ciphers in the API server configuration in api_server.py.
- Enabled speculative decoding in GPU model runners for improved performance in gpu_generation_model_runner.py and gpu_model_runner.py.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-25 10:02:23 +00:00
tzhouam 22e01c416b Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-25 08:15:55 +00:00
Junhong LiuandHongsheng Liu c0908dd3fa [Bugfix] Forward all vllm-omni serve command parameters to model (#985) 
Signed-off-by: Junhong Liu <98734602+LJH-LBJ@users.noreply.github.com>
Signed-off-by: Junhong Liu <ljh_lbj@163.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 22:20:01 +08:00
wuhang汪志鹏Copilot
 		36b8f80266 [Feature] Support Stage Based Deployment CLI (#939) 
Signed-off-by: wuhang <wuhang6@huawei.com>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Signed-off-by: wuhang <whlbx@hotmail.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-02-24 16:55:23 +08:00
tzhouam 04ccb062fc Add signature compatibility tests for OmniLLM class 
- Introduced tests to ensure method signatures in OmniLLM match those of the base LLM class, preventing runtime TypeErrors due to parameter mismatches.
- Updated the _run_engine method in OmniLLM to include an optional output_type parameter.

Signed-off-by: [Your Name] <your.email@example.com>

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-24 06:58:39 +00:00
tzhouam e22ae62396 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-24 03:23:20 +00:00
zhou zhuoxin 40f72bb870 [Feat]: add ASCII art logo for vLLM-Omni (#1430) 2026-02-23 22:10:09 +08:00
tzhouam 292da4b0d8 Merge remote-tracking branch 'origin/main' into dev/vllm-align 2026-02-23 02:09:09 +00:00
Sy03andHongsheng Liu efbe4116ff [Feature] : Support disaggregated inference pipeline for Qwen3_TTS (#1161) 
Signed-off-by: Sy03 <1370724210@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-21 07:36:44 +08:00
tzhouam 9869ea007e Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data. 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-02-20 08:34:53 +00:00