anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
1,775 Commits 235 Branches 31 Tags
c69ed4037dcbae7306cba9094cb42bf94de464f0
Commit Graph
70 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Canlin Guo 52020c7af7 [UX] Rename default config to hunyuan_image_3_moe (#3835) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-26 01:41:15 +08:00
Canlin GuoandHongsheng Liu c890b3a99c [BugFix][NPU] Honor prefer_model_sampler in NPU AR runner (#3517) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-25 09:08:18 +08:00
Canlin Guo 2aafe0bf94 [Perf] Optimize sampler D2H sync for HY-Image (#3617) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-20 15:40:34 +08:00
Canlin Guo 4f13b871f9 [CI] Add Qwen3-TTS tests for ready tag (#3600) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-14 20:37:51 +08:00
Canlin Guo 0ab1d30056 [CI][Test] Add NPU nightly tests (#3480) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-13 16:58:17 +12:00
Canlin Guo 5e263b6929 [BugFix] Rename attention_config to diffusion_attention_config (#3489) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-11 11:27:35 +08:00
Canlin GuoandHongsheng Liu ef7f2f1cd2 [Docs] Refactor the attention backend docs/skill (#3475) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-11 09:13:07 +08:00
Canlin GuoandHongsheng Liu 11c4c7f0ff [Diffusion][Attention] Support per-role attention backend via CLI (#2681) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-05-10 11:59:05 +08:00
Canlin Guo 4a24a517ab [BugFix] Probe __dict__ instead of hasattr when patching WanRMS_norm (#3400) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-07 11:54:16 +08:00
Canlin Guo c007d40b14 [NPU] Upgrade to v0.20.0 & align with GPU model runner (#3325) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-05-04 22:38:43 +08:00
Canlin Guo 15a1143695 [Mergify] Init config (#812) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-30 08:48:53 +00:00
Canlin Guo 23b2a95df6 [Docs] Add Wan2.2 image-to-video recipe for Ascend NPU (A2/A3) (#2919) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-20 19:35:48 +08:00
Canlin Guo 9313f37056 [Agent] Add NPU main2main skill (#2858) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-18 21:57:39 +08:00
Canlin Guo 95b5b2ee43 [Bugfix] Fix UT for the missing of log_stats in Engine (#2706) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-12 22:14:43 +08:00
Canlin Guo 5d58abb11e [Log] Wire stat loggers into AsyncOmniEngine to match AsyncLLM (#2551) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-12 19:54:23 +08:00
Canlin GuoandHongsheng Liu e771842781 [Feature] Support vae tiling parallel encode (#2368) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-04-06 14:47:45 +00:00
Canlin Guo 6dc61c9a20 [Perf] Skip Wan2.2 cross attn Ulysses SP (#2459) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-03 16:23:36 +08:00
Canlin Guo 6be5d05a7c [Docs] Update docs to use vllm-ascend v0.18.0rc1 (#2453) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-03 09:54:47 +08:00
Canlin Guo e2892ef6c1 [Perf] Optimize Wan2.2 rotary embedding (#2393) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-04-02 14:28:44 +08:00
Canlin Guo 3fe46360f0 [Release] Upgrade NPU dockerfile & docs for v0.18.0 (#2271) 2026-03-27 20:37:16 +08:00
Canlin Guo 7bcd43f456 [Docs][skip ci] Fix omni and tts docs (#2130) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-24 22:39:23 +08:00
Canlin Guo fb9f20a921 [API] Add layers and resolution parameters to /v1/images/edits endpoint (#2053) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-24 15:47:41 +08:00
Canlin GuoSamitlishunyang lishunyang12@163.com
 		7217557950 [Refactor] Unify torch profiler for omni and diffusion models (#2099) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Samit <285365963@qq.com>
Co-authored-by: lishunyang lishunyang12@163.com
 2026-03-24 15:25:52 +08:00
Canlin Guo d636c32ff9 [Bugfix] Set PREEMPTED status when moving requests from running to waiting queue (#1893) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-18 17:56:13 +08:00
Canlin Guo 9718a9e369 [NPU] Upgrade to v0.17.0 (#1890) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-16 14:42:59 +08:00
Canlin Guo 29357027b0 [Bugfix] Allow to enable HSDP alone (#1567) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-09 06:36:13 +00:00
Canlin Guo c3a59f10f3 [NPU] Support Helios-Mid / Distilled (#1648) 2026-03-08 17:24:25 +08:00
Canlin Guo 074af30a9b Revert "[Profile] Adding metrics for Diffusion/DiT Single diffusion Pipeline (#668)" (#1724) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-08 00:11:47 +13:00
Canlin Guo 846167b237 [1/N][Refactor] Clean up dead code in output processor (#1579) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-06 14:45:53 +08:00
Canlin Guo d5ddc190f9 [UX] Add progress bar for diffusion models (#1652) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-03-05 23:56:14 +08:00
Canlin Guo c81266703a [NPU][Bugfix] Align GPU side and recover qwen3-tts (#1564) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-28 12:46:13 +08:00
Canlin GuoHongsheng LiuNicolò Lucchesi
 		eaef3a31a4 [Profiler] Support online profiling (#1136) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Canlin Guo <961750412@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Nicolò Lucchesi <nicolo.lucchesi@gmail.com>
 2026-02-27 14:41:20 +08:00
Canlin Guo 3b0c21f94d [Platform] Enable layerwise offload on all hardware (#1492) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-27 06:47:28 +08:00
Canlin GuoandHongsheng Liu f53152d340 [Feature] Support HSDP for diffusion models (#1339) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-26 21:31:10 +08:00
Canlin Guo 9913c00144 [Feature] Support Wan2.2 output with irregular shapes (#1279) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-26 17:03:43 +08:00
Canlin Guo 22c8cde328 [NPU] Upgrade to v0.16.0 (#1375) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-26 14:26:50 +08:00
Canlin GuoandHongsheng Liu af11b02081 [Minor][Refactor] Pass seq_token_counts explicitly (#1425) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-02-24 14:10:00 +08:00
Canlin Guo e192b0a6c5 [Misc] Add per-request generator_device to online image gen and edit (#1183) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-10 18:47:08 +08:00
Canlin Guo 285d71b26a [Refactor] Reuse upstream Qwen3MoeSparseMoeBlock (#1202) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-06 01:06:50 +00:00
Canlin Guo f95f8e936d [NPU] Update Dockerfile and docs for v0.14.0 (#671) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-04 15:10:50 +08:00
Canlin Guo 3bc3c95708 [Bugfix][Qwen-Image-Edit] Add a warning log for none negative_prompt (#1170) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-02-03 18:12:11 +08:00
Canlin Guomuziyuhui666Hongsheng Liu
 		ed89c8b043 [Bugfix] Fix NPU SDPA attention mask shape and semantics (#1031) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: muziyuhui666 <111362884+muziyuhui666@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-31 14:14:15 +08:00
Canlin Guo 722a69e8d1 [NPU] Align with GPUModelRunner (#1114) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-31 04:26:34 +00:00
Canlin Guo 4eeea68cff [Platform] Add supports_torch_inductor interface (#1108) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-30 15:21:43 +00:00
Canlin Guo 2c4974c6ea [Bugfix][Test] Re-enable the log simple tests (#1065) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-30 00:55:53 +08:00
Canlin Guo 0c8b5ac2d5 [Bugfix] Set VaeImageProcessor do_convert_rgb True (#1032) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-29 12:02:39 +00:00
Canlin Guo c9392dff01 [Hardware] Support platforms and plugin system (#774) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-28 22:12:52 +08:00
Canlin Guo cb48968294 [Bugfix] Modify output to model_runner_output (#1026) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-28 18:42:39 +08:00
Canlin Guo cadd77262d [NPU] Upgrade to v0.14.0 (#820) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-27 16:45:32 +08:00
Canlin GuoandHongsheng Liu b564132e44 [Diffusion][Acceleration] Support TeaCache for Z-Image (#817) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-21 16:53:18 +08:00
Canlin Guo dde96c2468 [Perf][Qwen3-Omni] Fuse QKV linear and gate_up proj (#734) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-11 11:06:24 +08:00
Canlin GuoandIsotr0py 9c2f7467a0 [Perf] Use vLLM's SharedFusedMoE in Qwen3-Omni (#560) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-01-10 04:30:06 +00:00
Canlin Guo 695e62e2b9 [Docs][NPU] Upgrade to v0.12.0 (#656) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-06 10:49:31 +08:00
Canlin Guoandlishunyang 52afcb6055 [Feature] Support torch profiler across omni stages (#553) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: lishunyang <lishunyang12@163.com>
Co-authored-by: lishunyang <lishunyang12@163.com>
 2026-01-05 17:32:20 +08:00
Canlin Guo 1c9e58ee19 [Bugfix] Removed the NPU-specific code path in _run_local_attention (#597) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2026-01-04 14:14:10 +08:00
Canlin GuoandHongsheng Liu 4f8cb95571 [Metric] Fix the computation of e2e_total_tokens (#519) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-01-01 00:13:17 +08:00
Canlin Guo 46afa046ce [NPU] Support mixed modalities for Qwen3-Omni (#537) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-31 16:15:20 +08:00
Canlin Guo be81443c8a [Bugfix][NPU] Add _model_forward for ModelRunner (#505) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-28 04:09:34 +08:00
Canlin GuoHongsheng LiuJiangyun Zhu
 		4913f9cc18 [NPU][Model] Use cos and sin cache for Z-Image to support NPU (#485) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>
 2025-12-26 21:52:25 +08:00
Canlin GuoandHongsheng Liu 52d15c4c04 [NPU][Model] Support Qwen3-Omni on NPU (#484) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-27 00:05:55 +13:00
Canlin GuoandHongsheng Liu fe3d9a6a81 [NPU][Upgrade] Upgrade to v0.12.0 (#447) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2025-12-26 14:10:23 +08:00
Canlin Guo 4f1c34099a [Refactor][Diffusion] Make NPUWorker inherit GPUWorker (#483) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-26 12:54:03 +08:00
Canlin Guo e1f9691120 [Clean] Remove the redundant decoding payloads logic (#404) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-26 04:48:34 +13:00
Canlin Guo 259fe1150c [Docs] Pin the installation on NPU as v0.11.0rc1 (#434) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-23 20:20:12 +08:00
Canlin Guo 97a5860681 [Security][Feature] Use Msgpack in OmniSerializer instead of pickle (#310) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-22 13:21:16 +13:00
Canlin Guo 2b6d8f17ba [NPU][CI] Add the CI workflow on NPU (#231) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-17 17:07:12 +08:00
Canlin GuoIsotr0pyGao Han
 		f75c21bcd0 [E2E] Add Qwen2.5-Omni model test with OmniRunner (#168) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Gao Han <gaohan19@huawei.com>
 2025-12-06 06:20:26 +00:00
Canlin Guo 6079b14040 [NPU][Refactor] Rename Diffusion* to Generation* (#211) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-05 08:23:37 +00:00
Canlin Guo 863cdb82b6 [Docs] remove Ascend word to make docs general (#190) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-03 23:21:11 -08:00
Canlin Guo 7e60c49ce4 [Bugfix] Add Fake VllmConfig on NPU and add slicing/tiling args in Qwen-Image (#145) 
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
 2025-12-01 16:40:01 +00:00