anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
3,468 Commits 235 Branches 31 Tags
321e8bb47ff43d18cc81990fb076d0f6f042f89b
Commit Graph
1035 Commits
Author SHA1 Message Date
admin 321e8bb47f fix(quant): load offline-quantized compressed-tensors checkpoints directly on CPU under offload 
The diffusion loader treated any quant config without the MXFP8/
is_checkpoint_quantized markers as online quantization: it loaded every
component on the accelerator before offloading, so serving the int8
weight-only compressed-tensors artifact OOM'd at model construction
(encoder 17 GiB + VAE + packed DiT skeleton > 24 GiB). compressed-tensors
is always a serialized format with no on-load quantization pass — detect
it (through component wrappers too) and load on the host, round-tripping
each quantized module through the accelerator only for the post-load
Marlin repack, which has no CPU path.
 2026-10-08 07:36:01 -04:00
hurukawa 3e705fcdb1 [Model] Add JoyAI-Image-Edit pipeline support (#4112)
Build Wheel / build (3.12) (push) Failing after 36s
Details
Build Wheel / build (3.11) (push) Failing after 36s
Details
pre-commit / pre-commit (push) Failing after 44s
Details 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
Signed-off-by: hurukawa <61525444+nagisa-kunhah@users.noreply.github.com>
 2026-10-08 08:40:55 +08:00
MaciejBalaNV e54e185593 [feature] Reduce memory usage with SP (#7728) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
 2026-10-07 22:36:32 +08:00
NumberWanIsotr0pyTianyu GuoCursorSamitHuangGao Han
 		3dc35694b3 [New model] Qwen-Image-2.1 support (rebase + review follow-ups) (#8099) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Signed-off-by: SamitHuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Co-authored-by: Isotr0py <Isotr0py@outlook.com>
Co-authored-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: SamitHuang <285365963@qq.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-10-07 21:23:59 +08:00
QianCyrusandHongsheng Liu 8ef77be66e [Model] Avoid repeated attention and MLP work in Klein TP (#8553) 
Signed-off-by: QianCyrus <101633534+QianCyrus@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-10-07 21:21:05 +08:00
Dong1017andamy-why-3459 ee9ab9ea88 [Perf] Fuse AdaLayerNorm CUDA eager path with Triton (#8231) 
Signed-off-by: GUOGUO <xwdong1998@163.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-10-06 15:11:45 +00:00
Nikita OsterovleffffArtem Borisov
 		5f95115e70 Kandinsky6 (#8537) 
Signed-off-by: leffff <levnovitskiy@gmail.com>
Signed-off-by: Artem Borisov <divotionchannel@gmail.com>
Co-authored-by: leffff <levnovitskiy@gmail.com>
Co-authored-by: Artem Borisov <divotionchannel@gmail.com>
 2026-10-06 15:55:15 +08:00
QianCyrus c2cd13c009 [Core] Add batch parallel VAE decode (#8162) 
Signed-off-by: QianCyrus <101633534+QianCyrus@users.noreply.github.com>
 2026-10-06 13:51:35 +08:00
QianCyrus 099f9b553d [Model] Optimize Wan VAE first-frame decoding with Conv2d (#8179) 
Signed-off-by: QianCyrus <101633534+QianCyrus@users.noreply.github.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-10-06 11:44:34 +08:00
Wenbo Ji　嵇文博 539c533762 [Bugfix] Keep client error status in the single-GPU diffusion executor (#8474) 
Signed-off-by: Wenbo Ji <36562829+fusheng-ji@users.noreply.github.com>
 2026-10-05 13:09:34 -04:00
Zhou Taichang 0b0d2d7009 [Rebase] Align vLLM-Omni with vLLM v0.31.0 (#8459) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-10-05 14:11:01 +08:00
MaciejBalaNVBartosz Stefaniak汪志鹏
 		bbd539d2b6 [feature] Enable multi-node inference through Ray backend (#7938) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Co-authored-by: Bartosz Stefaniak <bstefaniak@nvidia.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-10-05 02:53:36 +00:00
SunandOpenAI Codex 3f20252151 [Model] Fuse MammothModa2 QK norm and RoPE (#7969) 
Signed-off-by: levius <2114377220@qq.com>
Co-authored-by: OpenAI Codex <noreply@openai.com>
 2026-10-05 02:44:45 +08:00
quan-yi-aiandHongsheng Liu 65f5a1d5b3 [Bugfix][Examples][MammothModa2] Serve T2I via LLM-typed DiT stage (#7199) + benchmark harness and profiler wiring (#7293) 
Signed-off-by: quan-yi-ai <82917002+quan-yi-ai@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-10-05 02:42:58 +08:00
hurukawa b9cf74352a [Perf] Support continuous batching for MammothModa2 (#7954) 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
 2026-10-05 02:22:37 +08:00
hurukawa c9c478dd07 [Perf] Support TeaCache for MammothModa2 (#5357) 
Signed-off-by: nagisa-kun <1434936049@qq.com>
Signed-off-by: nagisa <1434936049@qq.com>
 2026-10-05 01:25:56 +08:00
Rahul SteigerandRahul Steiger 4934741df9 [Bugfix] Add TRTLLM attention custom op and execution contract (#7214) 
Signed-off-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-10-04 09:33:51 +00:00
nodeeeeee b65f97bc48 [Feature] Add MiniMax H3 res_multistep sampler (#8378) 
Signed-off-by: nodeeeeee <zhangkai.nodeee@gmail.com>
 2026-10-04 07:26:19 +00:00
YAN YIXINandClaude Opus 5.5 148cb943ce [Bugfix][CI/Build] Fix layerwise offload shutdown and enable it for the LTX-2 distilled T2V accuracy test (#8453) 
Signed-off-by: nikiwang-125 <yixinyan79@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-04 07:25:51 +00:00
Yueqian Lin 4025bb30f6 [Perf][AuK] Graph the encoder prefill, cache reference latents and fuse the codec activation (#8305) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-10-02 15:43:55 +00:00
Yueqian Lin 13c66da752 [Perf][AuK] Precompute adaLN modulations per request and run the conv position embedding as one GEMM (#8328) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-10-02 08:12:29 -07:00
Rahul Steiger 596487fe62 [Core] Prototype diffusion attention execution contracts with FA4 (#7379) 2026-10-02 17:58:47 +08:00
MaciejBalaNVand汪志鹏 97d6c693a6 [feature] Add HSDP pre-sharded option (#7948) 
Signed-off-by: Maciej Bala <mbala@nvidia.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
 2026-10-02 09:35:09 +00:00
wtz2333andYuanheng Zhao 68003cf6ad [Bugfix] Fix washed-out LingBot Video outputs (#7037) 
Signed-off-by: wtz2333 <2955110911@qq.com>
Co-authored-by: Yuanheng Zhao <54058983+yuanheng-zhao@users.noreply.github.com>
 2026-10-02 03:57:50 +00:00
汪志鹏 a038b38179 [Model] Add MiniMax-H3 latent super-resolution and hi-res refinement (#8322) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-10-01 14:24:40 +08:00
yzhautouskay 27a6321a63 [Model][Cosmos3] Keep diffusion sampling state in FP32 (#7592) 
Signed-off-by: Yuliya Zhautouskaya <yzhautouskay@nvidia.com>
 2026-09-30 11:35:51 -07:00
Yan CaoyancaocnZhou Taichang
 		004a5711dc [Bugfix][Helios] Guard cross-attn KV cache against address reuse (#8064) 
Signed-off-by: yancaocn <yancaochn@163.com>
Co-authored-by: yancaocn <yancaochn@163.com>
Co-authored-by: Zhou Taichang <tzhouam@connect.ust.hk>
 2026-09-30 16:44:51 +08:00
Armaan AmatyaYueqian LinHongsheng Liu
 		aa41a3ac97 [Bugfix] Fix the remaining Python 3.10 breakages (#7985) 
Signed-off-by: Armaan Amatya <armaanamatya2014@gmail.com>
Signed-off-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-30 11:38:23 +08:00
Mu GuanLinandHongsheng Liu 3a82c85880 [Experimental][Perf] Eliminate redundant host copies from generated-video transport (#7355) 
Signed-off-by: mglyn <1203789601@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-30 11:37:57 +08:00
vOv 25bdb38b11 [Bugfix] Do not block diffusion client shutdown on a dead subprocess (#7812) 
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
 2026-09-30 11:25:37 +08:00
Yueqian Lin 146ccc133a [Perf][AuK] Compile the DiT step, use cuDNN attention and warm up at startup (#8300) 
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-30 11:21:21 +08:00
boatmanHongsheng LiuYueqian Lin
 		35cf6ac355 [Feat][Perf][AuK]Support Online Serving and DiT CUDAGraph (#7469) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-30 00:18:38 +00:00
BruceLoveDecimalandYueqian Lin 3a28882946 [Perf][Auk] VAE cuda graph and tiling (#7881) 
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-29 23:27:01 +00:00
0z5a0z5aprincepride
 		7d6904aa94 SeedVR2: bounded long-video restoration (#8102) 
Signed-off-by: 0z5a <Dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@uni-tuebingen.de>
Signed-off-by: 0z5a <dezhen.lu@student.uni-tuebingen.de>
Signed-off-by: princepride <wangzhipeng628@gmail.com>
Co-authored-by: 0z5a <dezhen.lu@uni-tuebingen.de>
Co-authored-by: princepride <wangzhipeng628@gmail.com>
 2026-09-29 11:33:50 +00:00
Ding Zuhao 1c1fdf719c [Bugfix][BAGEL] Keep replicated prefill and cache-update phases off the SP strategy (#8177) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
 2026-09-29 03:45:49 +00:00
RuQing Xu 0173cb3748 [Kernel] Improve 2 backends importing FlashInfer: FLASHINFER_ATTN and TRTLLM_ATTN (#7431) 
Signed-off-by: Ruqing Xu <7891482+xrq-phys@users.noreply.github.com>
 2026-09-28 21:21:58 +00:00
Jierui (Jerry) XuandClaude Opus 5 965ad68f1d [Kernel][MammothModa2] Use shared RMSNorm in the DiT (#7489) 
Signed-off-by: Jerry Xu <xjr2423@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-29 00:12:16 +08:00
R0CKSTAR 29b818a785 feat(magi2): add fused BF16 routed MoE path (#7206) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-09-28 23:20:30 +08:00
DMANandHongsheng Liu 79a46a4f54 [Perf][Diffusion] support Sensenova online fp8 (#7955) 
Signed-off-by: Dmaner <2663515256@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-28 23:17:37 +08:00
Ding Zuhao 1b97ff1188 [Model] Support SenseNova-U1-A3B-MoT (MoE) (#8172) 
Signed-off-by: Ding Zuhao <e1583181@u.nus.edu>
 2026-09-27 08:48:08 +00:00
chickeytonClaude Fable 5.1Yueqian Lin
 		798d52ce55 [Core][Model] Fit PersonaPlex into the Unified Full-duplex Framework (RFC #7181 PR 2) (#7695) 
Signed-off-by: chickeyton <ngton2014@gmail.com>
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
 2026-09-27 03:02:24 +00:00
Shenglei FuandClaude eb67caaaa9 [Diffusion] Enable request-level batching for Boogu-Image (TI2I) (#7227) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-26 22:53:35 +08:00
R0CKSTAR e75278dccc feat(magi2): add fused SwiGLU7 activation (#7252) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
 2026-09-26 22:53:04 +08:00
Anjie HouandHongsheng Liu 8a49c65bf9 [Core][Diffusion] Add byte backpressure to chunk transfers (#7916) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-26 17:48:32 +08:00
Anjie HouClaude Opus 5Hongsheng Liu
 		acf6620233 [Refactor][Diffusion] Read the resolved offload policy outside the config layer (#7327) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-26 17:45:20 +08:00
Yuanheng Zhao 3bd5ac9685 [Bugfix] Align Ming-Image prompt templates and RNG, fix number of layer to decompose (#8118) 
Signed-off-by: Yuanheng <jonathan.zhaoyh@gmail.com>
 2026-09-25 22:47:29 +08:00
Anjie HouClaude Opus 5.5Gao Han
 		95781c31e0 [Bugfix][Diffusion] Skip AllGather DLO weight restore on worker shutdown (#8103) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-25 08:33:32 +00:00
R0CKSTARandHongsheng Liu 3bf96080f3 feat(magi2): add fused mHC post-processing (#7261) 
Signed-off-by: Xiaodong Ye <xiaodong.ye@mthreads.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-25 16:25:12 +08:00
XenorynClaudeHongsheng Liu
 		0c7042d0d5 [Feature][Diffusion] Add BOOGU Image sequence parallel support (#5718) 
Signed-off-by: Xenoryn <timberh2000@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-25 16:18:23 +08:00
NumberWan e5613794ca [BugFix][Cosmos3] Decode uploaded video paths in v2v preprocess (#8106) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-09-25 07:53:24 +00:00