anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,960 Commits 235 Branches 31 Tags
85d1f92ee58de7cf65bcfd9a02fc0a4c3e3dc9d3
Commit Graph
881 Commits
Author SHA1 Message Date
tzhouamandClaude Opus 5 9a399adba8 chore: satisfy changed-file policy 
Scoped pre-commit over the 27 files this rebase touches: ruff format
reflow, the vLLM-Omni SPDX copyright line on the new modules, and wording
the typos hook rejected.

The Python 3.10 mypy hook still reports 24 errors across five of these
files. All 24 sit on pre-existing lines; none fall on a line this rebase
changed, which the repository documents as the CI-skipped condition.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
tzhouamandClaude Opus 5 c306445ff2 rebase: migrate diffusion KV cache to the vLLM 0.29 layout model 
vLLM 0.29 removed AttentionSpec.indexes_kv_by_block_stride and reshaped
KVCacheTensor from (size, shared_by) to an explicit layout: layers in L
order plus layer_stride and block_stride. The block-stride decision now
lives on one physical CacheConfig.kv_cache_layout that the engine core
resolves once, and layout.is_block_outermost is the direct successor of
the old per-spec flag.

Diffusion backends subclass the Omni AttentionBackend ABC, so upstream
get_supported_kv_cache_layouts never sees them and the engine core never
resolves a layout for them. Add diffusion_kv/layout.py to pin it before
get_kv_cache_configs runs, defaulting to LBNHC, which reproduces the
indexes_kv_by_block_stride=False that every dense diffusion backend
declared. The backend hook is kept as an enforced contract rather than
deleted: a backend needing block-stride indexing selects a block-outermost
layout, and a contradiction raises instead of mis-addressing K/V.

Workers receive only a KVCacheConfig, so the resolved name travels on it
(KVCacheConfig.kv_cache_layout) and is adopted into each worker CacheConfig
before allocation. Without that, init_kv_cache raises in the worker because
get_resolved_kv_cache_layout has nothing to return.

init_kv_cache also lost attn_groups and cache_dtype in 0.29; the production
caller in model_runner_backend and the test stubs move to the six-argument
form.

Tensor construction goes through one derivation built on the upstream
compute_layout_strides helper, shared by production and tests, so a wrong
stride cannot be masked by a parallel test-only copy.

Also updates the sizing expectation: 0.29 subtracts the null block that
BlockPool permanently holds back before estimating, so sixteen physical
pages now yield fifteen usable ones.

One review point is addressed only in part, deliberately.
build_native_kv_cache_configs now takes indexes_kv_by_block_stride and
propagates a block-outermost layout end to end when it is set, with a test
covering control plane to worker adoption. Automatic discovery of that
requirement from the workers is not added: 0.29 removed the field that used
to carry it, every diffusion backend in tree declares False, and inventing
an executor RPC for a capability nothing requests would ship untestable
plumbing. Until a backend needs it, the seam is explicit and the mismatch
raises at worker init rather than corrupting addressing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
tzhouam f9879239ba chore: satisfy changed-file policy 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
tzhouam 87a1fd8da2 rebase: adapt to vLLM v0.29 APIs 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
hyw 6fb7b0a05e [Model] Skip unused frame packing in Wan2.2 S2V (#7155) 
Signed-off-by: hyw <yuweih205@gmail.com>
 2026-09-09 14:34:46 +00:00
NumberWan 038c9b948f [Bugfix] Resolve #6931 hub FA3 on torch 2.13 via kernels 0.16.1 (#7185) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-09-09 10:20:34 +00:00
Bo Li cdd4638b17 [Feature][Diffusion] Expose detailed pipeline timings (#6822) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-09 09:58:42 +00:00
Anjie HouandClaude Sonnet 5 ecdd762ff5 [2/N] Add a minimal temporal chunk callback for MiniMax-H3 (#7017) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-09 16:40:32 +08:00
Mu GuanLin ffa991b24f [Perf][Model] Add optimized LTX-2.5 DiffVAE operators (#7308) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 08:31:16 +00:00
Mu GuanLin ab56148505 [Bugfix] Make LTX vocoder decoding deterministic (#7231) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 06:37:01 +00:00
andyluo7 e0d0879794 [Bugfix][Magi2] Allow import without an active Triton driver (#7239) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-08 22:48:08 -07:00
boatman cdf027874c [Feat][OmniVoice]Support Varlen Attn, Request-Batch and Step-Execution (#6408) 
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com>
 2026-09-08 18:58:38 -07:00
Anjie HouandClaude Opus 5 b6f4df320d [Refactor][Diffusion] Resolve offload topology through one plan resolver (#7209) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-09 08:17:38 +08:00
Lei KeandKrystalRay 5e4baea64d [Bugfix][NPU] Limit MiniMax H3 modulation grid size (#6794) 
Signed-off-by: KrystalRay <keeleiray@gmail.com>
Co-authored-by: KrystalRay <keeleiray@gmail.com>
 2026-09-08 09:08:46 -07:00
Harry MellorHongsheng LiuClaude Opus 5
 		fdaf4e31bd Add vLLM-Omni library info to Hugging Face Hub requests (#5381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-08 10:08:38 -06:00
liangmenghuangandClaude Fable 5.1 7a80b79698 [Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise (#7253) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 15:53:20 +00:00
liangmenghuangandClaude Fable 5.1 d7740336ff [Bugfix][Model] GR00T-N1.7: honor processor_config image pipeline (use_albumentations / crop_fraction / letter_box_transform) to match Isaac-GR00T (#7083) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 18:59:01 +08:00
Honghan Zhu c086554d1f [Perf]Boogu-Image Skip dense attention masks (#6871) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
 2026-09-08 00:53:20 -07:00
Chenguang Zheng 087ac46318 [Refactor 1/N] Centralize Omni config resolution (#5140) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 10:30:26 +08:00
ZacheryAU b1ec136bd7 [Frontend] Add omni benchmark support for image and video endpoints (#4728) 2026-09-08 09:28:21 +08:00
Jierui (Jerry) Xu c3498fb869 [Model] Add Boogu-Image Turbo pipeline resolution (#6961) 
Signed-off-by: Jerry Xu <xjr2423@gmail.com>
 2026-09-08 08:59:23 +08:00
Anjie HouClaude Sonnet 5Hongsheng Liu
 		6942a5b600 [1/N] Stream Wan VAE chunks to the media consumer (#7016) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-08 07:41:13 +08:00
Rahul SteigerandNils Hoffmann 3b335814cc [Bugfix][Cosmos3] Fix distributed Transfer output envelope (#7205) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Co-authored-by: Nils Hoffmann <nhoffmann@nvidia.com>
 2026-09-08 07:34:08 +08:00
汪志鹏 9b6a308914 [Perf][Diffusion] Keep LoRA weights resident across activation cycles (#7195) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-09-07 14:27:24 +00:00
WeiQing Chen 30d6a0b4e6 [Bugfix][MiniMax-H3] Stabilize keyframe VAE encoding (#7191) 
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
 2026-09-07 21:52:54 +08:00
Mu GuanLin 3204a0b2ad [Model][Perf] Optimize LTX Ulysses sequence parallelism (#7079) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-07 10:48:26 +00:00
Rahul SteigerandRahul Steiger f080f41e03 [Bugfix][Cosmos3] Auto-pad non-divisible Ulysses GEN sequences (#6918) 
Signed-off-by: Rahul Steiger <rsteiger@nvl72d188-T17.cm.cluster>
Co-authored-by: Rahul Steiger <rsteiger@nvl72d188-T17.cm.cluster>
 2026-09-07 08:58:28 +00:00
FredHuangNV 85b0bad2d3 [Frontend][Model] Add Cosmos3 control uploads to video API (#7027) 
Signed-off-by: Fred Huang <frhuang@nvidia.com>
 2026-09-07 01:38:56 -07:00
Rahul SteigerRahul SteigerRahul Steiger
 		677265b64b [Bugfix][Diffusion] Make Cosmos3 transfer image arrays writable (#6915) 
Signed-off-by: Rahul Steiger <rsteiger@nvl72d179-T17.cm.cluster>
Co-authored-by: Rahul Steiger <rsteiger@nvl72d179-T17.cm.cluster>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-09-07 01:06:16 -07:00
Shenglei FuandClaude 91dfafcaab [Bugfix][Diffusion] Include guidance_scale_2_provided in the request-batch key (#7078) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-07 15:58:19 +08:00
summerandAlicia 4b81f692d1 [Bugfix][Diffusion] Canonicalize cache_backend=None to "none" (#7041) 
Signed-off-by: summer <128961079+zhang-keliang@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-07 15:56:51 +08:00
BruceLoveDecimal 5cd925631c [Feature][LingBot World]stepwise execution (#6844) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-07 13:38:23 +08:00
NumberWanandAlicia 7f32c326c6 [BugFix][Qwen-Image-Edit] restore txt_seq_lens RoPE width after #3588 regression (#5586) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-07 12:10:47 +08:00
Zeyu Huang | 黃澤宇 2df95de421 [Core][Diffusion] Refactor prompt updates into a modality-ready interaction framework (#6294) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-09-07 11:40:04 +08:00
汪志鹏 eeee257913 [Bugfix][Diffusion] Restore the TRTLLM attention default for the MiniMax-H3 modular alias (#7162) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-09-07 07:21:21 +08:00
SYLARandhsliuustc0106 14d2d00536 [Bugfix] Fix Hunyuan Image-3 expert mapping contract (#7021) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-06 21:43:37 +08:00
汪志鹏andClaude Opus 5 af74a5a155 [Feature][MiniMax-H3] Support the full LightX2V Turbo LoRA matrix (#7062) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-06 20:23:36 +08:00
XenorynandClaude 039808e0d9 [Perf][Diffusion] Skip the Ulysses length all-gather when shards are padded equal (#5717) 
Signed-off-by: Xenoryn <timberh2000@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-06 16:23:30 +08:00
DMAN 06541d1f52 [Model] FP8 Support For Boogu-Image (#6925) 
Signed-off-by: Dmaner <2663515256@qq.com>
 2026-09-06 04:48:38 +00:00
prettygirlisnotme 32370968d0 [Model] Add Boogu-Image Turbo native DMD path (#6699) 
Signed-off-by: Dragonborn <1078757180@qq.com>
 2026-09-06 08:44:15 +08:00
Zeyu Huang | 黃澤宇asukaqaq-sHongsheng Liu
 		f26f9c3a7f [Diffusion] Implement paged AR↔DiT KV connector ("v1", reusing vllm's native mooncake) (#6310) 
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-06 05:41:54 +08:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
Samit 44d3ae100a [Bugfix][Diffusion] Fuse distilled LoRA weights before HSDP sharding (#6948) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-09-06 01:29:59 +08:00
Hongsheng Liu d4096e01fe [Model] Add native MAGI-2 Preview diffusion support (#5918) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-05 15:32:30 +08:00
vOv 13c617a140 [Model] Add sequence parallelism to SANA-Video 2B (#5940) 
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
 2026-09-05 15:22:53 +08:00
TriggerandGao Han ea14b04e9f [Bugfix][BAGEL] Restore original denoise step count (#7049) 
Signed-off-by: Trigger <Meng.Bo.Wang@outlook.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-04 17:05:05 +00:00
SYLARandhsliuustc0106 22cefe04b4 [Bugfix] Update Hub kernel dependency floor (#7020) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-04 22:15:55 +08:00
MikeyDong1 41f4f9d8c9 [Perf][DreamZero] Remove redundant CUDA compile guard (#6983) 
Signed-off-by: MikeyDong1 <xdong-1219@outlook.com>
 2026-09-04 09:01:05 -04:00
heyuanliu-intel 10701e1838 [Bugfix] Release diffusion worker RPC results on ranks that do not reply (#6989) 
Signed-off-by: heyuanliu-intel <heyuan.liu@intel.com>
 2026-09-04 16:45:17 +08:00
Shenglei FuandClaude Opus 4.8 0e903e9695 [Diffusion] Enable request-level batching for Boogu-Image (T2I) (#6968) 
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-09-04 16:27:51 +08:00