anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,956 Commits 235 Branches 31 Tags
c306445ff2a3cee456a1fbc054b175c868893bda
Commit Graph
476 Commits
Author SHA1 Message Date
tzhouamandClaude Opus 5 c306445ff2 rebase: migrate diffusion KV cache to the vLLM 0.29 layout model 
vLLM 0.29 removed AttentionSpec.indexes_kv_by_block_stride and reshaped
KVCacheTensor from (size, shared_by) to an explicit layout: layers in L
order plus layer_stride and block_stride. The block-stride decision now
lives on one physical CacheConfig.kv_cache_layout that the engine core
resolves once, and layout.is_block_outermost is the direct successor of
the old per-spec flag.

Diffusion backends subclass the Omni AttentionBackend ABC, so upstream
get_supported_kv_cache_layouts never sees them and the engine core never
resolves a layout for them. Add diffusion_kv/layout.py to pin it before
get_kv_cache_configs runs, defaulting to LBNHC, which reproduces the
indexes_kv_by_block_stride=False that every dense diffusion backend
declared. The backend hook is kept as an enforced contract rather than
deleted: a backend needing block-stride indexing selects a block-outermost
layout, and a contradiction raises instead of mis-addressing K/V.

Workers receive only a KVCacheConfig, so the resolved name travels on it
(KVCacheConfig.kv_cache_layout) and is adopted into each worker CacheConfig
before allocation. Without that, init_kv_cache raises in the worker because
get_resolved_kv_cache_layout has nothing to return.

init_kv_cache also lost attn_groups and cache_dtype in 0.29; the production
caller in model_runner_backend and the test stubs move to the six-argument
form.

Tensor construction goes through one derivation built on the upstream
compute_layout_strides helper, shared by production and tests, so a wrong
stride cannot be masked by a parallel test-only copy.

Also updates the sizing expectation: 0.29 subtracts the null block that
BlockPool permanently holds back before estimating, so sixteen physical
pages now yield fifteen usable ones.

One review point is addressed only in part, deliberately.
build_native_kv_cache_configs now takes indexes_kv_by_block_stride and
propagates a block-outermost layout end to end when it is set, with a test
covering control plane to worker adoption. Automatic discovery of that
requirement from the workers is not added: 0.29 removed the field that used
to carry it, every diffusion backend in tree declares False, and inventing
an executor RPC for a capability nothing requests would ship untestable
plumbing. Until a backend needs it, the seam is explicit and the mismatch
raises at worker init rather than corrupting addressing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0146Xu5Sgj4vXxLxvLjV5YPj
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:53 +08:00
tzhouam f9879239ba chore: satisfy changed-file policy 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
tzhouam 87a1fd8da2 rebase: adapt to vLLM v0.29 APIs 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
 2026-09-10 00:41:52 +08:00
andyluo7 b9932aa17b [CI/Build][ROCm] Normalize SenseNova paged-decode hardware markers (#6935) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-09 07:32:38 -07:00
NumberWan 038c9b948f [Bugfix] Resolve #6931 hub FA3 on torch 2.13 via kernels 0.16.1 (#7185) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-09-09 10:20:34 +00:00
Bo Li cdd4638b17 [Feature][Diffusion] Expose detailed pipeline timings (#6822) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-09 09:58:42 +00:00
Anjie HouandClaude Sonnet 5 ecdd762ff5 [2/N] Add a minimal temporal chunk callback for MiniMax-H3 (#7017) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-09 16:40:32 +08:00
Mu GuanLin ffa991b24f [Perf][Model] Add optimized LTX-2.5 DiffVAE operators (#7308) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 08:31:16 +00:00
Mu GuanLin ab56148505 [Bugfix] Make LTX vocoder decoding deterministic (#7231) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 06:37:01 +00:00
Anjie HouandClaude Opus 5 b6f4df320d [Refactor][Diffusion] Resolve offload topology through one plan resolver (#7209) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-09 08:17:38 +08:00
Lei KeandKrystalRay 5e4baea64d [Bugfix][NPU] Limit MiniMax H3 modulation grid size (#6794) 
Signed-off-by: KrystalRay <keeleiray@gmail.com>
Co-authored-by: KrystalRay <keeleiray@gmail.com>
 2026-09-08 09:08:46 -07:00
Harry MellorHongsheng LiuClaude Opus 5
 		fdaf4e31bd Add vLLM-Omni library info to Hugging Face Hub requests (#5381) 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
 2026-09-08 10:08:38 -06:00
liangmenghuangandClaude Fable 5.1 7a80b79698 [Bugfix][Model] GR00T-N1.7: honor the per-request seed for flow-matching noise (#7253) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 15:53:20 +00:00
andyluo7 de1b0708bd [CI][ROCm] Route LTX2 Ulysses parity to two-GPU lane (#7234) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-08 22:59:37 +08:00
liangmenghuangandClaude Fable 5.1 d7740336ff [Bugfix][Model] GR00T-N1.7: honor processor_config image pipeline (use_albumentations / crop_fraction / letter_box_transform) to match Isaac-GR00T (#7083) 
Signed-off-by: liangmengh <liangmengh@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
 2026-09-08 18:59:01 +08:00
Honghan Zhu c086554d1f [Perf]Boogu-Image Skip dense attention masks (#6871) 
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com>
 2026-09-08 00:53:20 -07:00
Chenguang Zheng 087ac46318 [Refactor 1/N] Centralize Omni config resolution (#5140) 
Signed-off-by: Chenguang ZHENG <645327136@qq.com>
 2026-09-08 10:30:26 +08:00
Jierui (Jerry) Xu c3498fb869 [Model] Add Boogu-Image Turbo pipeline resolution (#6961) 
Signed-off-by: Jerry Xu <xjr2423@gmail.com>
 2026-09-08 08:59:23 +08:00
Anjie HouClaude Sonnet 5Hongsheng Liu
 		6942a5b600 [1/N] Stream Wan VAE chunks to the media consumer (#7016) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-08 07:41:13 +08:00
Rahul SteigerandNils Hoffmann 3b335814cc [Bugfix][Cosmos3] Fix distributed Transfer output envelope (#7205) 
Signed-off-by: Rahul Steiger <rsteiger@nvidia.com>
Co-authored-by: Nils Hoffmann <nhoffmann@nvidia.com>
 2026-09-08 07:34:08 +08:00
Lei KeandKrystalRay 7be014bce6 [Bugfix][NPU] Fix MiniMax-H3 INT8 quantization dispatch (#6876) 
Signed-off-by: KrystalRay <keeleiray@gmail.com>
Co-authored-by: KrystalRay <keeleiray@gmail.com>
 2026-09-07 14:50:44 +00:00
汪志鹏 9b6a308914 [Perf][Diffusion] Keep LoRA weights resident across activation cycles (#7195) 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
 2026-09-07 14:27:24 +00:00
WeiQing Chen 30d6a0b4e6 [Bugfix][MiniMax-H3] Stabilize keyframe VAE encoding (#7191) 
Signed-off-by: david6666666 <530634352@qq.com>
Signed-off-by: David Chen <530634352@qq.com>
 2026-09-07 21:52:54 +08:00
Mu GuanLin 3204a0b2ad [Model][Perf] Optimize LTX Ulysses sequence parallelism (#7079) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-07 10:48:26 +00:00
Rahul SteigerandRahul Steiger f080f41e03 [Bugfix][Cosmos3] Auto-pad non-divisible Ulysses GEN sequences (#6918) 
Signed-off-by: Rahul Steiger <rsteiger@nvl72d188-T17.cm.cluster>
Co-authored-by: Rahul Steiger <rsteiger@nvl72d188-T17.cm.cluster>
 2026-09-07 08:58:28 +00:00
Rahul SteigerRahul SteigerRahul Steiger
 		677265b64b [Bugfix][Diffusion] Make Cosmos3 transfer image arrays writable (#6915) 
Signed-off-by: Rahul Steiger <rsteiger@nvl72d179-T17.cm.cluster>
Co-authored-by: Rahul Steiger <rsteiger@nvl72d179-T17.cm.cluster>
Co-authored-by: Rahul Steiger <rsteiger@aws-cmh-slurm-1-vscode-04.cm.cluster>
 2026-09-07 01:06:16 -07:00
Shenglei FuandClaude 91dfafcaab [Bugfix][Diffusion] Include guidance_scale_2_provided in the request-batch key (#7078) 
Signed-off-by: Shenglei Fu <sfu@confluent.io>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-07 15:58:19 +08:00
summerandAlicia 4b81f692d1 [Bugfix][Diffusion] Canonicalize cache_backend=None to "none" (#7041) 
Signed-off-by: summer <128961079+zhang-keliang@users.noreply.github.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-07 15:56:51 +08:00
BruceLoveDecimal 5cd925631c [Feature][LingBot World]stepwise execution (#6844) 
Signed-off-by: BruceLoveDecimal <liuqihao970610@gmail.com>
Signed-off-by: liuqihao <liuqihao970610@gmail.com>
 2026-09-07 13:38:23 +08:00
NumberWanandAlicia 7f32c326c6 [BugFix][Qwen-Image-Edit] restore txt_seq_lens RoPE width after #3588 regression (#5586) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
Co-authored-by: Alicia <115451386+congw729@users.noreply.github.com>
 2026-09-07 12:10:47 +08:00
linzhenpl07andClaude Sonnet 5 38abc0ba28 [AR-Diffusion] Allocate K and V separately so the compiled path stops cloning the pool (#6463) 
Signed-off-by: linzhenpl07 <linzhenpl07@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-07 11:40:20 +08:00
Zeyu Huang | 黃澤宇 2df95de421 [Core][Diffusion] Refactor prompt updates into a modality-ready interaction framework (#6294) 
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
 2026-09-07 11:40:04 +08:00
Hongsheng Liu 715b8b8746 [Model] MiniMax-H3: validate versioned text-conditioning handoff (#6720) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-06 21:45:10 +08:00
汪志鹏andClaude Opus 5 af74a5a155 [Feature][MiniMax-H3] Support the full LightX2V Turbo LoRA matrix (#7062) 
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-06 20:23:36 +08:00
XenorynandClaude 039808e0d9 [Perf][Diffusion] Skip the Ulysses length all-gather when shards are padded equal (#5717) 
Signed-off-by: Xenoryn <timberh2000@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2026-09-06 16:23:30 +08:00
DMAN 06541d1f52 [Model] FP8 Support For Boogu-Image (#6925) 
Signed-off-by: Dmaner <2663515256@qq.com>
 2026-09-06 04:48:38 +00:00
prettygirlisnotme 32370968d0 [Model] Add Boogu-Image Turbo native DMD path (#6699) 
Signed-off-by: Dragonborn <1078757180@qq.com>
 2026-09-06 08:44:15 +08:00
Zeyu Huang | 黃澤宇asukaqaq-sHongsheng Liu
 		f26f9c3a7f [Diffusion] Implement paged AR↔DiT KV connector ("v1", reusing vllm's native mooncake) (#6310) 
Signed-off-by: asukaqaq-s <1311722138@qq.com>
Signed-off-by: Huang, Zeyu <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: asukaqaq-s <1311722138@qq.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-06 05:41:54 +08:00
SYLAR ea3cf99894 [diffusion][feature] Add component-selective offload policies (#5929) 
Signed-off-by: lishunyang12 <lishunyang12@163.com>
 2026-09-06 05:40:09 +08:00
Samit 44d3ae100a [Bugfix][Diffusion] Fuse distilled LoRA weights before HSDP sharding (#6948) 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
 2026-09-06 01:29:59 +08:00
Hongsheng Liu d4096e01fe [Model] Add native MAGI-2 Preview diffusion support (#5918) 
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-05 15:32:30 +08:00
vOv 13c617a140 [Model] Add sequence parallelism to SANA-Video 2B (#5940) 
Signed-off-by: cr-gao <gaochenrui@sjtu.edu.cn>
 2026-09-05 15:22:53 +08:00
TriggerandGao Han ea14b04e9f [Bugfix][BAGEL] Restore original denoise step count (#7049) 
Signed-off-by: Trigger <Meng.Bo.Wang@outlook.com>
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
 2026-09-04 17:05:05 +00:00
SYLARandhsliuustc0106 22cefe04b4 [Bugfix] Update Hub kernel dependency floor (#7020) 
Signed-off-by: hsliuustc0106 <liuhongsheng4@huawei.com>
Co-authored-by: hsliuustc0106 <liuhongsheng4@huawei.com>
 2026-09-04 22:15:55 +08:00
heyuanliu-intel 10701e1838 [Bugfix] Release diffusion worker RPC results on ranks that do not reply (#6989) 
Signed-off-by: heyuanliu-intel <heyuan.liu@intel.com>
 2026-09-04 16:45:17 +08:00
Shenglei FuandClaude Opus 4.8 0e903e9695 [Diffusion] Enable request-level batching for Boogu-Image (T2I) (#6968) 
Signed-off-by: Shenglei Fu <117230642+ShengleiFu@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-09-04 16:27:51 +08:00
Mu GuanLin 175dec11be [Perf][Model] Optimize LTX video output transport (#7000) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-04 06:47:28 +00:00
Trigger e43c787544 [Model] Add BAGEL step execution and continuous batching (#6359) 2026-09-04 01:36:11 +00:00
Bo Li fda3af8bbc [Bugfix] Preserve BF16 residual semantics in fused MiniMax-H3 modulation (#6878) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-04 02:01:34 +08:00
Joshna-MedisettyandCursor 24c49cf392 [Bugfix][XPU] Add MoTRMSNorm.forward_xpu delegating to forward_native (#5568) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Signed-off-by: Joshna-Medisetty <joshna.medisetty@intel.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
 2026-09-02 23:06:17 -05:00