anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
2,980 Commits 235 Branches 31 Tags
3ea9e605011fc2a0bfa9f3e8531286fb683068dd
Commit Graph
2980 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Tianyu Guo 3ea9e60501 Fix diffusion API initialization (#7868) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-20 12:08:50 +08:00
Tianyu Guo 9dabaf8b80 Default Qwen-Image 2.1 CFG scale to 1.0 (#7859) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-20 11:51:35 +08:00
Isotr0py 5b540f55d3 Fix decode cuda graph for Qwen-Image-2.1 model-level cpu offload (#7801) 2026-09-19 00:10:53 +08:00
Isotr0py 47e6acc1e3 Add FP8 KV cache support for Qwen-Image-2.1 DiT (#7797) 2026-09-19 00:06:58 +08:00
Isotr0py 2a92b71cb9 Add FP8 online qunatization support for Qwen-Image-2.1 (#7792) 
Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 22:33:16 +08:00
Tianyu Guo ab4cf24b3e Optimize Qwen-Image 2.1 attention and decode graph reuse (#7790) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 19:37:19 +08:00
Tianyu Guo f930348e79 Preserve Qwen-Image 2.1 condition latent order (#7766) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 18:07:45 +08:00
Tianyu Guo b0a309e5e7 Isolate Qwen-Image 2.1 prefill and decode scheduling (#7770) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 14:11:22 +08:00
Tianyu GuoandIsotr0py a26013145c Align Qwen-Image 2.1 preprocessing, CFG defaults and prefix cache (#7769) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
Co-authored-by: Isotr0py <mozf@mail2.sysu.edu.cn>
 2026-09-18 13:53:15 +08:00
Tianyu Guo 936c0cc7fe Align Qwen-Image-2.1 VAE normalization with Diffusers (#7768) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 13:51:06 +08:00
Tianyu Guo 93d7a06e7f Use pre-norm text features for Qwen-Image 2.1 (#7765) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 13:50:36 +08:00
Tianyu Guo eb54a3186b Separate Qwen-Image 2.1 batches by condition image layout (#7764) 
Signed-off-by: Tianyu Guo <guoty9@mail2.sysu.edu.cn>
 2026-09-18 13:47:35 +08:00
Isotr0py 3affc7766d Enable VAE tiling for Qwen-Image-2.1 (#5) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix(diffusion): Qwen-Image-2.1 VAE spatial ratio, tiling e2e verification

- AutoencoderKLQwenImage21: derive spatial_compression_ratio from the
  architecture (2 ** len(temperal_downsample) = 16x) instead of trusting
  the checkpoint's scale_factor_spatial=8, which corrupted tiled
  decode/encode geometry (output cropped to half size) and tiling gating.
- pipeline: _read_vae_scale_factor prefers the architecture-derived value
  for the same reason.
- models/qwen_image_21/__init__: lazy exports to break the circular import
  distributed VAE -> package __init__ -> pipeline -> distributed VAE.
- Add VAE unit tests: ratio derivation, tiled vs untiled decode/encode
  consistency, distributed tile_split/exec/merge == tiled_decode.
- e2e matrix (GB200, seed 42, 20 steps) all green; see e2e_out/qwen21_vae/.
- examples: qwen_image_21_edit.py gains --vae-use-tiling.
- docs: mark VAE patch-parallel verified for Qwen-Image-2.1; recipe notes
  the default-tile banding limitation at 2048px+.

* clean tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix cudnn determination

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:50 +08:00
Isotr0py 4fc3cf8c60 Enable CUDA Graph for DiT decode layers (#4) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

* test(diffusion): add Qwen-Image-2.1 unit tests

Signed-off-by: Isotr0py <mozf@inferact.ai>

* feat(qwen_image_21): opt-in CUDA graph capture for KV-cache decode steps

Decode steps (1..N) recompute only the target image's tokens at a fixed
shape, but the per-request dict KV cache made addresses unstable, so the
step could not be captured. Prefill now copies prefix K/V into persistent
fixed-address buffers (keyed by branch/batch/prefix_len/target_tokens/
dtype/device/backend) and re-points the caller's cache dicts at them;
decode replays a graph captured per key over the full 32-block forward.
Eager fallback (logged) for non-CUDA, SP/ring/TP>1, HSDP, compiled or
offload-hooked blocks, KV-cache quantization, dynamic LoRA wrappers,
padded text masks, unknown keys, and capture failure.

Off by default; opt in via OmniDiffusionConfig.enable_cuda_graph_decode
or the QwenImage21Transformer2DModel constructor kwarg.

E2E (GB200, T2I 1024x1024, 50 steps, CFG 4.0, CUDNN_ATTN):
171.2 -> 142.1 ms/step, 8.70s -> 7.24s total (1.20x), 44.9 dB PSNR
vs eager output.

* clean

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-18 10:02:39 +08:00
Isotr0py 2912672b2e Initialize QMI pipeline implementation (#2) 
* init

Signed-off-by: Isotr0py <mozf@inferact.ai>

* fix

Signed-off-by: Isotr0py <mozf@inferact.ai>

---------

Signed-off-by: Isotr0py <mozf@inferact.ai>
 2026-09-16 15:14:23 +08:00
LHXuuuandamy-why-3459 a7c295d544 [CI] Stabilize Qwen3-Omni Server VAD E2E (#7356) 
Signed-off-by: LHXuuu <xulianhao.xlh@antgroup.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-10 07:40:49 +00:00
herotai214 bbac4df7d3 [Refactor] P0.2: Migrate API server helpers out of api_server (#5453) 
Signed-off-by: herotai214 <herotai214@gmail.com>
 2026-09-10 15:03:21 +08:00
Zhou TaichangandClaude Opus 5 aff7d64948 [Rebase] Rebase to vLLM 0.29.0 (#7230) 
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
v0.29.0rc1 		2026-09-10 12:08:28 +08:00
Sy03 333f14e97e [Bugfix][MiniCPM-o] Align structured chat content with native omni rendering (#7344) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-10 11:38:11 +08:00
Mu GuanLin 4b8dcac681 [Kernel] Enable LTX DiffVAE fusions on SM100 and SM103 (#7350) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-10 03:07:35 +00:00
Weiming Liaoandwangyu 35a83a0885 [NPU][CI] Add A5 and 310P CI support (#6875) 
Signed-off-by: Weiming Liao <liaowm5@gmail.com>
Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
 2026-09-10 02:16:14 +00:00
andyluo7 3d71a2b3ad [CI/Build] Stabilize LTX2 vocoder autocast test on ROCm (#7336) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-10 10:09:39 +08:00
Gao Hanandamy-why-3459 6e63ea875e [Bugfix] Allow LLM replicas on different GPUs to initialize concurrently (#7292) 
Signed-off-by: Gao Han <hgaoaf@connect.ust.hk>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-10 09:54:20 +08:00
Hongsheng Liuandhsliu_ustc 0637454301 [Skills] Add perf-evidence isolation, stage-attribution, and realtime-contract requirements (#6820) 
Signed-off-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
Co-authored-by: hsliu_ustc <hsliu_ustc@noreply.gitcode.com>
 2026-09-10 08:05:47 +08:00
Guangjian Dong faad918588 [Doc] Document num_outputs_per_prompt for /v1/videos (#7341) 
Signed-off-by: Guangjian <hiro20833@gmail.com>
 2026-09-10 08:00:13 +08:00
Joshna-Medisetty 51cf142bc1 [Bugfix][XPU] Restore N-D output shape for W8A16 FP8 linear (#7301) 
Signed-off-by: Joshna Medisetty <joshna.medisetty@intel.com>
Signed-off-by: Joshna-Medisetty <joshna.medisetty@intel.com>
 2026-09-09 17:11:31 -05:00
Sy03 b02862beee [Model] Optimize MOSS-TTS Local batched execution and streaming codec (#7202) 
Signed-off-by: Sy03 <1370724210@qq.com>
 2026-09-09 13:02:12 -07:00
bojiang-li 504e5a6a6e [Doc] Add dual DGX Spark MiniMax-H3 results (#7343) 
Signed-off-by: bojiang-li <327132355+bojiang-li@users.noreply.github.com>
 2026-09-09 18:48:41 +00:00
hyw 6fb7b0a05e [Model] Skip unused frame packing in Wan2.2 S2V (#7155) 
Signed-off-by: hyw <yuweih205@gmail.com>
 2026-09-09 14:34:46 +00:00
andyluo7 b9932aa17b [CI/Build][ROCm] Normalize SenseNova paged-decode hardware markers (#6935) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-09 07:32:38 -07:00
shiyichuanandmershi 06f67695f1 [CI/Build] Avoid duplicate stage CLI deploy config (#7007) 
Signed-off-by: mershi <mershi@tencent.com>
Co-authored-by: mershi <mershi@tencent.com>
 2026-09-09 22:05:44 +08:00
andyluo7andHongsheng Liu c94cf4ddde [CI][ROCm] Match CUDA/NPU L2/L3 label routing (#6966) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
 2026-09-09 13:47:15 +00:00
Shaun WalshClaude Opus 4.6Nick Cao
 		0e602509aa [BugFix] Add field validators for /v1/audio/generate request (#4741) 
Signed-off-by: Shaun Walsh <shaunwalsh24@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Nick Cao <ncao@redhat.com>
 2026-09-09 13:38:42 +00:00
kunkun 2f1845fa4e fix: reserve Qwen3-Omni NVFP4 backend fix (#7200) 
Signed-off-by: kunkunblueberry <1833921874@qq.com>
 2026-09-09 20:35:16 +08:00
Zheng Wengangandamy-why-3459 7c1b1e4189 [Bugfix][Engine] Group overlapping device stages into one sequential init component (#7328) 
Signed-off-by: ZhengWG <zwg0606@gmail.com>
Co-authored-by: amy-why-3459 <wuhaiyan17@huawei.com>
 2026-09-09 20:25:41 +08:00
zyz111222 6e79d582e3 [Bugfix][Ascend] fix npu 310/a5 bugs (#6685) 
Signed-off-by: zouyizhou <zouyizhou@huawei.com>
 2026-09-09 19:03:09 +08:00
NumberWan 038c9b948f [Bugfix] Resolve #6931 hub FA3 on torch 2.13 via kernels 0.16.1 (#7185) 
Signed-off-by: NumberWan <wantszkin2003@gmail.com>
 2026-09-09 10:20:34 +00:00
Bo Li cdd4638b17 [Feature][Diffusion] Expose detailed pipeline timings (#6822) 
Signed-off-by: Bo Li <22713281+bobboli@users.noreply.github.com>
 2026-09-09 09:58:42 +00:00
Anjie HouandClaude Sonnet 5 ecdd762ff5 [2/N] Add a minimal temporal chunk callback for MiniMax-H3 (#7017) 
Signed-off-by: specture724 <specture724@gmail.com>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
 2026-09-09 16:40:32 +08:00
Mu GuanLin ffa991b24f [Perf][Model] Add optimized LTX-2.5 DiffVAE operators (#7308) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 08:31:16 +00:00
chi030303 c66df06c0f [Doc] Qwen3-TTS: add 0.6B on 1x A100 40GB (#7289) 
Signed-off-by: chi030303 <106855944+chi030303@users.noreply.github.com>
 2026-09-09 07:56:14 +00:00
Sparks 7712701578 [Doc] [Recipe] Add FLUX.1-schnell recipe for RTX 5090 32GB (#7299) 
Signed-off-by: Sparks-M <41097544+Sparks-M@users.noreply.github.com>
 2026-09-09 07:54:46 +00:00
Mu GuanLin ab56148505 [Bugfix] Make LTX vocoder decoding deterministic (#7231) 
Signed-off-by: mglyn <1203789601@qq.com>
 2026-09-09 06:37:01 +00:00
NATURE 5b002727b0 [Core] Split Omni connector model runner mixin (#6903) 
Signed-off-by: natureofnature <wzliu@connect.hku.hk>
 2026-09-09 13:49:43 +08:00
andyluo7 e0d0879794 [Bugfix][Magi2] Allow import without an active Triton driver (#7239) 
Signed-off-by: andyluo7 <andy.luo@amd.com>
 2026-09-08 22:48:08 -07:00
wangyu 09b720b0d2 [CI] Skip Qwen3-Omni Server VAD multi-turn realtime test (#7279) (#7314) 
Signed-off-by: wangyu <410167048@qq.com>
 2026-09-09 12:23:29 +08:00
xutianle 968f3e379c [Bugfix][CosyVoice3] Resolve hash snapshot pipeline (#6896) 
Signed-off-by: xutianle <xutianle@fudan.edu.cn>
 2026-09-08 20:33:38 -07:00
Allen WuandTRAE CLI f2e30f46b6 [Bugfix][TTS] Isolate shared Higgs v3 reference encode from request cancellation (#7076) 
Signed-off-by: Allen Wu <allenwu2795@gmail.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
 2026-09-08 20:32:05 -07:00
Jim Ban 358b7d79c1 [Bugfix][Frontend] Enforce image pixel limits for video input references (#6963) 
Signed-off-by: BANANASJIM <bananasjim1@gmail.com>
 2026-09-08 20:31:54 -07:00
Ziming Wang 43b697541a [Bugfix][Frontend] Clear the P0 multimodal cache through the renderer (#7003) 
Signed-off-by: ZenAlexa <zimingwang945@gmail.com>
 2026-09-08 20:31:42 -07:00