diff --git a/artifacts/pr7759-review/e2e_images/i2i_teapot_blue.png b/artifacts/pr7759-review/e2e_images/i2i_teapot_blue.png deleted file mode 100644 index f9ba01de4..000000000 Binary files a/artifacts/pr7759-review/e2e_images/i2i_teapot_blue.png and /dev/null differ diff --git a/artifacts/pr7759-review/e2e_images/t2i_teapot.png b/artifacts/pr7759-review/e2e_images/t2i_teapot.png deleted file mode 100644 index 0f62d99f4..000000000 Binary files a/artifacts/pr7759-review/e2e_images/t2i_teapot.png and /dev/null differ diff --git a/artifacts/pr7759-review/e2e_slurm.sh b/artifacts/pr7759-review/e2e_slurm.sh deleted file mode 100644 index a48597164..000000000 --- a/artifacts/pr7759-review/e2e_slurm.sh +++ /dev/null @@ -1,20 +0,0 @@ -#!/usr/bin/env bash -#SBATCH --partition=batch -#SBATCH --nodes=1 -#SBATCH --time=90 -#SBATCH --job-name=qi21-e2e -#SBATCH --output=/home/mozf/vllm-omni/artifacts/pr7759-review/e2e_slurm_%j.out - -set -euo pipefail -cd /home/mozf/vllm-omni - -export HF_HOME=/mnt/lustre/hf-models -export QWEN_IMAGE_21_TEST_MODEL=Qwen/Qwen-Image-2.1 -export VLLM_LOGGING_LEVEL=INFO - -echo "node: $(hostname)" -nvidia-smi -L | head -4 -.venv/bin/python -c 'import vllm; print("vllm", vllm.__version__)' - -.venv/bin/python -m pytest tests/e2e/online_serving/test_qwen_image_21.py \ - -o addopts="" -v -s 2>&1 diff --git a/artifacts/pr7759-review/e2e_slurm_30092.out b/artifacts/pr7759-review/e2e_slurm_30092.out deleted file mode 100644 index a1b0e6453..000000000 --- a/artifacts/pr7759-review/e2e_slurm_30092.out +++ /dev/null @@ -1,742 +0,0 @@ -node: gb200-rack1-01 -GPU 0: NVIDIA GB200 (UUID: GPU-56e80c56-4d0c-e81e-db5f-70f33aab4e18) -GPU 1: NVIDIA GB200 (UUID: GPU-2ce1a9bc-dbf1-3bbd-116d-6b4e61022280) -GPU 2: NVIDIA GB200 (UUID: GPU-f7cdefd9-3d67-384f-dbc1-53bc44d93985) -GPU 3: NVIDIA GB200 (UUID: GPU-7c9bf698-f3f3-4634-9309-dc1e32f62fba) -vllm 0.29.0 -INFO 09-21 08:43:18 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed. -INFO 09-21 08:43:21 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL). -INFO 09-21 08:43:21 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM. -INFO 09-21 08:43:21 [patch.py:503] inductor factorable-divisibility patch: installed. -INFO 09-21 08:43:21 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms. -============================= test session starts ============================== -platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0 -- /home/mozf/vllm-omni/.venv/bin/python -cachedir: .pytest_cache -rootdir: /home/mozf/vllm-omni -configfile: pyproject.toml -plugins: asyncio-1.4.0, mock-3.15.1, anyio-4.14.2 -asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function -collecting ... INFO 09-21 08:43:27 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell -collected 4 items - -tests/e2e/online_serving/test_qwen_image_21.py::test_text_to_image[request] INFO 09-21 08:43:32 [scheduler.py:277] Chunked prefill is enabled with max_num_batched_tokens=2048. -INFO 09-21 08:43:32 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']) -/home/mozf/vllm-omni/vllm_omni/version.py:55: RuntimeWarning: vLLM and vLLM-Omni appear to have mismatched major/minor versions: - --> vLLM-Omni version 0.26.1.dev1+g76de642c5 - --> vLLM version 0.29.0 -This will likely cause compatibility issues. - warn_if_misaligned_vllm_version() -INFO 09-21 08:43:50 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed. -INFO 09-21 08:43:50 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL). -INFO 09-21 08:43:50 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM. -INFO 09-21 08:43:50 [patch.py:503] inductor factorable-divisibility patch: installed. -INFO 09-21 08:43:50 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms. -INFO 09-21 08:43:55 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell -INFO 09-21 08:43:57 [serve.py:313] Detected diffusion model: Qwen/Qwen-Image-2.1 -INFO 09-21 08:43:57 [logo.py:52] █ █ █▄ ▄█ ▄▀▀▀▀▄ █▄ ▄█ █▄ █ ▀█▀ -INFO 09-21 08:43:57 [logo.py:52] ▄▄ ▄█ █ █ █ ▀▄▀ █ ▄▄▄ █ █ █ ▀▄▀ █ █ ▀▄ █ █ -INFO 09-21 08:43:57 [logo.py:52] █▄█▀ █ █ █ █ █ █ █ █ █ ▀▄█ █ -INFO 09-21 08:43:57 [logo.py:52] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀ ▀▀▀▀ ▀ ▀ ▀ ▀ ▀▀▀ -INFO 09-21 08:43:57 [logo.py:52] -(APIServer pid=2856381) INFO 09-21 08:43:57 [api_utils.py:347] vLLM server version 0.29.0, serving model Qwen/Qwen-Image-2.1 -(APIServer pid=2856381) INFO 09-21 08:43:57 [api_utils.py:286] non-default args: {'model_tag': 'Qwen/Qwen-Image-2.1', 'host': '127.0.0.1', 'port': 45115, 'model': 'Qwen/Qwen-Image-2.1', 'max_num_seqs': 2} -(APIServer pid=2856381) INFO 09-21 08:43:58 [weight_utils.py:58] Using model weights format ['*'] -(APIServer pid=2856381) INFO 09-21 08:45:02 [weight_utils.py:90] Time spent downloading weights for Qwen/Qwen-Image-2.1: 64.568430 seconds -(APIServer pid=2856381) INFO 09-21 08:45:02 [omni_base.py:208] [AsyncOmni] Initializing with model Qwen/Qwen-Image-2.1 -(APIServer pid=2856381) INFO 09-21 08:45:02 [omni_engine_base.py:163] [OmniEngine] Initializing with model Qwen/Qwen-Image-2.1 -(APIServer pid=2856381) INFO 09-21 08:45:05 [config_factory.py:697] Resolved generic diffusion final_output_type='image' for model_class_name='QwenImage21Pipeline'. -(APIServer pid=2856381) INFO 09-21 08:45:05 [omni_engine_base.py:312] [OmniEngine] Launching Orchestrator thread with 1 stages -(APIServer pid=2856381) INFO 09-21 08:45:05 [weight_utils.py:58] Using model weights format ['*'] -(APIServer pid=2856381) INFO 09-21 08:45:05 [weight_utils.py:58] Using model weights format ['*'] -(APIServer pid=2856381) INFO 09-21 08:45:06 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']) -(APIServer pid=2856381) INFO 09-21 08:45:06 [diffusion_worker.py:332] Final IR op priority after setting vLLM-Omni overrides: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native']) -(APIServer pid=2856381) INFO 09-21 08:45:06 [diffusion_worker.py:344] Worker 0: Initialized device and distributed environment. -(APIServer pid=2856381) INFO 09-21 08:45:06 [parallel_state.py:610] Building SP subgroups from explicit sp_group_ranks (sp_size=1, ulysses=1, ring=1, use_ulysses_low=True). -(APIServer pid=2856381) INFO 09-21 08:45:06 [parallel_state.py:652] SP group details for rank 0: sp_group=[0], ulysses_group=[0], ring_group=[0] -(DiffusionWorker pid=2856381) INFO 09-21 08:45:07 [hub_prefetch.py:299] Prefetching Qwen/Qwen-Image-2.1 subfolders: ['scheduler', 'text_encoder', 'vae', 'processor'] -(DiffusionWorker pid=2856381) INFO 09-21 08:45:07 [hub_prefetch.py:224] Acquired flock prefetch lock for Qwen/Qwen-Image-2.1 at /mnt/lustre/hf-models/locks/vllm-omni-prefetch/Qwen__Qwen-Image-2.1.lock -(DiffusionWorker pid=2856381) Fetching 22 files: 0%| | 0/22 [00:00 vLLM-Omni version 0.26.1.dev1+g76de642c5 - --> vLLM version 0.29.0 -This will likely cause compatibility issues. - warn_if_misaligned_vllm_version() -INFO 09-21 08:45:52 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed. -INFO 09-21 08:45:53 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL). -INFO 09-21 08:45:53 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM. -INFO 09-21 08:45:53 [patch.py:503] inductor factorable-divisibility patch: installed. -INFO 09-21 08:45:53 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms. -INFO 09-21 08:45:57 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell -INFO 09-21 08:45:58 [serve.py:313] Detected diffusion model: Qwen/Qwen-Image-2.1 -INFO 09-21 08:45:58 [logo.py:52] █ █ █▄ ▄█ ▄▀▀▀▀▄ █▄ ▄█ █▄ █ ▀█▀ -INFO 09-21 08:45:58 [logo.py:52] ▄▄ ▄█ █ █ █ ▀▄▀ █ ▄▄▄ █ █ █ ▀▄▀ █ █ ▀▄ █ █ -INFO 09-21 08:45:58 [logo.py:52] █▄█▀ █ █ █ █ █ █ █ █ █ ▀▄█ █ -INFO 09-21 08:45:58 [logo.py:52] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀ ▀▀▀▀ ▀ ▀ ▀ ▀ ▀▀▀ -INFO 09-21 08:45:58 [logo.py:52] -(APIServer pid=2864057) INFO 09-21 08:45:58 [api_utils.py:347] vLLM server version 0.29.0, serving model Qwen/Qwen-Image-2.1 -(APIServer pid=2864057) INFO 09-21 08:45:58 [api_utils.py:286] non-default args: {'model_tag': 'Qwen/Qwen-Image-2.1', 'host': '127.0.0.1', 'port': 44729, 'model': 'Qwen/Qwen-Image-2.1', 'max_num_seqs': 2} -(APIServer pid=2864057) INFO 09-21 08:45:58 [weight_utils.py:58] Using model weights format ['*'] -(APIServer pid=2864057) INFO 09-21 08:45:58 [omni_base.py:208] [AsyncOmni] Initializing with model Qwen/Qwen-Image-2.1 -(APIServer pid=2864057) INFO 09-21 08:45:58 [omni_engine_base.py:163] [OmniEngine] Initializing with model Qwen/Qwen-Image-2.1 -(APIServer pid=2864057) INFO 09-21 08:46:00 [config_factory.py:697] Resolved generic diffusion final_output_type='image' for model_class_name='QwenImage21Pipeline'. -(APIServer pid=2864057) INFO 09-21 08:46:00 [omni_engine_base.py:312] [OmniEngine] Launching Orchestrator thread with 1 stages -(APIServer pid=2864057) INFO 09-21 08:46:00 [weight_utils.py:58] Using model weights format ['*'] -(APIServer pid=2864057) INFO 09-21 08:46:00 [weight_utils.py:58] Using model weights format ['*'] -(APIServer pid=2864057) INFO 09-21 08:46:01 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']) -(APIServer pid=2864057) INFO 09-21 08:46:01 [diffusion_worker.py:332] Final IR op priority after setting vLLM-Omni overrides: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native']) -(APIServer pid=2864057) INFO 09-21 08:46:01 [diffusion_worker.py:344] Worker 0: Initialized device and distributed environment. -(APIServer pid=2864057) INFO 09-21 08:46:01 [parallel_state.py:610] Building SP subgroups from explicit sp_group_ranks (sp_size=1, ulysses=1, ring=1, use_ulysses_low=True). -(APIServer pid=2864057) INFO 09-21 08:46:01 [parallel_state.py:652] SP group details for rank 0: sp_group=[0], ulysses_group=[0], ring_group=[0] -(DiffusionWorker pid=2864057) INFO 09-21 08:46:01 [hub_prefetch.py:299] Prefetching Qwen/Qwen-Image-2.1 subfolders: ['scheduler', 'text_encoder', 'vae', 'processor'] -(DiffusionWorker pid=2864057) INFO 09-21 08:46:01 [hub_prefetch.py:224] Acquired flock prefetch lock for Qwen/Qwen-Image-2.1 at /mnt/lustre/hf-models/locks/vllm-omni-prefetch/Qwen__Qwen-Image-2.1.lock -(DiffusionWorker pid=2864057) Fetching 22 files: 0%| | 0/22 [00:00 vLLM-Omni version 0.26.1.dev1+g76de642c5 - --> vLLM version 0.29.0 - This will likely cause compatibility issues. - warn_if_misaligned_vllm_version() - --- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ---- Running Summary -================== 4 passed, 15 warnings in 173.25s (0:02:53) ================== diff --git a/artifacts/pr7759-review/gen_images_30094.out b/artifacts/pr7759-review/gen_images_30094.out deleted file mode 100644 index d2c9d150d..000000000 --- a/artifacts/pr7759-review/gen_images_30094.out +++ /dev/null @@ -1,204 +0,0 @@ -node: gb200-rack1-01 -=== T2I === -/home/mozf/vllm-omni/vllm_omni/version.py:55: RuntimeWarning: vLLM and vLLM-Omni appear to have mismatched major/minor versions: - --> vLLM-Omni version 0.26.1.dev1+g76de642c5 - --> vLLM version 0.29.0 -This will likely cause compatibility issues. - warn_if_misaligned_vllm_version() -INFO 09-21 08:49:45 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed. -INFO 09-21 08:49:46 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL). -INFO 09-21 08:49:46 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM. -INFO 09-21 08:49:46 [patch.py:503] inductor factorable-divisibility patch: installed. -INFO 09-21 08:49:46 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms. -INFO 09-21 08:49:47 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell -INFO 09-21 08:49:49 [weight_utils.py:58] Using model weights format ['*'] -INFO 09-21 08:49:49 [omni_base.py:208] [Omni] Initializing with model Qwen/Qwen-Image-2.1 -INFO 09-21 08:49:49 [omni_engine_base.py:163] [OmniEngine] Initializing with model Qwen/Qwen-Image-2.1 -INFO 09-21 08:49:51 [config_factory.py:697] Resolved generic diffusion final_output_type='image' for model_class_name='QwenImage21Pipeline'. -INFO 09-21 08:49:51 [omni_engine_base.py:312] [OmniEngine] Launching Orchestrator thread with 1 stages -INFO 09-21 08:49:51 [stage_init_utils.py:775] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn -INFO 09-21 08:49:51 [weight_utils.py:58] Using model weights format ['*'] -INFO 09-21 08:49:51 [weight_utils.py:58] Using model weights format ['*'] -INFO 09-21 08:49:51 [scheduler.py:277] Chunked prefill is enabled with max_num_batched_tokens=2048. -INFO 09-21 08:49:51 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']) -INFO 09-21 08:49:51 [diffusion_worker.py:332] Final IR op priority after setting vLLM-Omni overrides: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native']) -INFO 09-21 08:49:51 [diffusion_worker.py:344] Worker 0: Initialized device and distributed environment. -INFO 09-21 08:49:51 [parallel_state.py:610] Building SP subgroups from explicit sp_group_ranks (sp_size=1, ulysses=1, ring=1, use_ulysses_low=True). -INFO 09-21 08:49:51 [parallel_state.py:652] SP group details for rank 0: sp_group=[0], ulysses_group=[0], ring_group=[0] -(DiffusionWorker pid=2877460) INFO 09-21 08:49:52 [hub_prefetch.py:299] Prefetching Qwen/Qwen-Image-2.1 subfolders: ['scheduler', 'text_encoder', 'vae', 'processor'] -(DiffusionWorker pid=2877460) INFO 09-21 08:49:52 [hub_prefetch.py:224] Acquired flock prefetch lock for Qwen/Qwen-Image-2.1 at /mnt/lustre/hf-models/locks/vllm-omni-prefetch/Qwen__Qwen-Image-2.1.lock -(DiffusionWorker pid=2877460) Fetching 22 files: 0%| | 0/22 [00:00 vLLM-Omni version 0.26.1.dev1+g76de642c5 - --> vLLM version 0.29.0 -This will likely cause compatibility issues. - warn_if_misaligned_vllm_version() -INFO 09-21 08:50:24 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed. -INFO 09-21 08:50:26 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL). -INFO 09-21 08:50:26 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM. -INFO 09-21 08:50:26 [patch.py:503] inductor factorable-divisibility patch: installed. -INFO 09-21 08:50:26 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms. -INFO 09-21 08:50:29 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell -INFO 09-21 08:50:31 [weight_utils.py:58] Using model weights format ['*'] -INFO 09-21 08:50:31 [omni_base.py:208] [Omni] Initializing with model Qwen/Qwen-Image-2.1 -INFO 09-21 08:50:31 [omni_engine_base.py:163] [OmniEngine] Initializing with model Qwen/Qwen-Image-2.1 -INFO 09-21 08:50:33 [config_factory.py:697] Resolved generic diffusion final_output_type='image' for model_class_name='QwenImage21Pipeline'. -INFO 09-21 08:50:33 [omni_engine_base.py:312] [OmniEngine] Launching Orchestrator thread with 1 stages -INFO 09-21 08:50:33 [stage_init_utils.py:775] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn -INFO 09-21 08:50:33 [weight_utils.py:58] Using model weights format ['*'] -INFO 09-21 08:50:33 [weight_utils.py:58] Using model weights format ['*'] -INFO 09-21 08:50:33 [scheduler.py:277] Chunked prefill is enabled with max_num_batched_tokens=2048. -INFO 09-21 08:50:33 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native']) -INFO 09-21 08:50:33 [diffusion_worker.py:332] Final IR op priority after setting vLLM-Omni overrides: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native']) -INFO 09-21 08:50:33 [diffusion_worker.py:344] Worker 0: Initialized device and distributed environment. -INFO 09-21 08:50:33 [parallel_state.py:610] Building SP subgroups from explicit sp_group_ranks (sp_size=1, ulysses=1, ring=1, use_ulysses_low=True). -INFO 09-21 08:50:33 [parallel_state.py:652] SP group details for rank 0: sp_group=[0], ulysses_group=[0], ring_group=[0] -(DiffusionWorker pid=2879936) INFO 09-21 08:50:34 [hub_prefetch.py:299] Prefetching Qwen/Qwen-Image-2.1 subfolders: ['scheduler', 'text_encoder', 'vae', 'processor'] -(DiffusionWorker pid=2879936) INFO 09-21 08:50:34 [hub_prefetch.py:224] Acquired flock prefetch lock for Qwen/Qwen-Image-2.1 at /mnt/lustre/hf-models/locks/vllm-omni-prefetch/Qwen__Qwen-Image-2.1.lock -(DiffusionWorker pid=2879936) Fetching 22 files: 0%| | 0/22 [00:00 - -## Codex Review Summary - -This comment shows the latest Codex review activity on this pull request. - -| Review | Status | Commit | Review trigger | -| --- | --- | --- | --- | -| 📝 **Code Review** | ✅ **Completed** 2026-09-18T02:12:54.670114Z | `3affc77` | PR opened | - - - -
ℹ️ About Codex in GitHub -
- -[Your team has set up Codex to review pull requests in this repo](https://chatgpt.com/codex/cloud/settings/general). Reviews are triggered when you -- Open a pull request for review -- Mark a draft as ready -- Comment "@codex review" or "@codex security review". - -Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. - -
- -=== vllm-omni-review-bot === -This PR appears to belong to: **docs/design/module/diffusion/index.md**, **docs/design/module/diffusion/parallelism.md**. - -Module owners: @princepride @xuechendi @Bounty-hunter - -Routing: @princepride via module of the changed files, module named in the PR description; @xuechendi via module of the changed files, module named in the PR description; @Bounty-hunter via module named in the PR description, CODEOWNERS - -@Isotr0py, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer. - -Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment. - - - -=== vllm-omni-review-bot === -## Omni ReviewBot triage note - -Resolved as of `a26013145cfa`: the high-priority or low-quality signal noted on an earlier commit no longer applies. - - - - - - -=== Bounty-hunter === -is prefix cache depend on https://github.com/vllm-project/vllm-omni/pull/7719 ? - -=== SamitHuang === -Pls resolve the conflicts for further review, thx - diff --git a/artifacts/pr7759-review/review_comments.txt b/artifacts/pr7759-review/review_comments.txt deleted file mode 100644 index b68f3e163..000000000 --- a/artifacts/pr7759-review/review_comments.txt +++ /dev/null @@ -1,93 +0,0 @@ -=== id:4043203379 | chatgpt-codex-connector[bot] | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:1247 | reply_to:null === -**![P1 Badge](https://img.shields.io/badge/P1-orange?style=flat) Keep new requests out of populated KV-cache batches** - -With step execution and `max_num_seqs > 1`, the scheduler explicitly admits compatible waiting requests alongside already-running requests, so a request arriving after the first denoise step produces exactly this mixed `prefill`/`decode` phase set and aborts the whole batch here. Since the recipe advertises continuous step execution with `--max-num-seqs 8`, the pipeline must either handle mixed phases or prevent such requests from joining an in-flight batch rather than raising. - -Useful? React with 👍 / 👎. - -=== id:4043203389 | chatgpt-codex-connector[bot] | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:193 | reply_to:null === -**![P1 Badge](https://img.shields.io/badge/P1-orange?style=flat) Partition batches by image-conditioning layout** - -The preprocessor never sets `request.batch_compatibility_key`, so the generic request and step schedulers consider text-only requests, image-conditioned requests, and requests with different condition-image layouts compatible when their sampling parameters match. Such requests are then placed in one batch and rejected later by `_prepare_generation_context` or `denoise_step`, turning otherwise valid concurrent requests into batch-wide failures; record the conditioning mode, image count, and resized slot sizes in the compatibility key on both the empty-image and image paths. - -Useful? React with 👍 / 👎. - -=== id:4043203392 | chatgpt-codex-connector[bot] | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:693 | reply_to:null === -**![P1 Badge](https://img.shields.io/badge/P1-orange?style=flat) Preserve request order when duplicating condition latents** - -When a request batch contains more than one prompt and `num_outputs_per_prompt > 1`, prompt embeddings are ordered per request (`A, A, B, B`), but concatenating the encoded batch repeatedly orders condition latents as `A, B, A, B`. The second and third generations therefore receive another request's condition image; duplicate each encoded row consecutively so its order matches the repeated prompt embeddings and output splitting. - -Useful? React with 👍 / 👎. - -=== id:4043203401 | chatgpt-codex-connector[bot] | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:147 | reply_to:null === -**![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat) Avoid truth-testing tensor and ndarray image inputs** - -For a condition image supplied as a multi-element `torch.Tensor` or `numpy.ndarray`, this truth-value check raises an ambiguity error before `_to_pil` can process it, despite `_to_pil` explicitly supporting both input types. Check explicitly for `None` or an empty list instead so these supported image representations reach conversion. - -Useful? React with 👍 / 👎. - -=== id:4043275121 | SamitHuang | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:283 | reply_to:null === -This pipeline is the entire new model surface (~1.3k lines here, plus transformer/VAE/CUDA graph) and the PR adds **zero tests**. Commits `4fc3cf8` / `3affc77` mention Qwen-Image-2.1 unit tests and then `clean tests` removes them from the branch. Add at least (1) unit tests for preprocess, `prepare_latents` duplication, mixed-phase `_assemble_kv_cache`, and VAE `spatial_compression_ratio=16`, and (2) a `tests/e2e/online_serving/test_qwen_image_21.py` smoke wired into `.buildkite/cuda/test-ready.yml` the same way `test_qwen_image.py` is. Without that, the TP / VAE-tiling / CUDA-graph claims in the commit messages are not merge-blocking evidence. - -=== id:4043275125 | SamitHuang | vllm_omni/diffusion/models/qwen_image_21/decode_graph.py:274 | reply_to:null === -`register_prefill` copies prefix K/V into a **shared** entry keyed only by `(branch, batch, prefix_len, target_tokens, dtype, device, backend)` and then **re-points every caller’s `kv_cache` dict at those tensors**. A second in-flight request with the same key `copy_`s over the first request’s prefix while the first request’s cache still aliases the same storage, so decode attends to another request’s text/image tokens. Sequential reuse of one buffer is fine; concurrent ownership is not. Keep per-request (or per-generation) static buffers, or refuse CUDA-graph decode whenever more than one live cache aliases the same key. This interacts directly with the recipe’s `--max-num-seqs 8` serving path. - -=== id:4043275130 | SamitHuang | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:693 | reply_to:null === -`encode_prompt` expands with `prompt_embeds.repeat(1, num_images_per_prompt, 1).view(batch * n, ...)` which is request-major (`A, A, B, B`). `torch.cat([encoded] * (batch_size // encoded.shape[0]), dim=0)` is tile-major (`A, B, A, B`). For a 2-prompt batch with `num_outputs_per_prompt=2`, generations 2 and 3 therefore get the other request’s condition latents (and `split_diffusion_output_by_request` will pair them as if they were correct). Use `repeat_interleave(repeats=n, dim=0)` (or encode-then-repeat per row) so condition latents match the prompt-embed order. This needs a unit test with `batch_size=2, n=2`. - -=== id:4043275138 | SamitHuang | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:147 | reply_to:null === -`if not raw_image` is an ambiguous truth-value check. `_to_pil` documents `torch.Tensor` and `numpy.ndarray` inputs; a multi-element tensor/array raises `RuntimeError: Boolean value of Tensor with more than one value is ambiguous` before conversion. Treat T2I as `raw_image is None or raw_image == []` (and maybe empty tuple), then always wrap a single image in a list. Same pattern should apply to whatever OpenAI image-edit path feeds tensors into `multi_modal_data["image"]`. - -=== id:4043275145 | SamitHuang | recipes/Qwen/Qwen-Image-2.1.md:107 | reply_to:null === -This example advertises `--step-execution --max-num-seqs 8`, but `QwenImage21Pipeline._assemble_kv_cache` raises if a new request’s first denoise step joins a batch that is already in decode (`Cannot batch requests at mixed KV-cache phases`). With the default step scheduler admitting waiting requests into a running batch, that is the normal continuous-batching case, not an edge case. Either keep mixed-phase requests out of the same `InputBatch` (and document the restriction here), or do not publish this command until that TODO is implemented. As written, following the recipe under load fails the whole batch. - -=== id:4043327545 | SamitHuang | vllm_omni/diffusion/data.py:909 | reply_to:null === -the CUDA graph for decode seems only used by QwenImage2.1? It's better to claim how it works together with eager mode or cudagraph_mode. - -=== id:4043342420 | SamitHuang | examples/offline_inference/text_to_image/text_to_image.py:1 | reply_to:null === -can provide a `examples/offline_inference/image_to_image/run_qwen_image_21.sh` script as well - -=== id:4043666922 | hsliuustc0106 | vllm_omni/diffusion/data.py:909 | reply_to:null === -This flag has no ingress, so the CUDA-graph decode path is unreachable for users: it's not in `OmniEngineArgs`/`StageDeployConfig`/`_DiffusionConfigProjection` (contrast `vae_use_tiling`, which flows through all three), and `Omni(enable_cuda_graph_decode=True)` is silently dropped by `from_kwargs`' field-name filter (`data.py:1604`). The only writer is the `False` default, so `decode_graph.py` + the transformer machinery only run under tests. Either wire it like `enforce_eager`/`vae_use_tiling` (deploy knob + projection) or document it as test-only in the recipe — the body's checked "[x] CUDA Graph for decode steps" reads as user-available today. - -=== id:4043667674 | hsliuustc0106 | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:1246 | reply_to:null === -Reachable under the recipe's own `--step-execution --max-num-seqs 8`, and it fails the whole wave, not just the joiner: the default scheduler admits waiting requests while running ones are mid-generation (`base_scheduler.py` schedules running first, then admits waiting while capacity remains; `_can_schedule_waiting` compares only `StepBatchSamplingParamsKey` — no KV phase), and a `denoise_step` exception terminally fails every scheduled request via `diffusion_engine.py`'s `Execution failed for diffusion requests ...` handler. So one fresh arrival converts all in-flight requests into errors. Fix direction: phase-aware admission (defer joins while any running request is past step 0) or a per-model `batch_compatibility_key`; until then the features-table ✅ for Step Execution and the recipe's serving block overstate support vs. the unchecked TODO box. - -=== id:4043668653 | hsliuustc0106 | examples/offline_inference/image_to_image/qwen_image_21_edit.py:1 | reply_to:null === -Blocking per the examples policy: a newly added model-specific Python example (the file names one model family), while this directory's convention is the generic runner (`image_edit.py`, which dispatches via `get_model_class_name`/`build_image_to_image_prompt` — the same helpers this script calls). Smallest fix: register a `QwenImage21Pipeline` spec in `model_extras/_EXTRA_SPECS` (it's missing, so `build_image_to_image_prompt` currently falls through to the generic builder and silently ignores the `height`/`width` args this script passes) and fold this into `image_edit.py`; drop the model-specific file. - -=== id:4043668703 | hsliuustc0106 | vllm_omni/diffusion/models/qwen_image_21/__init__.py:1 | reply_to:null === -No tests anywhere in the PR. Sibling new models each ship a `tests/diffusion/models//` suite (sensenova_u1, glm_image, boogu_image incl. registry tests, ovis_image, z_image), and no CI lane touches `qwen_image_21` today. Given the new surface — registry entry, pre/post-process funcs, KV-cache prefill/decode protocol, step-mode merge/scatter, graph fallback gates — at least a CPU-runnable registry + loading + pre/post-process test would protect it. - -=== id:4043668767 | hsliuustc0106 | recipes/Qwen/Qwen-Image-2.1.md:3 | reply_to:null === -Missing the head-vs-reference comparison the model-addition checklist requires: the body's Test Plan is empty and there's no diffusers-#14804 quality/latency A/B (same checkpoint, inputs, seed, precision, hardware) — the recipe's only number is the tile-size PSNR probe. The checked TODO boxes (TP=2/4 vs single-card output, VAE tiling, CUDA graphs) also have no attached measurements. Please add the reference A/B with exact commands and both sides' outputs, or state why it can't run yet. - -=== id:4044943764 | RuixiangMa | tests/diffusion/models/qwen_image_21/test_batch_compatibility.py:1 | reply_to:null === -Test coverage is insufficient. - -=== id:4045080762 | RuixiangMa | vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl_qwenimage21.py:2 | reply_to:null === -```suggestion -# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -``` - -=== id:4045084688 | RuixiangMa | vllm_omni/diffusion/models/qwen_image_21/__init__.py:2 | reply_to:null === -```suggestion -# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project -``` - -=== id:4059060032 | FayeSpica | vllm_omni/diffusion/models/qwen_image_21/qwen_image_21_transformer.py:388 | reply_to:null === -```suggestion - height_index = torch.where(height_index < 0, height_index + self.freqs[1].shape[0], height_index) - width_index = torch.where(width_index < 0, width_index + self.freqs[2].shape[0], width_index) - return torch.cat( - [ - self.freqs[0].index_select(0, frame_index), - self.freqs[1].index_select(0, height_index), - self.freqs[2].index_select(0, width_index), - ], - dim=-1, - ) -``` - -Could we use `torch.index_select` here for NPU compatibility, similar to #6320? This was the only issue preventing the model from starting on NPU in my tests. -