anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

ooops

Browse Source 
Signed-off-by: Isotr0py <Isotr0py@outlook.com>
This commit is contained in:
Isotr0py committed 2026-09-22 01:26:57 +00:00
1 parent 3e274a0024
commit c05a3527f5
8 files changed
-1153
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

artifacts/pr7759-review/e2e_images/i2i_teapot_blue.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 2.2 MiB

artifacts/pr7759-review/e2e_images/t2i_teapot.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 1.5 MiB

artifacts/pr7759-review/e2e_slurm.sh
-20
View File
@@ -1,20 +0,0 @@
#!/usr/bin/env bash
#SBATCH --partition=batch
#SBATCH --nodes=1
#SBATCH --time=90
#SBATCH --job-name=qi21-e2e
#SBATCH --output=/home/mozf/vllm-omni/artifacts/pr7759-review/e2e_slurm_%j.out
set -euo pipefail
cd /home/mozf/vllm-omni
export HF_HOME=/mnt/lustre/hf-models
export QWEN_IMAGE_21_TEST_MODEL=Qwen/Qwen-Image-2.1
export VLLM_LOGGING_LEVEL=INFO
echo "node: $(hostname)"
nvidia-smi -L | head -4
.venv/bin/python -c 'import vllm; print("vllm", vllm.__version__)'
.venv/bin/python -m pytest tests/e2e/online_serving/test_qwen_image_21.py \
-o addopts="" -v -s 2>&1
artifacts/pr7759-review/e2e_slurm_30092.out
-742
View File
@@ -1,742 +0,0 @@
node: gb200-rack1-01
GPU 0: NVIDIA GB200 (UUID: GPU-56e80c56-4d0c-e81e-db5f-70f33aab4e18)
GPU 1: NVIDIA GB200 (UUID: GPU-2ce1a9bc-dbf1-3bbd-116d-6b4e61022280)
GPU 2: NVIDIA GB200 (UUID: GPU-f7cdefd9-3d67-384f-dbc1-53bc44d93985)
GPU 3: NVIDIA GB200 (UUID: GPU-7c9bf698-f3f3-4634-9309-dc1e32f62fba)
vllm 0.29.0
INFO 09-21 08:43:18 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed.
INFO 09-21 08:43:21 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL).
INFO 09-21 08:43:21 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM.
INFO 09-21 08:43:21 [patch.py:503] inductor factorable-divisibility patch: installed.
INFO 09-21 08:43:21 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms.
============================= test session starts ==============================
platform linux -- Python 3.12.3, pytest-9.1.1, pluggy-1.6.0 -- /home/mozf/vllm-omni/.venv/bin/python
cachedir: .pytest_cache
rootdir: /home/mozf/vllm-omni
configfile: pyproject.toml
plugins: asyncio-1.4.0, mock-3.15.1, anyio-4.14.2
asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collecting ... INFO 09-21 08:43:27 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell
collected 4 items
tests/e2e/online_serving/test_qwen_image_21.py::test_text_to_image[request] INFO 09-21 08:43:32 [scheduler.py:277] Chunked prefill is enabled with max_num_batched_tokens=2048.
INFO 09-21 08:43:32 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
/home/mozf/vllm-omni/vllm_omni/version.py:55: RuntimeWarning: vLLM and vLLM-Omni appear to have mismatched major/minor versions:
--> vLLM-Omni version 0.26.1.dev1+g76de642c5
--> vLLM version 0.29.0
This will likely cause compatibility issues.
warn_if_misaligned_vllm_version()
INFO 09-21 08:43:50 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed.
INFO 09-21 08:43:50 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL).
INFO 09-21 08:43:50 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM.
INFO 09-21 08:43:50 [patch.py:503] inductor factorable-divisibility patch: installed.
INFO 09-21 08:43:50 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms.
INFO 09-21 08:43:55 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell
INFO 09-21 08:43:57 [serve.py:313] Detected diffusion model: Qwen/Qwen-Image-2.1
INFO 09-21 08:43:57 [logo.py:52] █ █ █▄ ▄█ ▄▀▀▀▀▄ █▄ ▄█ █▄ █ ▀█▀
INFO 09-21 08:43:57 [logo.py:52] ▄▄ ▄█ █ █ █ ▀▄▀ █ ▄▄▄ █ █ █ ▀▄▀ █ █ ▀▄ █ █
INFO 09-21 08:43:57 [logo.py:52] █▄█▀ █ █ █ █ █ █ █ █ █ ▀▄█ █
INFO 09-21 08:43:57 [logo.py:52] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀ ▀▀▀▀ ▀ ▀ ▀ ▀ ▀▀▀
INFO 09-21 08:43:57 [logo.py:52]
(APIServer pid=2856381) INFO 09-21 08:43:57 [api_utils.py:347] vLLM server version 0.29.0, serving model Qwen/Qwen-Image-2.1
(APIServer pid=2856381) INFO 09-21 08:43:57 [api_utils.py:286] non-default args: {'model_tag': 'Qwen/Qwen-Image-2.1', 'host': '127.0.0.1', 'port': 45115, 'model': 'Qwen/Qwen-Image-2.1', 'max_num_seqs': 2}
(APIServer pid=2856381) INFO 09-21 08:43:58 [weight_utils.py:58] Using model weights format ['*']
(APIServer pid=2856381) INFO 09-21 08:45:02 [weight_utils.py:90] Time spent downloading weights for Qwen/Qwen-Image-2.1: 64.568430 seconds
(APIServer pid=2856381) INFO 09-21 08:45:02 [omni_base.py:208] [AsyncOmni] Initializing with model Qwen/Qwen-Image-2.1
(APIServer pid=2856381) INFO 09-21 08:45:02 [omni_engine_base.py:163] [OmniEngine] Initializing with model Qwen/Qwen-Image-2.1
(APIServer pid=2856381) INFO 09-21 08:45:05 [config_factory.py:697] Resolved generic diffusion final_output_type='image' for model_class_name='QwenImage21Pipeline'.
(APIServer pid=2856381) INFO 09-21 08:45:05 [omni_engine_base.py:312] [OmniEngine] Launching Orchestrator thread with 1 stages
(APIServer pid=2856381) INFO 09-21 08:45:05 [weight_utils.py:58] Using model weights format ['*']
(APIServer pid=2856381) INFO 09-21 08:45:05 [weight_utils.py:58] Using model weights format ['*']
(APIServer pid=2856381) INFO 09-21 08:45:06 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=2856381) INFO 09-21 08:45:06 [diffusion_worker.py:332] Final IR op priority after setting vLLM-Omni overrides: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'])
(APIServer pid=2856381) INFO 09-21 08:45:06 [diffusion_worker.py:344] Worker 0: Initialized device and distributed environment.
(APIServer pid=2856381) INFO 09-21 08:45:06 [parallel_state.py:610] Building SP subgroups from explicit sp_group_ranks (sp_size=1, ulysses=1, ring=1, use_ulysses_low=True).
(APIServer pid=2856381) INFO 09-21 08:45:06 [parallel_state.py:652] SP group details for rank 0: sp_group=[0], ulysses_group=[0], ring_group=[0]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:07 [hub_prefetch.py:299] Prefetching Qwen/Qwen-Image-2.1 subfolders: ['scheduler', 'text_encoder', 'vae', 'processor']
(DiffusionWorker pid=2856381) INFO 09-21 08:45:07 [hub_prefetch.py:224] Acquired flock prefetch lock for Qwen/Qwen-Image-2.1 at /mnt/lustre/hf-models/locks/vllm-omni-prefetch/Qwen__Qwen-Image-2.1.lock
(DiffusionWorker pid=2856381) ␍Fetching 22 files: 0%| | 0/22 [00:00<?, ?it/s]␍Fetching 22 files: 100%|██████████| 22/22 [00:00<00:00, 1912.94it/s]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:07 [hub_prefetch.py:344] Prefetch complete for Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2856381) /home/mozf/vllm-omni/.venv/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py:205: UserWarning: The `local_dir_use_symlinks` argument is deprecated and ignored in `hf_hub_download`. Downloading to a local directory does not use symlinks anymore.
(DiffusionWorker pid=2856381) warnings.warn(
(DiffusionWorker pid=2856381) ␍Fetching 4 files: 0%| | 0/4 [00:00<?, ?it/s]␍Fetching 4 files: 100%|██████████| 4/4 [00:00<00:00, 1459.90it/s]
(DiffusionWorker pid=2856381) ␍Loading weights: 0%| | 0/750 [00:00<?, ?it/s]␍Loading weights: 0%| | 2/750 [00:00<02:10, 5.73it/s]␍Loading weights: 2%|▏ | 17/750 [00:00<00:15, 46.47it/s]␍Loading weights: 4%|▍ | 32/750 [00:00<00:09, 73.11it/s]␍Loading weights: 6%|▋ | 48/750 [00:00<00:07, 92.32it/s]␍Loading weights: 8%|▊ | 61/750 [00:00<00:07, 96.35it/s]␍Loading weights: 10%|▉ | 73/750 [00:00<00:06, 101.61it/s]␍Loading weights: 12%|█▏ | 89/750 [00:01<00:05, 115.39it/s]␍Loading weights: 14%|█▎ | 103/750 [00:01<00:05, 120.74it/s]␍Loading weights: 15%|█▌ | 116/750 [00:01<00:05, 107.27it/s]␍Loading weights: 17%|█▋ | 131/750 [00:01<00:05, 116.83it/s]␍Loading weights: 20%|█▉ | 147/750 [00:01<00:04, 121.08it/s]␍Loading weights: 21%|██▏ | 160/750 [00:01<00:05, 116.44it/s]␍Loading weights: 24%|██▎ | 178/750 [00:01<00:04, 132.80it/s]␍Loading weights: 26%|██▌ | 192/750 [00:01<00:04, 121.63it/s]␍Loading weights: 27%|██▋ | 205/750 [00:01<00:04, 121.55it/s]␍Loading weights: 29%|██▉ | 219/750 [00:02<00:04, 125.35it/s]␍Loading weights: 31%|███ | 234/750 [00:02<00:04, 127.29it/s]␍Loading weights: 33%|███▎ | 248/750 [00:02<00:03, 128.37it/s]␍Loading weights: 35%|███▌ | 265/750 [00:02<00:03, 138.24it/s]␍Loading weights: 37%|███▋ | 279/750 [00:02<00:03, 128.84it/s]␍Loading weights: 39%|███▉ | 293/750 [00:02<00:03, 130.26it/s]␍Loading weights: 41%|████ | 307/750 [00:02<00:03, 128.54it/s]␍Loading weights: 43%|████▎ | 323/750 [00:02<00:03, 132.60it/s]␍Loading weights: 45%|████▍ | 337/750 [00:02<00:03, 132.95it/s]␍Loading weights: 47%|████▋ | 353/750 [00:03<00:02, 140.36it/s]␍Loading weights: 49%|████▉ | 368/750 [00:03<00:02, 133.40it/s]␍Loading weights: 51%|█████ | 382/750 [00:03<00:02, 133.67it/s]␍Loading weights: 56%|█████▌ | 419/750 [00:03<00:01, 199.39it/s]␍Loading weights: 59%|█████▉ | 443/750 [00:03<00:01, 209.10it/s]␍Loading weights: 81%|████████ | 607/750 [00:03<00:00, 616.28it/s]␍Loading weights: 97%|█████████▋| 727/750 [00:03<00:00, 768.60it/s]␍Loading weights: 100%|██████████| 750/750 [00:03<00:00, 199.34it/s]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:13 [platform.py:338] Defaulting to diffusion attention backend CUDNN_ATTN (Blackwell sm_100, cuDNN 92000, head_dim 128)
(DiffusionWorker pid=2856381) INFO 09-21 08:45:13 [selector.py:93] Resolved diffusion attention backend 'CUDNN_ATTN' for role='self' via platform default
(DiffusionWorker pid=2856381) INFO 09-21 08:45:15 [weight_utils.py:58] Using model weights format ['transformer/diffusion_pytorch_model-00001-of-00002.safetensors', 'transformer/diffusion_pytorch_model-00002-of-00002.safetensors']
(DiffusionWorker pid=2856381) ␍Multi-thread loading shards: 0% Completed | 0/2 [00:00<?, ?it/s]
(DiffusionWorker pid=2856381) ␍Multi-thread loading shards: 50% Completed | 1/2 [00:00<00:00, 1.35it/s]
(DiffusionWorker pid=2856381) ␍Multi-thread loading shards: 100% Completed | 2/2 [00:02<00:00, 1.11s/it]
(DiffusionWorker pid=2856381) ␍Multi-thread loading shards: 100% Completed | 2/2 [00:02<00:00, 1.05s/it]
(DiffusionWorker pid=2856381)
(DiffusionWorker pid=2856381) INFO 09-21 08:45:17 [diffusers_loader.py:1098] Loading weights took 2.17 seconds
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [diffusion_model_runner.py:351] Model loading took 30.2190 GiB and 11.749674 seconds
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [diffusion_model_runner.py:357] Model runner: Model loaded successfully.
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [diffusion_model_runner.py:243] Model runner: transformer uses CUDA graph decode instead of torch.compile.
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [diffusion_model_runner.py:448] Model runner: Initialization complete.
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [diffusion_worker.py:425] Worker 0: Process-scoped GPU memory after model loading: 31.01 GiB.
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [manager.py:112] Initializing DiffusionLoRAManager: device=cuda:0, dtype=torch.bfloat16, max_cached_adapters=1, static_lora_path=None
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [diffusion_worker.py:302] Worker 0: Initialization complete.
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [uniproc_executor.py:74] Diffusion worker initialized in-process (uniproc executor)
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [diffusion_engine.py:402] [RequestBatch] engine init max_num_seqs=2 max_wait_ms=0.0
(DiffusionWorker pid=2856381) INFO 09-21 08:45:18 [diffusion_engine.py:1289] dummy run to warm up the model
(DiffusionWorker pid=2856381) ␍ 0%| | 0/2 [00:00<?, ?it/s]␍ 50%|█████ | 1/2 [00:02<00:02, 2.16s/it](DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 10, ((1, 32, 32),), torch.bfloat16, 0, 'CUDNN_ATTN')
␍100%|██████████| 2/2 [00:02<00:00, 1.01s/it]␍100%|██████████| 2/2 [00:02<00:00, 1.18s/it]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [inline_stage_diffusion_client.py:82] [InlineStageDiffusionClient] stage-0 [rep-0] initialized inline (max_num_seqs=2)
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [stage_runtime.py:1226] [StageRuntime] Stage 0 replica 0 initialized (diffusion)
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [orchestrator.py:420] [Orchestrator] Starting event loop
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [omni_engine_base.py:344] [OmniEngine] Orchestrator ready with 1 stages
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [omni_base.py:227] [AsyncOmni] AsyncOmniEngine initialized in 19.83 seconds
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [omni_base.py:246] [AsyncOmni] Initialized with 1 stages for model Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [api_server.py:767] Detected pure diffusion mode (single diffusion stage)
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [serving_video.py:170] Video response frame conversion pool configured: workers=8
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [speaker_cache.py:239] Speaker cache ready (max_bytes=536870912)
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [serving_speech.py:264] Speaker storage: dir=/home/mozf/.cache/vllm-omni/speakers, max_speakers=1000, restored=0
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [serving.py:91] Robot OpenPI serving disabled for model Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [api_server.py:855] Pure diffusion API server initialized for model: Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [api_server.py:367] Starting vLLM API server (pure diffusion mode) on http://127.0.0.1:45115
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:61] Available routes are:
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /openapi.json, Methods: HEAD, GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /docs, Methods: HEAD, GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /docs/oauth2-redirect, Methods: HEAD, GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /redoc, Methods: HEAD, GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /load, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /version, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /metrics, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /tokenize, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /detokenize, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /ping, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /ping, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /invocations, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/responses, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/responses/{response_id}, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/responses/{response_id}/cancel, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/completions, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/messages, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/messages/count_tokens, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /generative_scoring, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /scale_elastic_ep, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /is_scaling_elastic_ep, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/chat/completions/render, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/messages/render, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/completions/render, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/chat/completions/derender, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/completions/derender, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /inference/v1/generate, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/chat/completions, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/chat/completions/batch, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/audio/speech, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/audio/speech/batch, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/audio/generate, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/audio/voices, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/audio/voices, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/audio/voices/{name}, Methods: DELETE
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/realtime/sessions, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/realtime/sessions/{session_id}/step, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/realtime/sessions/{session_id}/reset, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/realtime/sessions/{session_id}/close, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/realtime/sessions/{session_id}/status, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /health, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/models, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/images/generations, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/images/edits, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/videos, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/videos/sync, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/videos, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/videos/{video_id}, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/videos/{video_id}, Methods: DELETE
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/videos/{video_id}/content, Methods: GET
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/omni/sleep, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:70] Route: /v1/omni/wakeup, Methods: POST
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:81] Route: /v1/audio/speech/stream, Endpoint: streaming_speech
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:81] Route: /v1/video/chat/stream, Endpoint: streaming_video_chat
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:81] Route: /v1/realtime/video, Endpoint: streaming_video_output
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:81] Route: /v1/realtime, Endpoint: realtime_websocket
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:81] Route: /v1/duplex, Endpoint: duplex_websocket
(DiffusionWorker pid=2856381) INFO 09-21 08:45:22 [launcher.py:81] Route: /v1/realtime/robot/openpi, Endpoint: realtime_robot_openpi
(DiffusionWorker pid=2856381) INFO: Started server process [2856381]
(DiffusionWorker pid=2856381) INFO: Waiting for application startup.
(DiffusionWorker pid=2856381) INFO: Application startup complete.
Pre-test device status:
[Device Memory Monitor] Waiting for device(s) 0, 1, 2, 3 to free memory, Condition: Memory usage ratio ≤ 5.0%
[Device Memory Status] Current usage:
Device 0: 0.7GiB/184.3GiB (0.4%)
Device 1: 0.7GiB/184.3GiB (0.4%)
Device 2: 0.7GiB/184.3GiB (0.4%)
Device 3: 0.7GiB/184.3GiB (0.4%)
[Device Memory Freed] Device(s) 0, 1, 2, 3 meet memory condition
Condition: Memory usage ratio ≤ 5.0%
Wait time: 0.6 seconds (0.0 minutes)
Post-test device status:
================================================================================
NVIDIA GPU Information (nvidia-smi)
================================================================================
Mon Sep 21 08:43:32 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.09 Driver Version: 580.126.09 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GB200 Off | 00000000:89:00.0 Off | 0 |
| N/A 26C P0 212W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GB200 Off | 00000000:8A:00.0 Off | 0 |
| N/A 25C P0 213W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA GB200 Off | 00000000:C1:00.0 Off | 0 |
| N/A 26C P0 176W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
... (showing first 20 of 35 lines)
================================================================================
Detailed GPU Processes (nvidia-smi pmon)
================================================================================
# gpu pid type sm mem enc dec jpg ofa command
# Idx # C/G % % % % % % name
0 2853490 C 0 0 - - - - python
1 2853490 C 0 0 - - - - python
2 2853490 C 0 0 - - - - python
3 2853490 C 0 0 - - - - python
Launching OmniServer with: /home/mozf/vllm-omni/.venv/bin/python -m vllm_omni.entrypoints.cli.main serve Qwen/Qwen-Image-2.1 --host 127.0.0.1 --port 45115 --omni --vae-use-tiling --max-num-seqs 2 --stage-init-timeout 600 --init-timeout 900 --log-stats
Server ready on 127.0.0.1:45115 (OmniServer startup took 112.093s)
(DiffusionWorker pid=2856381) INFO 09-21 08:45:28 [serving_chat.py:3651] Diffusion chat request chatcmpl-a2e0b3f285fd47cf: ref_images=0, params={'num_inference_steps': 4, 'true_cfg_scale': 4.0, 'width': 512, 'height': 512, 'negative_prompt': 'blurry, low quality', 'seed': 42}
(DiffusionWorker pid=2856381) ␍ 0%| | 0/4 [00:00<?, ?it/s]␍ 25%|██▌ | 1/4 [00:00<00:02, 1.50it/s](DiffusionWorker pid=2856381) INFO 09-21 08:45:29 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 18, ((1, 32, 32),), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('uncond', 1, 13, ((1, 32, 32),), torch.bfloat16, 0, 'CUDNN_ATTN')
␍ 50%|█████ | 2/4 [00:01<00:01, 1.89it/s]␍100%|██████████| 4/4 [00:01<00:00, 3.39it/s]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811] [Overall Summary]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811] +-----------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811] | Field | Value |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811] +-----------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811] | e2e_requests | 1 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811] | e2e_wall_time_ms | 1,390.033 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811] | e2e_avg_time_per_request_ms | 1,390.033 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811] | e2e_stage_0_wall_time_ms | 1,389.915 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:811] +-----------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:837]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:837] [RequestE2EStats [request_id=chatcmpl-a2e0b3f285fd47cf-940070ef]]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:837] +--------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:837] | Field | Value |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:837] +--------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:837] | e2e_total_ms | 1,389.915 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:837] +--------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:878] [OmniTiming] req=chatcmpl-a2e0b3f285fd47cf-940070ef total=1.39s engine=1.39s stages=[0:1.39s]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] [StageRequestStats [request_id=chatcmpl-a2e0b3f285fd47cf-940070ef]]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] +---------------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | Field | 0 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] +---------------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | batch_id | 1 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | batch_size | 1 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | denoise_step_latency_ms | 347.310 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | diffusion_engine_exec_time_s | 1.382 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | height | 512.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | image_num | 1.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | image_pixels | 262,144 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | image_time_to_first_output_ms | 1,389.218 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | inter_output_latencies_ms | |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | kv_recv_time_s | 0.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | num_inference_steps | 4.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | output_unit_count | 1 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | output_unit_type | image |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | postprocess_time_s | 0.007 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | preprocess_time_s | 0.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | resolution | 640.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | scheduler_queue_wait_s | 0.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | serving_time_to_first_output_ms | 1,389.586 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | stage_gen_time_ms | 1,389.240 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | vllm_itls_ms | |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] | width | 512.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [stats.py:920] +---------------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [serving_chat.py:3943] Diffusion chat completed for request chatcmpl-a2e0b3f285fd47cf: output_type=image, image_count=1
(DiffusionWorker pid=2856381) INFO: 127.0.0.1:58082 - "POST /v1/chat/completions HTTP/1.1" 200 OK
OmniServer started successfully
--- Running test: test_text_to_image[request]
[diffusion] request#1 success in 5.256s
PASSED
tests/e2e/online_serving/test_qwen_image_21.py::test_image_edit[request] (DiffusionWorker pid=2856381) INFO 09-21 08:45:30 [serving_chat.py:3651] Diffusion chat request chatcmpl-b801b6b9b3664cfd: ref_images=1, params={'num_inference_steps': 4, 'true_cfg_scale': 4.0, 'width': 512, 'height': 512, 'negative_prompt': 'blurry, low quality', 'seed': 42}
(DiffusionWorker pid=2856381) ␍ 0%| | 0/4 [00:00<?, ?it/s]␍ 25%|██▌ | 1/4 [00:01<00:04, 1.39s/it](DiffusionWorker pid=2856381) INFO 09-21 08:45:32 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 4115, ((1, 64, 64), (1, 32, 32)), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('uncond', 1, 4115, ((1, 64, 64), (1, 32, 32)), torch.bfloat16, 0, 'CUDNN_ATTN')
␍ 50%|█████ | 2/4 [00:01<00:01, 1.25it/s]␍100%|██████████| 4/4 [00:01<00:00, 3.02it/s]␍100%|██████████| 4/4 [00:01<00:00, 2.13it/s]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811] [Overall Summary]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811] +-----------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811] | Field | Value |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811] +-----------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811] | e2e_requests | 1 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811] | e2e_wall_time_ms | 2,587.491 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811] | e2e_avg_time_per_request_ms | 2,587.491 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811] | e2e_stage_0_wall_time_ms | 2,587.400 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:811] +-----------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:837]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:837] [RequestE2EStats [request_id=chatcmpl-b801b6b9b3664cfd-9f06e779]]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:837] +--------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:837] | Field | Value |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:837] +--------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:837] | e2e_total_ms | 2,587.400 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:837] +--------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:878] [OmniTiming] req=chatcmpl-b801b6b9b3664cfd-9f06e779 total=2.59s engine=2.59s stages=[0:2.59s]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] [StageRequestStats [request_id=chatcmpl-b801b6b9b3664cfd-9f06e779]]
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] +---------------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | Field | 0 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] +---------------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | batch_id | 2 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | batch_size | 1 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | denoise_step_latency_ms | 646.529 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | diffusion_engine_exec_time_s | 2.530 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | height | 512.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | image_num | 1.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | image_pixels | 262,144 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | image_time_to_first_output_ms | 2,586.083 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | inter_output_latencies_ms | |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | kv_recv_time_s | 0.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | num_inference_steps | 4.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | output_unit_count | 1 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | output_unit_type | image |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | postprocess_time_s | 0.013 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | preprocess_time_s | 0.042 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | resolution | 640.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | scheduler_queue_wait_s | 0.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | serving_time_to_first_output_ms | 2,586.323 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | stage_gen_time_ms | 2,586.114 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | vllm_itls_ms | |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] | width | 512.000 |
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stats.py:920] +---------------------------------+-----------+
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [serving_chat.py:3943] Diffusion chat completed for request chatcmpl-b801b6b9b3664cfd: output_type=image, image_count=1
(DiffusionWorker pid=2856381) INFO: 127.0.0.1:39992 - "POST /v1/chat/completions HTTP/1.1" 200 OK
--- Running test: test_image_edit[request]
[diffusion] request#1 success in 2.645s
PASSED
tests/e2e/online_serving/test_qwen_image_21.py::test_text_to_image[step] (DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [launcher.py:124] [shutdown] API server: shutdown triggered
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [launcher.py:140] [shutdown] API server: stopping engine client mode=abort timeout=0.0s
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [omni_base.py:771] [AsyncOmni] Shutting down
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [omni_engine_base.py:1144] [OmniEngine] Shutting down Orchestrator
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [orchestrator.py:512] [Orchestrator] Received shutdown signal
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [orchestrator.py:2720] [Orchestrator] Shutting down all 1 client(s)
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [stage_pool.py:1362] [StagePool] Stage 0 replica 0 shut down
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [launcher.py:149] [shutdown] API server: engine client stopped
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [launcher.py:152] [shutdown] API server: signalling HTTP server shutdown
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [launcher.py:173] [shutdown] API server: shutting down FastAPI HTTP server
(DiffusionWorker pid=2856381) INFO: Shutting down
(DiffusionWorker pid=2856381) INFO: Waiting for application shutdown.
(DiffusionWorker pid=2856381) INFO: Application shutdown complete.
(DiffusionWorker pid=2856381) INFO 09-21 08:45:33 [omni_base.py:771] [AsyncOmni] Shutting down
/home/mozf/vllm-omni/vllm_omni/version.py:55: RuntimeWarning: vLLM and vLLM-Omni appear to have mismatched major/minor versions:
--> vLLM-Omni version 0.26.1.dev1+g76de642c5
--> vLLM version 0.29.0
This will likely cause compatibility issues.
warn_if_misaligned_vllm_version()
INFO 09-21 08:45:52 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed.
INFO 09-21 08:45:53 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL).
INFO 09-21 08:45:53 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM.
INFO 09-21 08:45:53 [patch.py:503] inductor factorable-divisibility patch: installed.
INFO 09-21 08:45:53 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms.
INFO 09-21 08:45:57 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell
INFO 09-21 08:45:58 [serve.py:313] Detected diffusion model: Qwen/Qwen-Image-2.1
INFO 09-21 08:45:58 [logo.py:52] █ █ █▄ ▄█ ▄▀▀▀▀▄ █▄ ▄█ █▄ █ ▀█▀
INFO 09-21 08:45:58 [logo.py:52] ▄▄ ▄█ █ █ █ ▀▄▀ █ ▄▄▄ █ █ █ ▀▄▀ █ █ ▀▄ █ █
INFO 09-21 08:45:58 [logo.py:52] █▄█▀ █ █ █ █ █ █ █ █ █ ▀▄█ █
INFO 09-21 08:45:58 [logo.py:52] ▀▀ ▀▀▀▀▀ ▀▀▀▀▀ ▀ ▀ ▀▀▀▀ ▀ ▀ ▀ ▀ ▀▀▀
INFO 09-21 08:45:58 [logo.py:52]
(APIServer pid=2864057) INFO 09-21 08:45:58 [api_utils.py:347] vLLM server version 0.29.0, serving model Qwen/Qwen-Image-2.1
(APIServer pid=2864057) INFO 09-21 08:45:58 [api_utils.py:286] non-default args: {'model_tag': 'Qwen/Qwen-Image-2.1', 'host': '127.0.0.1', 'port': 44729, 'model': 'Qwen/Qwen-Image-2.1', 'max_num_seqs': 2}
(APIServer pid=2864057) INFO 09-21 08:45:58 [weight_utils.py:58] Using model weights format ['*']
(APIServer pid=2864057) INFO 09-21 08:45:58 [omni_base.py:208] [AsyncOmni] Initializing with model Qwen/Qwen-Image-2.1
(APIServer pid=2864057) INFO 09-21 08:45:58 [omni_engine_base.py:163] [OmniEngine] Initializing with model Qwen/Qwen-Image-2.1
(APIServer pid=2864057) INFO 09-21 08:46:00 [config_factory.py:697] Resolved generic diffusion final_output_type='image' for model_class_name='QwenImage21Pipeline'.
(APIServer pid=2864057) INFO 09-21 08:46:00 [omni_engine_base.py:312] [OmniEngine] Launching Orchestrator thread with 1 stages
(APIServer pid=2864057) INFO 09-21 08:46:00 [weight_utils.py:58] Using model weights format ['*']
(APIServer pid=2864057) INFO 09-21 08:46:00 [weight_utils.py:58] Using model weights format ['*']
(APIServer pid=2864057) INFO 09-21 08:46:01 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
(APIServer pid=2864057) INFO 09-21 08:46:01 [diffusion_worker.py:332] Final IR op priority after setting vLLM-Omni overrides: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'])
(APIServer pid=2864057) INFO 09-21 08:46:01 [diffusion_worker.py:344] Worker 0: Initialized device and distributed environment.
(APIServer pid=2864057) INFO 09-21 08:46:01 [parallel_state.py:610] Building SP subgroups from explicit sp_group_ranks (sp_size=1, ulysses=1, ring=1, use_ulysses_low=True).
(APIServer pid=2864057) INFO 09-21 08:46:01 [parallel_state.py:652] SP group details for rank 0: sp_group=[0], ulysses_group=[0], ring_group=[0]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:01 [hub_prefetch.py:299] Prefetching Qwen/Qwen-Image-2.1 subfolders: ['scheduler', 'text_encoder', 'vae', 'processor']
(DiffusionWorker pid=2864057) INFO 09-21 08:46:01 [hub_prefetch.py:224] Acquired flock prefetch lock for Qwen/Qwen-Image-2.1 at /mnt/lustre/hf-models/locks/vllm-omni-prefetch/Qwen__Qwen-Image-2.1.lock
(DiffusionWorker pid=2864057) ␍Fetching 22 files: 0%| | 0/22 [00:00<?, ?it/s]␍Fetching 22 files: 100%|██████████| 22/22 [00:00<00:00, 2099.82it/s]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:01 [hub_prefetch.py:344] Prefetch complete for Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2864057) /home/mozf/vllm-omni/.venv/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py:205: UserWarning: The `local_dir_use_symlinks` argument is deprecated and ignored in `hf_hub_download`. Downloading to a local directory does not use symlinks anymore.
(DiffusionWorker pid=2864057) warnings.warn(
(DiffusionWorker pid=2864057) ␍Fetching 4 files: 0%| | 0/4 [00:00<?, ?it/s]␍Fetching 4 files: 100%|██████████| 4/4 [00:00<00:00, 1604.55it/s]
(DiffusionWorker pid=2864057) ␍Loading weights: 0%| | 0/750 [00:00<?, ?it/s]␍Loading weights: 3%|▎ | 26/750 [00:00<00:03, 241.16it/s]␍Loading weights: 7%|▋ | 51/750 [00:00<00:03, 232.24it/s]␍Loading weights: 11%|█ | 81/750 [00:00<00:02, 239.92it/s]␍Loading weights: 15%|█▍ | 109/750 [00:00<00:02, 251.29it/s]␍Loading weights: 18%|█▊ | 136/750 [00:00<00:02, 255.29it/s]␍Loading weights: 22%|██▏ | 162/750 [00:00<00:02, 250.35it/s]␍Loading weights: 25%|██▌ | 191/750 [00:00<00:02, 259.13it/s]␍Loading weights: 29%|██▉ | 217/750 [00:00<00:02, 254.52it/s]␍Loading weights: 33%|███▎ | 246/750 [00:00<00:01, 263.13it/s]␍Loading weights: 36%|███▋ | 273/750 [00:01<00:01, 261.43it/s]␍Loading weights: 40%|████ | 301/750 [00:01<00:01, 266.52it/s]␍Loading weights: 44%|████▎ | 328/750 [00:01<00:01, 265.10it/s]␍Loading weights: 48%|████▊ | 357/750 [00:01<00:01, 264.28it/s]␍Loading weights: 51%|█████▏ | 385/750 [00:01<00:01, 268.44it/s]␍Loading weights: 72%|███████▏ | 539/750 [00:01<00:00, 640.17it/s]␍Loading weights: 99%|█████████▊| 739/750 [00:01<00:00, 1035.77it/s]␍Loading weights: 100%|██████████| 750/750 [00:01<00:00, 441.58it/s]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:04 [platform.py:338] Defaulting to diffusion attention backend CUDNN_ATTN (Blackwell sm_100, cuDNN 92000, head_dim 128)
(DiffusionWorker pid=2864057) INFO 09-21 08:46:04 [selector.py:93] Resolved diffusion attention backend 'CUDNN_ATTN' for role='self' via platform default
(DiffusionWorker pid=2864057) INFO 09-21 08:46:06 [weight_utils.py:58] Using model weights format ['transformer/diffusion_pytorch_model-00001-of-00002.safetensors', 'transformer/diffusion_pytorch_model-00002-of-00002.safetensors']
(DiffusionWorker pid=2864057) ␍Multi-thread loading shards: 0% Completed | 0/2 [00:00<?, ?it/s]
(DiffusionWorker pid=2864057) ␍Multi-thread loading shards: 50% Completed | 1/2 [00:00<00:00, 1.88it/s]
(DiffusionWorker pid=2864057) ␍Multi-thread loading shards: 100% Completed | 2/2 [00:01<00:00, 1.06it/s]
(DiffusionWorker pid=2864057) ␍Multi-thread loading shards: 100% Completed | 2/2 [00:01<00:00, 1.14it/s]
(DiffusionWorker pid=2864057)
(DiffusionWorker pid=2864057) INFO 09-21 08:46:08 [diffusers_loader.py:1098] Loading weights took 1.83 seconds
(DiffusionWorker pid=2864057) INFO 09-21 08:46:09 [diffusion_model_runner.py:351] Model loading took 30.2190 GiB and 7.928883 seconds
(DiffusionWorker pid=2864057) INFO 09-21 08:46:09 [diffusion_model_runner.py:357] Model runner: Model loaded successfully.
(DiffusionWorker pid=2864057) INFO 09-21 08:46:09 [diffusion_model_runner.py:243] Model runner: transformer uses CUDA graph decode instead of torch.compile.
(DiffusionWorker pid=2864057) INFO 09-21 08:46:09 [diffusion_model_runner.py:448] Model runner: Initialization complete.
(DiffusionWorker pid=2864057) INFO 09-21 08:46:09 [diffusion_worker.py:425] Worker 0: Process-scoped GPU memory after model loading: 31.01 GiB.
(DiffusionWorker pid=2864057) INFO 09-21 08:46:09 [manager.py:112] Initializing DiffusionLoRAManager: device=cuda:0, dtype=torch.bfloat16, max_cached_adapters=1, static_lora_path=None
(DiffusionWorker pid=2864057) INFO 09-21 08:46:09 [diffusion_worker.py:302] Worker 0: Initialization complete.
(DiffusionWorker pid=2864057) INFO 09-21 08:46:09 [uniproc_executor.py:74] Diffusion worker initialized in-process (uniproc executor)
(DiffusionWorker pid=2864057) INFO 09-21 08:46:09 [diffusion_engine.py:1289] dummy run to warm up the model
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 10, ((1, 32, 32),), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [inline_stage_diffusion_client.py:82] [InlineStageDiffusionClient] stage-0 [rep-0] initialized inline (max_num_seqs=2)
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [stage_runtime.py:1226] [StageRuntime] Stage 0 replica 0 initialized (diffusion)
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [orchestrator.py:420] [Orchestrator] Starting event loop
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [omni_engine_base.py:344] [OmniEngine] Orchestrator ready with 1 stages
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [omni_base.py:227] [AsyncOmni] AsyncOmniEngine initialized in 11.77 seconds
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [omni_base.py:246] [AsyncOmni] Initialized with 1 stages for model Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [api_server.py:767] Detected pure diffusion mode (single diffusion stage)
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [serving_video.py:170] Video response frame conversion pool configured: workers=8
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [speaker_cache.py:239] Speaker cache ready (max_bytes=536870912)
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [serving_speech.py:264] Speaker storage: dir=/home/mozf/.cache/vllm-omni/speakers, max_speakers=1000, restored=0
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [serving.py:91] Robot OpenPI serving disabled for model Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [api_server.py:855] Pure diffusion API server initialized for model: Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [api_server.py:367] Starting vLLM API server (pure diffusion mode) on http://127.0.0.1:44729
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:61] Available routes are:
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /openapi.json, Methods: HEAD, GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /docs, Methods: HEAD, GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /docs/oauth2-redirect, Methods: HEAD, GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /redoc, Methods: HEAD, GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /load, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /version, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /metrics, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /tokenize, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /detokenize, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /ping, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /ping, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /invocations, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/responses, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/responses/{response_id}, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/responses/{response_id}/cancel, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/completions, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/messages, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/messages/count_tokens, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /generative_scoring, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /scale_elastic_ep, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /is_scaling_elastic_ep, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/chat/completions/render, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/messages/render, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/completions/render, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/chat/completions/derender, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/completions/derender, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /inference/v1/generate, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/chat/completions, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/chat/completions/batch, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/audio/speech, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/audio/speech/batch, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/audio/generate, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/audio/voices, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/audio/voices, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/audio/voices/{name}, Methods: DELETE
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/realtime/sessions, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/realtime/sessions/{session_id}/step, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/realtime/sessions/{session_id}/reset, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/realtime/sessions/{session_id}/close, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/realtime/sessions/{session_id}/status, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /health, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/models, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/images/generations, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/images/edits, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/videos, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/videos/sync, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/videos, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/videos/{video_id}, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/videos/{video_id}, Methods: DELETE
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/videos/{video_id}/content, Methods: GET
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/omni/sleep, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:70] Route: /v1/omni/wakeup, Methods: POST
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:81] Route: /v1/audio/speech/stream, Endpoint: streaming_speech
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:81] Route: /v1/video/chat/stream, Endpoint: streaming_video_chat
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:81] Route: /v1/realtime/video, Endpoint: streaming_video_output
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:81] Route: /v1/realtime, Endpoint: realtime_websocket
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:81] Route: /v1/duplex, Endpoint: duplex_websocket
(DiffusionWorker pid=2864057) INFO 09-21 08:46:10 [launcher.py:81] Route: /v1/realtime/robot/openpi, Endpoint: realtime_robot_openpi
(DiffusionWorker pid=2864057) INFO: Started server process [2864057]
(DiffusionWorker pid=2864057) INFO: Waiting for application startup.
(DiffusionWorker pid=2864057) INFO: Application startup complete.
OmniServer stopping...
Pre-test device status:
[Device Memory Monitor] Waiting for device(s) 0, 1, 2, 3 to free memory, Condition: Memory usage ratio ≤ 5.0%
[Device Memory Status] Current usage:
Device 0: 0.7GiB/184.3GiB (0.4%)
Device 1: 0.7GiB/184.3GiB (0.4%)
Device 2: 0.7GiB/184.3GiB (0.4%)
Device 3: 0.7GiB/184.3GiB (0.4%)
[Device Memory Freed] Device(s) 0, 1, 2, 3 meet memory condition
Condition: Memory usage ratio ≤ 5.0%
Wait time: 0.0 seconds (0.0 minutes)
Post-test device status:
================================================================================
NVIDIA GPU Information (nvidia-smi)
================================================================================
Mon Sep 21 08:45:35 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.09 Driver Version: 580.126.09 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GB200 Off | 00000000:89:00.0 Off | 0 |
| N/A 27C P0 219W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GB200 Off | 00000000:8A:00.0 Off | 0 |
| N/A 26C P0 229W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA GB200 Off | 00000000:C1:00.0 Off | 0 |
| N/A 26C P0 201W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
... (showing first 20 of 35 lines)
================================================================================
Detailed GPU Processes (nvidia-smi pmon)
================================================================================
# gpu pid type sm mem enc dec jpg ofa command
# Idx # C/G % % % % % % name
0 2853490 C - - - - - - python
1 2853490 C 0 0 - - - - python
2 2853490 C 0 0 - - - - python
3 2853490 C 0 0 - - - - python
OmniServer stopped
Pre-test device status:
[Device Memory Monitor] Waiting for device(s) 0, 1, 2, 3 to free memory, Condition: Memory usage ratio ≤ 5.0%
[Device Memory Status] Current usage:
Device 0: 0.7GiB/184.3GiB (0.4%)
Device 1: 0.7GiB/184.3GiB (0.4%)
Device 2: 0.7GiB/184.3GiB (0.4%)
Device 3: 0.7GiB/184.3GiB (0.4%)
[Device Memory Freed] Device(s) 0, 1, 2, 3 meet memory condition
Condition: Memory usage ratio ≤ 5.0%
Wait time: 0.0 seconds (0.0 minutes)
Post-test device status:
================================================================================
NVIDIA GPU Information (nvidia-smi)
================================================================================
Mon Sep 21 08:45:36 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.09 Driver Version: 580.126.09 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GB200 Off | 00000000:89:00.0 Off | 0 |
| N/A 27C P0 219W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GB200 Off | 00000000:8A:00.0 Off | 0 |
| N/A 26C P0 229W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA GB200 Off | 00000000:C1:00.0 Off | 0 |
| N/A 26C P0 201W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
... (showing first 20 of 35 lines)
================================================================================
Detailed GPU Processes (nvidia-smi pmon)
================================================================================
# gpu pid type sm mem enc dec jpg ofa command
# Idx # C/G % % % % % % name
0 2853490 C - - - - - - python
1 2853490 C 0 0 - - - - python
2 2853490 C 0 0 - - - - python
3 2853490 C 0 0 - - - - python
Launching OmniServer with: /home/mozf/vllm-omni/.venv/bin/python -m vllm_omni.entrypoints.cli.main serve Qwen/Qwen-Image-2.1 --host 127.0.0.1 --port 44729 --omni --vae-use-tiling --max-num-seqs 2 --step-execution --stage-init-timeout 600 --init-timeout 900 --log-stats
Server ready on 127.0.0.1:44729 (OmniServer startup took 36.070s)
(DiffusionWorker pid=2864057) INFO 09-21 08:46:12 [serving_chat.py:3651] Diffusion chat request chatcmpl-298a1e96b0c44416: ref_images=0, params={'negative_prompt': 'blurry, low quality', 'height': 512, 'seed': 42, 'true_cfg_scale': 4.0, 'width': 512, 'num_inference_steps': 4}
(DiffusionWorker pid=2864057) INFO 09-21 08:46:12 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 18, ((1, 32, 32),), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('uncond', 1, 13, ((1, 32, 32),), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811] [Overall Summary]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811] +-----------------------------+---------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811] | Field | Value |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811] +-----------------------------+---------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811] | e2e_requests | 1 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811] | e2e_wall_time_ms | 964.303 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811] | e2e_avg_time_per_request_ms | 964.303 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811] | e2e_stage_0_wall_time_ms | 964.184 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:811] +-----------------------------+---------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:837]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:837] [RequestE2EStats [request_id=chatcmpl-298a1e96b0c44416-a113efb5]]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:837] +--------------+---------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:837] | Field | Value |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:837] +--------------+---------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:837] | e2e_total_ms | 964.184 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:837] +--------------+---------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:878] [OmniTiming] req=chatcmpl-298a1e96b0c44416-a113efb5 total=0.96s engine=0.96s stages=[0:0.96s]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] [StageRequestStats [request_id=chatcmpl-298a1e96b0c44416-a113efb5]]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] +---------------------------------+---------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | Field | 0 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] +---------------------------------+---------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | batch_id | 1 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | batch_size | 1 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | denoise_step_latency_ms | 240.872 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | diffusion_engine_exec_time_s | 0.957 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | height | 512.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | image_num | 1.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | image_pixels | 262,144 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | image_time_to_first_output_ms | 963.466 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | inter_output_latencies_ms | |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | num_inference_steps | 4.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | output_unit_count | 1 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | output_unit_type | image |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | postprocess_time_s | 0.006 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | preprocess_time_s | 0.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | resolution | 640.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | scheduler_queue_wait_s | 0.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | serving_time_to_first_output_ms | 963.828 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | stage_gen_time_ms | 963.487 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | vllm_itls_ms | |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] | width | 512.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [stats.py:920] +---------------------------------+---------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [serving_chat.py:3943] Diffusion chat completed for request chatcmpl-298a1e96b0c44416: output_type=image, image_count=1
(DiffusionWorker pid=2864057) INFO: 127.0.0.1:48938 - "POST /v1/chat/completions HTTP/1.1" 200 OK
OmniServer started successfully
--- Running test: test_text_to_image[step]
[diffusion] request#1 success in 1.110s
PASSED
tests/e2e/online_serving/test_qwen_image_21.py::test_image_edit[step] (DiffusionWorker pid=2864057) INFO 09-21 08:46:13 [serving_chat.py:3651] Diffusion chat request chatcmpl-6b7e1eb7fb774af6: ref_images=1, params={'negative_prompt': 'blurry, low quality', 'height': 512, 'seed': 42, 'true_cfg_scale': 4.0, 'width': 512, 'num_inference_steps': 4}
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 4115, ((1, 64, 64), (1, 32, 32)), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('uncond', 1, 4115, ((1, 64, 64), (1, 32, 32)), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811] [Overall Summary]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811] +-----------------------------+-----------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811] | Field | Value |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811] +-----------------------------+-----------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811] | e2e_requests | 1 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811] | e2e_wall_time_ms | 1,422.300 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811] | e2e_avg_time_per_request_ms | 1,422.300 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811] | e2e_stage_0_wall_time_ms | 1,422.209 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:811] +-----------------------------+-----------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:837]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:837] [RequestE2EStats [request_id=chatcmpl-6b7e1eb7fb774af6-9450ba42]]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:837] +--------------+-----------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:837] | Field | Value |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:837] +--------------+-----------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:837] | e2e_total_ms | 1,422.209 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:837] +--------------+-----------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:878] [OmniTiming] req=chatcmpl-6b7e1eb7fb774af6-9450ba42 total=1.42s engine=1.42s stages=[0:1.42s]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] [StageRequestStats [request_id=chatcmpl-6b7e1eb7fb774af6-9450ba42]]
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] +---------------------------------+-----------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | Field | 0 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] +---------------------------------+-----------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | batch_id | 2 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | batch_size | 1 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | denoise_step_latency_ms | 355.232 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | diffusion_engine_exec_time_s | 1.370 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | height | 512.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | image_num | 1.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | image_pixels | 262,144 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | image_time_to_first_output_ms | 1,420.895 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | inter_output_latencies_ms | |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | num_inference_steps | 4.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | output_unit_count | 1 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | output_unit_type | image |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | postprocess_time_s | 0.012 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | preprocess_time_s | 0.039 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | resolution | 640.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | scheduler_queue_wait_s | 0.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | serving_time_to_first_output_ms | 1,421.129 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | stage_gen_time_ms | 1,420.927 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | vllm_itls_ms | |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] | width | 512.000 |
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [stats.py:920] +---------------------------------+-----------+
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [serving_chat.py:3943] Diffusion chat completed for request chatcmpl-6b7e1eb7fb774af6: output_type=image, image_count=1
(DiffusionWorker pid=2864057) INFO: 127.0.0.1:48944 - "POST /v1/chat/completions HTTP/1.1" 200 OK
--- Running test: test_image_edit[step]
[diffusion] request#1 success in 1.479s
PASSED(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [launcher.py:124] [shutdown] API server: shutdown triggered
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [launcher.py:140] [shutdown] API server: stopping engine client mode=abort timeout=0.0s
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [omni_base.py:771] [AsyncOmni] Shutting down
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [omni_engine_base.py:1144] [OmniEngine] Shutting down Orchestrator
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [orchestrator.py:512] [Orchestrator] Received shutdown signal
(DiffusionWorker pid=2864057) INFO 09-21 08:46:14 [orchestrator.py:2720] [Orchestrator] Shutting down all 1 client(s)
(DiffusionWorker pid=2864057) INFO 09-21 08:46:15 [stage_pool.py:1362] [StagePool] Stage 0 replica 0 shut down
(DiffusionWorker pid=2864057) INFO 09-21 08:46:15 [launcher.py:149] [shutdown] API server: engine client stopped
(DiffusionWorker pid=2864057) INFO 09-21 08:46:15 [launcher.py:152] [shutdown] API server: signalling HTTP server shutdown
(DiffusionWorker pid=2864057) INFO 09-21 08:46:15 [launcher.py:173] [shutdown] API server: shutting down FastAPI HTTP server
(DiffusionWorker pid=2864057) INFO: Shutting down
(DiffusionWorker pid=2864057) INFO: Waiting for application shutdown.
(DiffusionWorker pid=2864057) INFO: Application shutdown complete.
(DiffusionWorker pid=2864057) INFO 09-21 08:46:15 [omni_base.py:771] [AsyncOmni] Shutting down
OmniServer stopping...
Pre-test device status:
[Device Memory Monitor] Waiting for device(s) 0, 1, 2, 3 to free memory, Condition: Memory usage ratio ≤ 5.0%
[Device Memory Status] Current usage:
Device 0: 0.7GiB/184.3GiB (0.4%)
Device 1: 0.7GiB/184.3GiB (0.4%)
Device 2: 0.7GiB/184.3GiB (0.4%)
Device 3: 0.7GiB/184.3GiB (0.4%)
[Device Memory Freed] Device(s) 0, 1, 2, 3 meet memory condition
Condition: Memory usage ratio ≤ 5.0%
Wait time: 0.0 seconds (0.0 minutes)
Post-test device status:
================================================================================
NVIDIA GPU Information (nvidia-smi)
================================================================================
Mon Sep 21 08:46:17 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 580.126.09 Driver Version: 580.126.09 CUDA Version: 13.0 |
+-----------------------------------------+------------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GB200 Off | 00000000:89:00.0 Off | 0 |
| N/A 27C P0 219W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 1 NVIDIA GB200 Off | 00000000:8A:00.0 Off | 0 |
| N/A 26C P0 229W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
| 2 NVIDIA GB200 Off | 00000000:C1:00.0 Off | 0 |
| N/A 26C P0 201W / 1200W | 695MiB / 189471MiB | 0% Default |
| | | Disabled |
+-----------------------------------------+------------------------+----------------------+
... (showing first 20 of 35 lines)
================================================================================
Detailed GPU Processes (nvidia-smi pmon)
================================================================================
# gpu pid type sm mem enc dec jpg ofa command
# Idx # C/G % % % % % % name
0 2853490 C - - - - - - python
1 2853490 C 0 0 - - - - python
2 2853490 C 0 0 - - - - python
3 2853490 C 0 0 - - - - python
OmniServer stopped
=============================== warnings summary ===============================
.venv/lib/python3.12/site-packages/torch/jit/_script.py:365: 14 warnings
/home/mozf/vllm-omni/.venv/lib/python3.12/site-packages/torch/jit/_script.py:365: DeprecationWarning: `torch.jit.script_method` is deprecated. Please switch to `torch.compile` or `torch.export`.
warnings.warn(
vllm_omni/version.py:55
/home/mozf/vllm-omni/vllm_omni/version.py:55: RuntimeWarning: vLLM and vLLM-Omni appear to have mismatched major/minor versions:
--> vLLM-Omni version 0.26.1.dev1+g76de642c5
--> vLLM version 0.29.0
This will likely cause compatibility issues.
warn_if_misaligned_vllm_version()
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
--- Running Summary
================== 4 passed, 15 warnings in 173.25s (0:02:53) ==================
artifacts/pr7759-review/gen_images_30094.out
-204
View File
@@ -1,204 +0,0 @@
node: gb200-rack1-01
=== T2I ===
/home/mozf/vllm-omni/vllm_omni/version.py:55: RuntimeWarning: vLLM and vLLM-Omni appear to have mismatched major/minor versions:
--> vLLM-Omni version 0.26.1.dev1+g76de642c5
--> vLLM version 0.29.0
This will likely cause compatibility issues.
warn_if_misaligned_vllm_version()
INFO 09-21 08:49:45 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed.
INFO 09-21 08:49:46 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL).
INFO 09-21 08:49:46 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM.
INFO 09-21 08:49:46 [patch.py:503] inductor factorable-divisibility patch: installed.
INFO 09-21 08:49:46 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms.
INFO 09-21 08:49:47 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell
INFO 09-21 08:49:49 [weight_utils.py:58] Using model weights format ['*']
INFO 09-21 08:49:49 [omni_base.py:208] [Omni] Initializing with model Qwen/Qwen-Image-2.1
INFO 09-21 08:49:49 [omni_engine_base.py:163] [OmniEngine] Initializing with model Qwen/Qwen-Image-2.1
INFO 09-21 08:49:51 [config_factory.py:697] Resolved generic diffusion final_output_type='image' for model_class_name='QwenImage21Pipeline'.
INFO 09-21 08:49:51 [omni_engine_base.py:312] [OmniEngine] Launching Orchestrator thread with 1 stages
INFO 09-21 08:49:51 [stage_init_utils.py:775] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
INFO 09-21 08:49:51 [weight_utils.py:58] Using model weights format ['*']
INFO 09-21 08:49:51 [weight_utils.py:58] Using model weights format ['*']
INFO 09-21 08:49:51 [scheduler.py:277] Chunked prefill is enabled with max_num_batched_tokens=2048.
INFO 09-21 08:49:51 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
INFO 09-21 08:49:51 [diffusion_worker.py:332] Final IR op priority after setting vLLM-Omni overrides: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'])
INFO 09-21 08:49:51 [diffusion_worker.py:344] Worker 0: Initialized device and distributed environment.
INFO 09-21 08:49:51 [parallel_state.py:610] Building SP subgroups from explicit sp_group_ranks (sp_size=1, ulysses=1, ring=1, use_ulysses_low=True).
INFO 09-21 08:49:51 [parallel_state.py:652] SP group details for rank 0: sp_group=[0], ulysses_group=[0], ring_group=[0]
(DiffusionWorker pid=2877460) INFO 09-21 08:49:52 [hub_prefetch.py:299] Prefetching Qwen/Qwen-Image-2.1 subfolders: ['scheduler', 'text_encoder', 'vae', 'processor']
(DiffusionWorker pid=2877460) INFO 09-21 08:49:52 [hub_prefetch.py:224] Acquired flock prefetch lock for Qwen/Qwen-Image-2.1 at /mnt/lustre/hf-models/locks/vllm-omni-prefetch/Qwen__Qwen-Image-2.1.lock
(DiffusionWorker pid=2877460) ␍Fetching 22 files: 0%| | 0/22 [00:00<?, ?it/s]␍Fetching 22 files: 100%|██████████| 22/22 [00:00<00:00, 1979.08it/s]
(DiffusionWorker pid=2877460) INFO 09-21 08:49:52 [hub_prefetch.py:344] Prefetch complete for Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2877460) /home/mozf/vllm-omni/.venv/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py:205: UserWarning: The `local_dir_use_symlinks` argument is deprecated and ignored in `hf_hub_download`. Downloading to a local directory does not use symlinks anymore.
(DiffusionWorker pid=2877460) warnings.warn(
(DiffusionWorker pid=2877460) ␍Fetching 4 files: 0%| | 0/4 [00:00<?, ?it/s]␍Fetching 4 files: 100%|██████████| 4/4 [00:00<00:00, 1477.00it/s]
(DiffusionWorker pid=2877460) ␍Loading weights: 0%| | 0/750 [00:00<?, ?it/s]␍Loading weights: 3%|▎ | 23/750 [00:00<00:03, 225.47it/s]␍Loading weights: 6%|▋ | 48/750 [00:00<00:02, 237.05it/s]␍Loading weights: 10%|▉ | 72/750 [00:00<00:03, 225.49it/s]␍Loading weights: 13%|█▎ | 101/750 [00:00<00:02, 249.63it/s]␍Loading weights: 17%|█▋ | 127/750 [00:00<00:02, 232.96it/s]␍Loading weights: 21%|██ | 158/750 [00:00<00:02, 248.63it/s]␍Loading weights: 25%|██▍ | 184/750 [00:00<00:02, 243.05it/s]␍Loading weights: 28%|██▊ | 213/750 [00:00<00:02, 250.02it/s]␍Loading weights: 32%|███▏ | 239/750 [00:00<00:02, 245.90it/s]␍Loading weights: 36%|███▌ | 268/750 [00:01<00:01, 255.40it/s]␍Loading weights: 39%|███▉ | 294/750 [00:01<00:01, 252.61it/s]␍Loading weights: 43%|████▎ | 323/750 [00:01<00:01, 261.60it/s]␍Loading weights: 47%|████▋ | 350/750 [00:01<00:01, 260.07it/s]␍Loading weights: 50%|█████ | 378/750 [00:01<00:01, 261.11it/s]␍Loading weights: 62%|██████▏ | 467/750 [00:01<00:00, 442.11it/s]␍Loading weights: 95%|█████████▌| 716/750 [00:01<00:00, 1039.01it/s]␍Loading weights: 100%|██████████| 750/750 [00:01<00:00, 428.30it/s]
(DiffusionWorker pid=2877460) INFO 09-21 08:49:55 [platform.py:338] Defaulting to diffusion attention backend CUDNN_ATTN (Blackwell sm_100, cuDNN 92000, head_dim 128)
(DiffusionWorker pid=2877460) INFO 09-21 08:49:55 [selector.py:93] Resolved diffusion attention backend 'CUDNN_ATTN' for role='self' via platform default
(DiffusionWorker pid=2877460) INFO 09-21 08:49:57 [weight_utils.py:58] Using model weights format ['transformer/diffusion_pytorch_model-00001-of-00002.safetensors', 'transformer/diffusion_pytorch_model-00002-of-00002.safetensors']
(DiffusionWorker pid=2877460) ␍Multi-thread loading shards: 0% Completed | 0/2 [00:00<?, ?it/s]
(DiffusionWorker pid=2877460) ␍Multi-thread loading shards: 50% Completed | 1/2 [00:00<00:00, 1.88it/s]
(DiffusionWorker pid=2877460) ␍Multi-thread loading shards: 100% Completed | 2/2 [00:01<00:00, 1.05it/s]
(DiffusionWorker pid=2877460) ␍Multi-thread loading shards: 100% Completed | 2/2 [00:01<00:00, 1.12it/s]
(DiffusionWorker pid=2877460)
(DiffusionWorker pid=2877460) INFO 09-21 08:49:59 [diffusers_loader.py:1098] Loading weights took 1.86 seconds
(DiffusionWorker pid=2877460) INFO 09-21 08:49:59 [diffusion_model_runner.py:351] Model loading took 30.2190 GiB and 8.080112 seconds
(DiffusionWorker pid=2877460) INFO 09-21 08:49:59 [diffusion_model_runner.py:357] Model runner: Model loaded successfully.
(DiffusionWorker pid=2877460) INFO 09-21 08:49:59 [diffusion_model_runner.py:243] Model runner: transformer uses CUDA graph decode instead of torch.compile.
(DiffusionWorker pid=2877460) INFO 09-21 08:49:59 [diffusion_model_runner.py:448] Model runner: Initialization complete.
(DiffusionWorker pid=2877460) INFO 09-21 08:50:00 [diffusion_worker.py:425] Worker 0: Process-scoped GPU memory after model loading: 31.01 GiB.
(DiffusionWorker pid=2877460) INFO 09-21 08:50:00 [manager.py:112] Initializing DiffusionLoRAManager: device=cuda:0, dtype=torch.bfloat16, max_cached_adapters=1, static_lora_path=None
(DiffusionWorker pid=2877460) INFO 09-21 08:50:00 [diffusion_worker.py:302] Worker 0: Initialization complete.
(DiffusionWorker pid=2877460) INFO 09-21 08:50:00 [uniproc_executor.py:74] Diffusion worker initialized in-process (uniproc executor)
(DiffusionWorker pid=2877460) INFO 09-21 08:50:00 [diffusion_engine.py:402] [RequestBatch] engine init max_num_seqs=1 max_wait_ms=0.0
(DiffusionWorker pid=2877460) INFO 09-21 08:50:00 [diffusion_engine.py:1289] dummy run to warm up the model
(DiffusionWorker pid=2877460) ␍ 0%| | 0/2 [00:00<?, ?it/s]␍ 50%|█████ | 1/2 [00:00<00:00, 4.72it/s](DiffusionWorker pid=2877460) INFO 09-21 08:50:01 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 10, ((1, 32, 32),), torch.bfloat16, 0, 'CUDNN_ATTN')
␍100%|██████████| 2/2 [00:00<00:00, 4.59it/s]␍100%|██████████| 2/2 [00:00<00:00, 4.60it/s]
(DiffusionWorker pid=2877460) INFO 09-21 08:50:01 [inline_stage_diffusion_client.py:82] [InlineStageDiffusionClient] stage-0 [rep-0] initialized inline (max_num_seqs=1)
(DiffusionWorker pid=2877460) INFO 09-21 08:50:01 [stage_runtime.py:1226] [StageRuntime] Stage 0 replica 0 initialized (diffusion)
(DiffusionWorker pid=2877460) INFO 09-21 08:50:01 [orchestrator.py:420] [Orchestrator] Starting event loop
(DiffusionWorker pid=2877460) INFO 09-21 08:50:01 [omni_engine_base.py:344] [OmniEngine] Orchestrator ready with 1 stages
(DiffusionWorker pid=2877460) INFO 09-21 08:50:01 [omni_base.py:227] [Omni] AsyncOmniEngine initialized in 11.73 seconds
(DiffusionWorker pid=2877460) INFO 09-21 08:50:01 [omni_base.py:246] [Omni] Initialized with 1 stages for model Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2877460)
(DiffusionWorker pid=2877460) ============================================================
(DiffusionWorker pid=2877460) Generation Configuration:
(DiffusionWorker pid=2877460) Model: Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2877460) Inference steps: 20
(DiffusionWorker pid=2877460) Cache backend: None (no acceleration)
(DiffusionWorker pid=2877460) Quantization: None (BF16)
(DiffusionWorker pid=2877460) Parallel configuration: tensor_parallel_size=deploy/default, ulysses_degree=1, ulysses_mode=strict, ring_degree=1, cfg_parallel_size=1, vae_patch_parallel_size=1, enable_expert_parallel=False.
(DiffusionWorker pid=2877460) CPU offload: False; CPU Layerwise Offload: False
(DiffusionWorker pid=2877460) Image size: 1024x1024
(DiffusionWorker pid=2877460) ============================================================
(DiffusionWorker pid=2877460)
(DiffusionWorker pid=2877460) ␍Processed prompts: 0%| | 0/1 [00:00<?, ?it/s]
(DiffusionWorker pid=2877460) ␍ 0%| | 0/20 [00:00<?, ?it/s]
(DiffusionWorker pid=2877460) ␍ 5%|▌ | 1/20 [00:00<00:05, 3.22it/s](DiffusionWorker pid=2877460) INFO 09-21 08:50:01 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 18, ((1, 64, 64),), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2877460) INFO 09-21 08:50:02 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('uncond', 1, 13, ((1, 64, 64),), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2877460) ␍ 10%|█ | 2/20 [00:00<00:08, 2.03it/s]
(DiffusionWorker pid=2877460) ␍ 15%|█▌ | 3/20 [00:01<00:05, 2.96it/s]
(DiffusionWorker pid=2877460) ␍ 20%|██ | 4/20 [00:01<00:04, 3.77it/s]
(DiffusionWorker pid=2877460) ␍ 25%|██▌ | 5/20 [00:01<00:03, 4.44it/s]
(DiffusionWorker pid=2877460) ␍ 30%|███ | 6/20 [00:01<00:02, 5.00it/s]
(DiffusionWorker pid=2877460) ␍ 35%|███▌ | 7/20 [00:01<00:02, 5.43it/s]
(DiffusionWorker pid=2877460) ␍ 40%|████ | 8/20 [00:01<00:02, 5.77it/s]
(DiffusionWorker pid=2877460) ␍ 45%|████▌ | 9/20 [00:01<00:01, 6.01it/s]
(DiffusionWorker pid=2877460) ␍ 50%|█████ | 10/20 [00:02<00:01, 6.21it/s]
(DiffusionWorker pid=2877460) ␍ 55%|█████▌ | 11/20 [00:02<00:01, 6.35it/s]
(DiffusionWorker pid=2877460) ␍ 60%|██████ | 12/20 [00:02<00:01, 6.40it/s]
(DiffusionWorker pid=2877460) ␍ 65%|██████▌ | 13/20 [00:02<00:01, 6.46it/s]
(DiffusionWorker pid=2877460) ␍ 70%|███████ | 14/20 [00:02<00:00, 6.53it/s]
(DiffusionWorker pid=2877460) ␍ 75%|███████▌ | 15/20 [00:02<00:00, 6.57it/s]
(DiffusionWorker pid=2877460) ␍ 80%|████████ | 16/20 [00:03<00:00, 6.57it/s]
(DiffusionWorker pid=2877460) ␍ 85%|████████▌ | 17/20 [00:03<00:00, 6.57it/s]
(DiffusionWorker pid=2877460) ␍ 90%|█████████ | 18/20 [00:03<00:00, 6.60it/s]
(DiffusionWorker pid=2877460) ␍ 95%|█████████▌| 19/20 [00:03<00:00, 6.61it/s]
(DiffusionWorker pid=2877460) ␍100%|██████████| 20/20 [00:03<00:00, 6.63it/s]␍100%|██████████| 20/20 [00:03<00:00, 5.47it/s]
(DiffusionWorker pid=2877460) ␍Processed prompts: 100%|██████████| 1/1 [00:03<00:00, 3.94s/it]␍Processed prompts: 100%|██████████| 1/1 [00:03<00:00, 3.94s/it]
(DiffusionWorker pid=2877460) Total generation time: 3.9453 seconds (3945.34 ms)
(DiffusionWorker pid=2877460) INFO 09-21 08:50:05 [text_to_image.py:794] Outputs: [OmniRequestOutput(request_id='0_770e0883-477e-4c55-8e47-3328e985b949', finished=True, stage_id=0, final_output_type='image', prompt_token_ids=None, outputs=[], images=[1 PIL Images], prompt={'prompt': 'A red ceramic teapot on a wooden table.', 'modalities': ['image'], 'negative_prompt': 'blurry, low quality', 'additional_information': {}}, latents=None, metrics={'stage_metrics': {'0': {'stage_id': 0, 'final_output_type': 'image', 'finish_reason': None, 'num_tokens_in': 0, 'num_tokens_out': 0, 'stage_gen_time_ms': 3943.441867828369, 'postprocess_time_ms': 0.0, 'audio_frames': 0, 'audio_sample_rate': 0, 'audio_duration_s': 0.0, 'image_pixels': 1048576, 'denoise_step_latency_ms': 197.17209339141846, 'output_unit_type': 'image', 'output_unit_count': 1, 'serving_time_to_first_output_ms': 3943.847417831421, 'image_time_to_first_output_ms': 3943.420171737671, 'time_per_output_unit_ms': 0.0, 'inter_output_latency_ms': 0.0, 'inter_output_latencies_ms': [], 'vllm_ttft_ms': 0.0, 'vllm_tpot_ms': 0.0, 'vllm_itl_ms': 0.0, 'vllm_itls_ms': [], 'stage_name': 'diffusion'}}, 'stage_id': 0, 'final_output_type': 'image', 'num_tokens_in': 0, 'num_tokens_out': 0}, multimodal_output={}, custom_output={}, stage_durations={'queue_wait_ms': 0.4212159983580932, 'stage_0_gen_ms': 3943.441867828369}, peak_memory_mb=40320.0)]
(DiffusionWorker pid=2877460) Saved generated image to /home/mozf/vllm-omni/artifacts/pr7759-review/e2e_images/t2i_teapot.png
(DiffusionWorker pid=2877460) INFO 09-21 08:50:05 [omni_engine_base.py:1144] [OmniEngine] Shutting down Orchestrator
(DiffusionWorker pid=2877460) INFO 09-21 08:50:05 [orchestrator.py:512] [Orchestrator] Received shutdown signal
(DiffusionWorker pid=2877460) INFO 09-21 08:50:05 [orchestrator.py:2720] [Orchestrator] Shutting down all 1 client(s)
(DiffusionWorker pid=2877460) INFO 09-21 08:50:06 [stage_pool.py:1362] [StagePool] Stage 0 replica 0 shut down
=== I2I (edit the T2I output) ===
/home/mozf/vllm-omni/vllm_omni/version.py:55: RuntimeWarning: vLLM and vLLM-Omni appear to have mismatched major/minor versions:
--> vLLM-Omni version 0.26.1.dev1+g76de642c5
--> vLLM version 0.29.0
This will likely cause compatibility issues.
warn_if_misaligned_vllm_version()
INFO 09-21 08:50:24 [patch.py:252] NVFP4 W4A4 weight_scale NaN-clamp: installed.
INFO 09-21 08:50:26 [quack_fp8.py:76] Quack FP8 fused-bias GEMM enabled (CuteDSL).
INFO 09-21 08:50:26 [quack_fp8.py:155] Patched FlashInfer FP8 ScaledMM to use quack fused-bias GEMM.
INFO 09-21 08:50:26 [patch.py:503] inductor factorable-divisibility patch: installed.
INFO 09-21 08:50:26 [patch.py:568] [cumem-cuda] CuMemAllocator._python_free_callback patched: asleep guard extended to all platforms.
INFO 09-21 08:50:29 [fa.py:105] Using CuTe FlashAttention-4 on Blackwell
INFO 09-21 08:50:31 [weight_utils.py:58] Using model weights format ['*']
INFO 09-21 08:50:31 [omni_base.py:208] [Omni] Initializing with model Qwen/Qwen-Image-2.1
INFO 09-21 08:50:31 [omni_engine_base.py:163] [OmniEngine] Initializing with model Qwen/Qwen-Image-2.1
INFO 09-21 08:50:33 [config_factory.py:697] Resolved generic diffusion final_output_type='image' for model_class_name='QwenImage21Pipeline'.
INFO 09-21 08:50:33 [omni_engine_base.py:312] [OmniEngine] Launching Orchestrator thread with 1 stages
INFO 09-21 08:50:33 [stage_init_utils.py:775] [stage_init] Set VLLM_WORKER_MULTIPROC_METHOD=spawn
INFO 09-21 08:50:33 [weight_utils.py:58] Using model weights format ['*']
INFO 09-21 08:50:33 [weight_utils.py:58] Using model weights format ['*']
INFO 09-21 08:50:33 [scheduler.py:277] Chunked prefill is enabled with max_num_batched_tokens=2048.
INFO 09-21 08:50:33 [kernel.py:369] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['native'], fused_add_rms_norm=['native'])
INFO 09-21 08:50:33 [diffusion_worker.py:332] Final IR op priority after setting vLLM-Omni overrides: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'])
INFO 09-21 08:50:33 [diffusion_worker.py:344] Worker 0: Initialized device and distributed environment.
INFO 09-21 08:50:33 [parallel_state.py:610] Building SP subgroups from explicit sp_group_ranks (sp_size=1, ulysses=1, ring=1, use_ulysses_low=True).
INFO 09-21 08:50:33 [parallel_state.py:652] SP group details for rank 0: sp_group=[0], ulysses_group=[0], ring_group=[0]
(DiffusionWorker pid=2879936) INFO 09-21 08:50:34 [hub_prefetch.py:299] Prefetching Qwen/Qwen-Image-2.1 subfolders: ['scheduler', 'text_encoder', 'vae', 'processor']
(DiffusionWorker pid=2879936) INFO 09-21 08:50:34 [hub_prefetch.py:224] Acquired flock prefetch lock for Qwen/Qwen-Image-2.1 at /mnt/lustre/hf-models/locks/vllm-omni-prefetch/Qwen__Qwen-Image-2.1.lock
(DiffusionWorker pid=2879936) ␍Fetching 22 files: 0%| | 0/22 [00:00<?, ?it/s]␍Fetching 22 files: 100%|██████████| 22/22 [00:00<00:00, 1980.48it/s]
(DiffusionWorker pid=2879936) INFO 09-21 08:50:34 [hub_prefetch.py:344] Prefetch complete for Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2879936) /home/mozf/vllm-omni/.venv/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py:205: UserWarning: The `local_dir_use_symlinks` argument is deprecated and ignored in `hf_hub_download`. Downloading to a local directory does not use symlinks anymore.
(DiffusionWorker pid=2879936) warnings.warn(
(DiffusionWorker pid=2879936) ␍Fetching 4 files: 0%| | 0/4 [00:00<?, ?it/s]␍Fetching 4 files: 100%|██████████| 4/4 [00:00<00:00, 1553.88it/s]
(DiffusionWorker pid=2879936) ␍Loading weights: 0%| | 0/750 [00:00<?, ?it/s]␍Loading weights: 3%|▎ | 22/750 [00:00<00:03, 218.48it/s]␍Loading weights: 6%|▋ | 48/750 [00:00<00:03, 223.49it/s]␍Loading weights: 10%|▉ | 72/750 [00:00<00:02, 229.47it/s]␍Loading weights: 13%|█▎ | 99/750 [00:00<00:02, 244.50it/s]␍Loading weights: 17%|█▋ | 124/750 [00:00<00:02, 243.42it/s]␍Loading weights: 20%|█▉ | 149/750 [00:00<00:02, 234.90it/s]␍Loading weights: 24%|██▍ | 180/750 [00:00<00:02, 249.70it/s]␍Loading weights: 27%|██▋ | 205/750 [00:00<00:02, 240.97it/s]␍Loading weights: 31%|███▏ | 235/750 [00:00<00:02, 252.01it/s]␍Loading weights: 35%|███▍ | 261/750 [00:01<00:01, 247.02it/s]␍Loading weights: 39%|███▊ | 290/750 [00:01<00:01, 253.05it/s]␍Loading weights: 42%|████▏ | 316/750 [00:01<00:01, 249.97it/s]␍Loading weights: 46%|████▌ | 345/750 [00:01<00:01, 250.71it/s]␍Loading weights: 50%|████▉ | 373/750 [00:01<00:01, 257.52it/s]␍Loading weights: 57%|█████▋ | 424/750 [00:01<00:00, 328.28it/s]␍Loading weights: 85%|████████▍ | 635/750 [00:01<00:00, 842.34it/s]␍Loading weights: 100%|██████████| 750/750 [00:01<00:00, 417.44it/s]
(DiffusionWorker pid=2879936) INFO 09-21 08:50:37 [platform.py:338] Defaulting to diffusion attention backend CUDNN_ATTN (Blackwell sm_100, cuDNN 92000, head_dim 128)
(DiffusionWorker pid=2879936) INFO 09-21 08:50:37 [selector.py:93] Resolved diffusion attention backend 'CUDNN_ATTN' for role='self' via platform default
(DiffusionWorker pid=2879936) INFO 09-21 08:50:39 [weight_utils.py:58] Using model weights format ['transformer/diffusion_pytorch_model-00001-of-00002.safetensors', 'transformer/diffusion_pytorch_model-00002-of-00002.safetensors']
(DiffusionWorker pid=2879936) ␍Multi-thread loading shards: 0% Completed | 0/2 [00:00<?, ?it/s]
(DiffusionWorker pid=2879936) ␍Multi-thread loading shards: 50% Completed | 1/2 [00:00<00:00, 1.87it/s]
(DiffusionWorker pid=2879936) ␍Multi-thread loading shards: 100% Completed | 2/2 [00:01<00:00, 1.05it/s]
(DiffusionWorker pid=2879936) ␍Multi-thread loading shards: 100% Completed | 2/2 [00:01<00:00, 1.12it/s]
(DiffusionWorker pid=2879936)
(DiffusionWorker pid=2879936) INFO 09-21 08:50:41 [diffusers_loader.py:1098] Loading weights took 1.86 seconds
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [diffusion_model_runner.py:351] Model loading took 30.2190 GiB and 8.371849 seconds
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [diffusion_model_runner.py:357] Model runner: Model loaded successfully.
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [diffusion_model_runner.py:243] Model runner: transformer uses CUDA graph decode instead of torch.compile.
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [diffusion_model_runner.py:448] Model runner: Initialization complete.
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [diffusion_worker.py:425] Worker 0: Process-scoped GPU memory after model loading: 31.01 GiB.
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [manager.py:112] Initializing DiffusionLoRAManager: device=cuda:0, dtype=torch.bfloat16, max_cached_adapters=1, static_lora_path=None
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [diffusion_worker.py:302] Worker 0: Initialization complete.
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [uniproc_executor.py:74] Diffusion worker initialized in-process (uniproc executor)
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [diffusion_engine.py:402] [RequestBatch] engine init max_num_seqs=1 max_wait_ms=0.0
(DiffusionWorker pid=2879936) INFO 09-21 08:50:42 [diffusion_engine.py:1289] dummy run to warm up the model
(DiffusionWorker pid=2879936) ␍ 0%| | 0/2 [00:00<?, ?it/s]␍ 50%|█████ | 1/2 [00:00<00:00, 4.82it/s](DiffusionWorker pid=2879936) INFO 09-21 08:50:43 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 10, ((1, 32, 32),), torch.bfloat16, 0, 'CUDNN_ATTN')
␍100%|██████████| 2/2 [00:00<00:00, 4.93it/s]␍100%|██████████| 2/2 [00:00<00:00, 4.91it/s]
(DiffusionWorker pid=2879936) INFO 09-21 08:50:43 [inline_stage_diffusion_client.py:82] [InlineStageDiffusionClient] stage-0 [rep-0] initialized inline (max_num_seqs=1)
(DiffusionWorker pid=2879936) INFO 09-21 08:50:43 [stage_runtime.py:1226] [StageRuntime] Stage 0 replica 0 initialized (diffusion)
(DiffusionWorker pid=2879936) INFO 09-21 08:50:43 [orchestrator.py:420] [Orchestrator] Starting event loop
(DiffusionWorker pid=2879936) INFO 09-21 08:50:43 [omni_engine_base.py:344] [OmniEngine] Orchestrator ready with 1 stages
(DiffusionWorker pid=2879936) INFO 09-21 08:50:43 [omni_base.py:227] [Omni] AsyncOmniEngine initialized in 12.25 seconds
(DiffusionWorker pid=2879936) INFO 09-21 08:50:43 [omni_base.py:246] [Omni] Initialized with 1 stages for model Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2879936) Pipeline loaded
(DiffusionWorker pid=2879936)
(DiffusionWorker pid=2879936) ============================================================
(DiffusionWorker pid=2879936) Generation Configuration:
(DiffusionWorker pid=2879936) Model: Qwen/Qwen-Image-2.1
(DiffusionWorker pid=2879936) Inference steps: 20
(DiffusionWorker pid=2879936) Cache backend: None (no acceleration)
(DiffusionWorker pid=2879936) Input image size: (1024, 1024)
(DiffusionWorker pid=2879936) Parallel configuration: ulysses_degree=1, ring_degree=1, cfg_parallel_size=1, tensor_parallel_size=1, enable_expert_parallel: False
(DiffusionWorker pid=2879936) ============================================================
(DiffusionWorker pid=2879936)
(DiffusionWorker pid=2879936) ␍Processed prompts: 0%| | 0/1 [00:00<?, ?it/s]
(DiffusionWorker pid=2879936) ␍ 0%| | 0/20 [00:00<?, ?it/s]
(DiffusionWorker pid=2879936) ␍ 5%|▌ | 1/20 [00:00<00:12, 1.52it/s](DiffusionWorker pid=2879936) INFO 09-21 08:50:44 [decode_graph.py:446] Captured Qwen-Image-2.1 decode graph for key=('cond', 1, 4116, ((1, 64, 64), (1, 64, 64)), torch.bfloat16, 0, 'CUDNN_ATTN')
(DiffusionWorker pid=2879936) ␍ 10%|█ | 2/20 [00:00<00:08, 2.24it/s]
(DiffusionWorker pid=2879936) ␍ 20%|██ | 4/20 [00:01<00:03, 4.57it/s]
(DiffusionWorker pid=2879936) ␍ 30%|███ | 6/20 [00:01<00:02, 6.40it/s]
(DiffusionWorker pid=2879936) ␍ 40%|████ | 8/20 [00:01<00:01, 7.85it/s]
(DiffusionWorker pid=2879936) ␍ 50%|█████ | 10/20 [00:01<00:01, 8.95it/s]
(DiffusionWorker pid=2879936) ␍ 60%|██████ | 12/20 [00:01<00:00, 9.76it/s]
(DiffusionWorker pid=2879936) ␍ 70%|███████ | 14/20 [00:01<00:00, 10.40it/s]
(DiffusionWorker pid=2879936) ␍ 80%|████████ | 16/20 [00:02<00:00, 10.87it/s]
(DiffusionWorker pid=2879936) ␍ 90%|█████████ | 18/20 [00:02<00:00, 11.19it/s]
(DiffusionWorker pid=2879936) ␍100%|██████████| 20/20 [00:02<00:00, 11.42it/s]␍100%|██████████| 20/20 [00:02<00:00, 8.09it/s]
(DiffusionWorker pid=2879936) ␍Processed prompts: 100%|██████████| 1/1 [00:02<00:00, 3.00s/it]␍Processed prompts: 100%|██████████| 1/1 [00:02<00:00, 3.00s/it]
(DiffusionWorker pid=2879936) Total generation time: 2.9985 seconds (2998.46 ms)
(DiffusionWorker pid=2879936) Saved edited image to /home/mozf/vllm-omni/artifacts/pr7759-review/e2e_images/i2i_teapot_blue.png
(DiffusionWorker pid=2879936) INFO 09-21 08:50:46 [omni_engine_base.py:1144] [OmniEngine] Shutting down Orchestrator
(DiffusionWorker pid=2879936) INFO 09-21 08:50:46 [orchestrator.py:512] [Orchestrator] Received shutdown signal
(DiffusionWorker pid=2879936) INFO 09-21 08:50:46 [orchestrator.py:2720] [Orchestrator] Shutting down all 1 client(s)
(DiffusionWorker pid=2879936) INFO 09-21 08:50:47 [stage_pool.py:1362] [StagePool] Stage 0 replica 0 shut down
total 1589
drwxrwxr-x 2 mozf mozf 4096 Sep 21 08:50 .
drwxrwxr-x 3 mozf mozf 4096 Sep 21 08:49 ..
-rw-rw-r-- 1 mozf mozf 2279235 Sep 21 08:50 i2i_teapot_blue.png
-rw-rw-r-- 1 mozf mozf 1615157 Sep 21 08:50 t2i_teapot.png
artifacts/pr7759-review/gen_images_slurm.sh
-40
View File
@@ -1,40 +0,0 @@
#!/usr/bin/env bash
#SBATCH --partition=batch
#SBATCH --nodes=1
#SBATCH --time=60
#SBATCH --job-name=qi21-imgout
#SBATCH --output=/home/mozf/vllm-omni/artifacts/pr7759-review/gen_images_%j.out
set -euo pipefail
cd /home/mozf/vllm-omni
export HF_HOME=/mnt/lustre/hf-models
export VLLM_LOGGING_LEVEL=INFO
OUT=/home/mozf/vllm-omni/artifacts/pr7759-review/e2e_images
mkdir -p "$OUT"
echo "node: $(hostname)"
PY=.venv/bin/python
echo "=== T2I ==="
$PY examples/offline_inference/text_to_image/text_to_image.py \
--model Qwen/Qwen-Image-2.1 \
--prompt "A red ceramic teapot on a wooden table." \
--negative-prompt "blurry, low quality" \
--seed 42 \
--height 1024 --width 1024 \
--num-inference-steps 20 \
--output "$OUT/t2i_teapot.png"
echo "=== I2I (edit the T2I output) ==="
$PY examples/offline_inference/image_to_image/image_edit.py \
--model Qwen/Qwen-Image-2.1 \
--image "$OUT/t2i_teapot.png" \
--prompt "Make the teapot blue." \
--seed 42 \
--num-inference-steps 20 \
--cfg-scale 1.0 \
--output "$OUT/i2i_teapot_blue.png"
ls -la "$OUT"
artifacts/pr7759-review/issue_comments.txt
-54
View File
@@ -1,54 +0,0 @@
=== chatgpt-codex-connector ===
<!-- codex-pull-request-review-summary -->
## Codex Review Summary
This comment shows the latest Codex review activity on this pull request.
| Review | Status | Commit | Review trigger |
| --- | --- | --- | --- |
| 📝 **Code Review** | ✅ **Completed** <relative-time datetime="2026-09-18T02:12:54.670114Z">2026-09-18T02:12:54.670114Z</relative-time> | `3affc77` | PR opened |
<details> <summary>ℹ️ About Codex in GitHub</summary>
<br/>
[Your team has set up Codex to review pull requests in this repo](https://chatgpt.com/codex/cloud/settings/general). Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review" or "@codex security review".
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.
</details>
=== vllm-omni-review-bot ===
This PR appears to belong to: **docs/design/module/diffusion/index.md**, **docs/design/module/diffusion/parallelism.md**.
Module owners: @princepride @xuechendi @Bounty-hunter
Routing: @princepride via module of the changed files, module named in the PR description; @xuechendi via module of the changed files, module named in the PR description; @Bounty-hunter via module named in the PR description, CODEOWNERS
@Isotr0py, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.
Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.
<!-- omni-reviewbot:assignment:v1 repo=vllm-project/vllm-omni pr=7759 -->
=== vllm-omni-review-bot ===
## Omni ReviewBot triage note
Resolved as of `a26013145cfa`: the high-priority or low-quality signal noted on an earlier commit no longer applies.
<!-- omni-reviewbot:quality-note:v1 repo=vllm-project/vllm-omni pr=7759 -->
<!-- omni-reviewbot:quality-note-head:a26013145cfa0e710cc401bf086a68e2a7f215e4 -->
<!-- omni-reviewbot:quality-note-state:retracted -->
=== Bounty-hunter ===
is prefix cache depend on https://github.com/vllm-project/vllm-omni/pull/7719 ?
=== SamitHuang ===
Pls resolve the conflicts for further review, thx
artifacts/pr7759-review/review_comments.txt
-93
View File
@@ -1,93 +0,0 @@
=== id:4043203379 | chatgpt-codex-connector[bot] | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:1247 | reply_to:null ===
**<sub><sub>![P1 Badge](https://img.shields.io/badge/P1-orange?style=flat)</sub></sub> Keep new requests out of populated KV-cache batches**
With step execution and `max_num_seqs > 1`, the scheduler explicitly admits compatible waiting requests alongside already-running requests, so a request arriving after the first denoise step produces exactly this mixed `prefill`/`decode` phase set and aborts the whole batch here. Since the recipe advertises continuous step execution with `--max-num-seqs 8`, the pipeline must either handle mixed phases or prevent such requests from joining an in-flight batch rather than raising.
Useful? React with 👍 / 👎.
=== id:4043203389 | chatgpt-codex-connector[bot] | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:193 | reply_to:null ===
**<sub><sub>![P1 Badge](https://img.shields.io/badge/P1-orange?style=flat)</sub></sub> Partition batches by image-conditioning layout**
The preprocessor never sets `request.batch_compatibility_key`, so the generic request and step schedulers consider text-only requests, image-conditioned requests, and requests with different condition-image layouts compatible when their sampling parameters match. Such requests are then placed in one batch and rejected later by `_prepare_generation_context` or `denoise_step`, turning otherwise valid concurrent requests into batch-wide failures; record the conditioning mode, image count, and resized slot sizes in the compatibility key on both the empty-image and image paths.
Useful? React with 👍 / 👎.
=== id:4043203392 | chatgpt-codex-connector[bot] | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:693 | reply_to:null ===
**<sub><sub>![P1 Badge](https://img.shields.io/badge/P1-orange?style=flat)</sub></sub> Preserve request order when duplicating condition latents**
When a request batch contains more than one prompt and `num_outputs_per_prompt > 1`, prompt embeddings are ordered per request (`A, A, B, B`), but concatenating the encoded batch repeatedly orders condition latents as `A, B, A, B`. The second and third generations therefore receive another request's condition image; duplicate each encoded row consecutively so its order matches the repeated prompt embeddings and output splitting.
Useful? React with 👍 / 👎.
=== id:4043203401 | chatgpt-codex-connector[bot] | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:147 | reply_to:null ===
**<sub><sub>![P2 Badge](https://img.shields.io/badge/P2-yellow?style=flat)</sub></sub> Avoid truth-testing tensor and ndarray image inputs**
For a condition image supplied as a multi-element `torch.Tensor` or `numpy.ndarray`, this truth-value check raises an ambiguity error before `_to_pil` can process it, despite `_to_pil` explicitly supporting both input types. Check explicitly for `None` or an empty list instead so these supported image representations reach conversion.
Useful? React with 👍 / 👎.
=== id:4043275121 | SamitHuang | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:283 | reply_to:null ===
This pipeline is the entire new model surface (~1.3k lines here, plus transformer/VAE/CUDA graph) and the PR adds **zero tests**. Commits `4fc3cf8` / `3affc77` mention Qwen-Image-2.1 unit tests and then `clean tests` removes them from the branch. Add at least (1) unit tests for preprocess, `prepare_latents` duplication, mixed-phase `_assemble_kv_cache`, and VAE `spatial_compression_ratio=16`, and (2) a `tests/e2e/online_serving/test_qwen_image_21.py` smoke wired into `.buildkite/cuda/test-ready.yml` the same way `test_qwen_image.py` is. Without that, the TP / VAE-tiling / CUDA-graph claims in the commit messages are not merge-blocking evidence.
=== id:4043275125 | SamitHuang | vllm_omni/diffusion/models/qwen_image_21/decode_graph.py:274 | reply_to:null ===
`register_prefill` copies prefix K/V into a **shared** entry keyed only by `(branch, batch, prefix_len, target_tokens, dtype, device, backend)` and then **re-points every caller’s `kv_cache` dict at those tensors**. A second in-flight request with the same key `copy_`s over the first request’s prefix while the first request’s cache still aliases the same storage, so decode attends to another request’s text/image tokens. Sequential reuse of one buffer is fine; concurrent ownership is not. Keep per-request (or per-generation) static buffers, or refuse CUDA-graph decode whenever more than one live cache aliases the same key. This interacts directly with the recipe’s `--max-num-seqs 8` serving path.
=== id:4043275130 | SamitHuang | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:693 | reply_to:null ===
`encode_prompt` expands with `prompt_embeds.repeat(1, num_images_per_prompt, 1).view(batch * n, ...)` which is request-major (`A, A, B, B`). `torch.cat([encoded] * (batch_size // encoded.shape[0]), dim=0)` is tile-major (`A, B, A, B`). For a 2-prompt batch with `num_outputs_per_prompt=2`, generations 2 and 3 therefore get the other request’s condition latents (and `split_diffusion_output_by_request` will pair them as if they were correct). Use `repeat_interleave(repeats=n, dim=0)` (or encode-then-repeat per row) so condition latents match the prompt-embed order. This needs a unit test with `batch_size=2, n=2`.
=== id:4043275138 | SamitHuang | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:147 | reply_to:null ===
`if not raw_image` is an ambiguous truth-value check. `_to_pil` documents `torch.Tensor` and `numpy.ndarray` inputs; a multi-element tensor/array raises `RuntimeError: Boolean value of Tensor with more than one value is ambiguous` before conversion. Treat T2I as `raw_image is None or raw_image == []` (and maybe empty tuple), then always wrap a single image in a list. Same pattern should apply to whatever OpenAI image-edit path feeds tensors into `multi_modal_data["image"]`.
=== id:4043275145 | SamitHuang | recipes/Qwen/Qwen-Image-2.1.md:107 | reply_to:null ===
This example advertises `--step-execution --max-num-seqs 8`, but `QwenImage21Pipeline._assemble_kv_cache` raises if a new request’s first denoise step joins a batch that is already in decode (`Cannot batch requests at mixed KV-cache phases`). With the default step scheduler admitting waiting requests into a running batch, that is the normal continuous-batching case, not an edge case. Either keep mixed-phase requests out of the same `InputBatch` (and document the restriction here), or do not publish this command until that TODO is implemented. As written, following the recipe under load fails the whole batch.
=== id:4043327545 | SamitHuang | vllm_omni/diffusion/data.py:909 | reply_to:null ===
the CUDA graph for decode seems only used by QwenImage2.1? It's better to claim how it works together with eager mode or cudagraph_mode.
=== id:4043342420 | SamitHuang | examples/offline_inference/text_to_image/text_to_image.py:1 | reply_to:null ===
can provide a `examples/offline_inference/image_to_image/run_qwen_image_21.sh` script as well
=== id:4043666922 | hsliuustc0106 | vllm_omni/diffusion/data.py:909 | reply_to:null ===
This flag has no ingress, so the CUDA-graph decode path is unreachable for users: it's not in `OmniEngineArgs`/`StageDeployConfig`/`_DiffusionConfigProjection` (contrast `vae_use_tiling`, which flows through all three), and `Omni(enable_cuda_graph_decode=True)` is silently dropped by `from_kwargs`' field-name filter (`data.py:1604`). The only writer is the `False` default, so `decode_graph.py` + the transformer machinery only run under tests. Either wire it like `enforce_eager`/`vae_use_tiling` (deploy knob + projection) or document it as test-only in the recipe — the body's checked "[x] CUDA Graph for decode steps" reads as user-available today.
=== id:4043667674 | hsliuustc0106 | vllm_omni/diffusion/models/qwen_image_21/pipeline_qwen_image_21.py:1246 | reply_to:null ===
Reachable under the recipe's own `--step-execution --max-num-seqs 8`, and it fails the whole wave, not just the joiner: the default scheduler admits waiting requests while running ones are mid-generation (`base_scheduler.py` schedules running first, then admits waiting while capacity remains; `_can_schedule_waiting` compares only `StepBatchSamplingParamsKey` — no KV phase), and a `denoise_step` exception terminally fails every scheduled request via `diffusion_engine.py`'s `Execution failed for diffusion requests ...` handler. So one fresh arrival converts all in-flight requests into errors. Fix direction: phase-aware admission (defer joins while any running request is past step 0) or a per-model `batch_compatibility_key`; until then the features-table ✅ for Step Execution and the recipe's serving block overstate support vs. the unchecked TODO box.
=== id:4043668653 | hsliuustc0106 | examples/offline_inference/image_to_image/qwen_image_21_edit.py:1 | reply_to:null ===
Blocking per the examples policy: a newly added model-specific Python example (the file names one model family), while this directory's convention is the generic runner (`image_edit.py`, which dispatches via `get_model_class_name`/`build_image_to_image_prompt` — the same helpers this script calls). Smallest fix: register a `QwenImage21Pipeline` spec in `model_extras/_EXTRA_SPECS` (it's missing, so `build_image_to_image_prompt` currently falls through to the generic builder and silently ignores the `height`/`width` args this script passes) and fold this into `image_edit.py`; drop the model-specific file.
=== id:4043668703 | hsliuustc0106 | vllm_omni/diffusion/models/qwen_image_21/__init__.py:1 | reply_to:null ===
No tests anywhere in the PR. Sibling new models each ship a `tests/diffusion/models/<model>/` suite (sensenova_u1, glm_image, boogu_image incl. registry tests, ovis_image, z_image), and no CI lane touches `qwen_image_21` today. Given the new surface — registry entry, pre/post-process funcs, KV-cache prefill/decode protocol, step-mode merge/scatter, graph fallback gates — at least a CPU-runnable registry + loading + pre/post-process test would protect it.
=== id:4043668767 | hsliuustc0106 | recipes/Qwen/Qwen-Image-2.1.md:3 | reply_to:null ===
Missing the head-vs-reference comparison the model-addition checklist requires: the body's Test Plan is empty and there's no diffusers-#14804 quality/latency A/B (same checkpoint, inputs, seed, precision, hardware) — the recipe's only number is the tile-size PSNR probe. The checked TODO boxes (TP=2/4 vs single-card output, VAE tiling, CUDA graphs) also have no attached measurements. Please add the reference A/B with exact commands and both sides' outputs, or state why it can't run yet.
=== id:4044943764 | RuixiangMa | tests/diffusion/models/qwen_image_21/test_batch_compatibility.py:1 | reply_to:null ===
Test coverage is insufficient.
=== id:4045080762 | RuixiangMa | vllm_omni/diffusion/distributed/autoencoders/autoencoder_kl_qwenimage21.py:2 | reply_to:null ===
```suggestion
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
```
=== id:4045084688 | RuixiangMa | vllm_omni/diffusion/models/qwen_image_21/__init__.py:2 | reply_to:null ===
```suggestion
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
```
=== id:4059060032 | FayeSpica | vllm_omni/diffusion/models/qwen_image_21/qwen_image_21_transformer.py:388 | reply_to:null ===
```suggestion
height_index = torch.where(height_index < 0, height_index + self.freqs[1].shape[0], height_index)
width_index = torch.where(width_index < 0, width_index + self.freqs[2].shape[0], width_index)
return torch.cat(
[
self.freqs[0].index_select(0, frame_index),
self.freqs[1].index_select(0, height_index),
self.freqs[2].index_select(0, width_index),
],
dim=-1,
)
```
Could we use `torch.index_select` here for NPU compatibility, similar to #6320? This was the only issue preventing the model from starting on NPU in my tests.