From 679cbe226aed110efa3acbdcaf33fdbd591e8420 Mon Sep 17 00:00:00 2001 From: Honghan Zhu Date: Wed, 17 Jun 2026 17:04:58 +0800 Subject: [PATCH] [Doc] Qwen image 2512 recipe (#4236) Signed-off-by: dph97 Co-authored-by: Claude Opus 4.8 Co-authored-by: zhuhonghan --- recipes/Qwen/Qwen-Image-2512.md | 289 ++++++++++++++++++++++++++++++++ recipes/README.md | 2 +- 2 files changed, 290 insertions(+), 1 deletion(-) create mode 100644 recipes/Qwen/Qwen-Image-2512.md diff --git a/recipes/Qwen/Qwen-Image-2512.md b/recipes/Qwen/Qwen-Image-2512.md new file mode 100644 index 000000000..3b7d23302 --- /dev/null +++ b/recipes/Qwen/Qwen-Image-2512.md @@ -0,0 +1,289 @@ +# Qwen-Image-2512 + +> Text-to-image serving for the 2512 (Dec-2025) refresh of Qwen-Image + +## Summary + +- Vendor: Qwen +- Model: `Qwen/Qwen-Image-2512` +- Task: Text-to-image generation +- Mode: Online serving with optional step-wise continuous batching +- Maintainer: Community + +## When to use this recipe + +Use this recipe when you want to serve the newer `Qwen/Qwen-Image-2512` +checkpoint instead of the original `Qwen/Qwen-Image`. `Qwen-Image-2512` is a +drop-in updated checkpoint that runs on the **same** `QwenImagePipeline`, so the +serving path, flags, and clients are identical to +[`Qwen-Image.md`](./Qwen-Image.md) — only the model id changes. Pick this recipe +to validate the normal online-serving path on a single 80 GB A800, or to run the +validated 2-GPU ModelOpt mixed FP8/NVFP4 quantized variant on B200. + +## Difference from Qwen-Image + +`Qwen-Image-2512` requires **no code, registry, or pipeline changes** relative to +`Qwen-Image`: + +- Same architecture / pipeline: `QwenImagePipeline` + (`vllm_omni/diffusion/registry.py`, see + [`docs/models/supported_models.md`](../../docs/models/supported_models.md)). +- Same serving entrypoint, CLI flags, OpenAI image-generation API, example + clients, and benchmark scripts. + +The only change is the **model id**: swap `Qwen/Qwen-Image` for +`Qwen/Qwen-Image-2512` everywhere (serve command, API `model` field, client +`--model`, benchmark `--model`). Everything else in +[`Qwen-Image.md`](./Qwen-Image.md) applies unchanged. + +## References + +- Sibling recipe (canonical, identical serving path): + [`recipes/Qwen/Qwen-Image.md`](./Qwen-Image.md) +- Upstream or canonical docs: + [`docs/user_guide/examples/online_serving/text_to_image.md`](../../docs/user_guide/examples/online_serving/text_to_image.md) +- Related example under `examples/`: + [`examples/online_serving/text_to_image/README.md`](../../examples/online_serving/text_to_image/README.md) +- Related benchmark: + [`benchmarks/diffusion/diffusion_benchmark_serving.py`](../../benchmarks/diffusion/diffusion_benchmark_serving.py) +- Supported models table: + [`docs/models/supported_models.md`](../../docs/models/supported_models.md) + +## Hardware Support + +This recipe documents a single-GPU BF16 serving configuration on A800 and a +validated 2-GPU ModelOpt mixed FP8/NVFP4 configuration on B200. Extend it with +more hardware sections as community validation lands. + +## GPU + +### 1x A800-SXM4-80GB + +#### Environment + +- OS: Linux +- Python: 3.12.13 +- PyTorch: 2.11.0+cu130 +- CUDA runtime: 13.0 (driver 580.126.09) +- GPU: NVIDIA A800-SXM4-80GB (81920 MiB) +- vLLM: 0.22.0 +- Transformers: 5.8.1 +- vLLM-Omni: v0.22.0rc1-70-ge91efc24 + +#### Command + +Start the baseline server: + +```bash +vllm serve Qwen/Qwen-Image-2512 --omni --port 8091 +``` + +To enable the step-wise runtime without batching: + +```bash +vllm serve Qwen/Qwen-Image-2512 --omni \ + --port 8091 \ + --step-execution \ + --max-num-seqs 1 +``` + +To enable experimental compatible-request batching: + +```bash +vllm serve Qwen/Qwen-Image-2512 --omni \ + --port 8091 \ + --step-execution \ + --max-num-seqs 8 +``` + +You can also use the example launcher and pass the model id and extra flags +through: + +```bash +bash examples/online_serving/text_to_image/run_server.sh \ + --model Qwen/Qwen-Image-2512 --step-execution --max-num-seqs 8 +``` + +#### Verification + +Run the existing client example after the server is ready: + +```bash +python examples/online_serving/text_to_image/openai_chat_client.py \ + --server http://localhost:8091 \ + --prompt "A ceramic teapot on a wooden table" \ + --output /tmp/qwen_image_2512_recipe.png +``` + +For a direct API smoke test: + +```bash +curl http://localhost:8091/v1/images/generations \ + -H "Content-Type: application/json" \ + -d '{ + "model": "Qwen/Qwen-Image-2512", + "prompt": "A ceramic teapot on a wooden table", + "size": "1024x1024", + "num_inference_steps": 20, + "seed": 42 + }' +``` + +To replay the batching benefit with matched warmup: + +```bash +python benchmarks/diffusion/diffusion_benchmark_serving.py \ + --endpoint /v1/chat/completions \ + --dataset vbench \ + --task t2i \ + --model Qwen/Qwen-Image-2512 \ + --num-prompts 10 \ + --height 1024 \ + --width 1024 \ + --num-inference-steps 50 \ + --seed 42 \ + --port 8089 \ + --max-concurrency 8 \ + --warmup-requests 8 \ + --warmup-concurrency 8 \ + --warmup-num-inference-steps 3 \ + --disable-tqdm +``` + +Run that once against `--max-num-seqs 1`, then rerun it against `--max-num-seqs 8` +and compare the output JSON or terminal metrics. + +#### Benchmark + +The numbers below are loaded-once request-level measurements, not an HTTP +concurrency benchmark. The model is loaded once, warmup requests are run, +then 100 sequential requests are measured. + +```text +Hardware: 1x A800-SXM4-80GB +Requests: 100 +Concurrency: 1 +Resolution: 1024x1024 +Denoising steps: 20 +``` + +| Config | Mean | P95 | P99 | Peak VRAM | +| --- | ---: | ---: | ---: | ---: | +| BF16 | 7.669s | 7.736s | 7.748s | 58,979 MiB | + +#### Notes + +- Memory usage: BF16 peak VRAM ~59 GiB on A800, leaving ~22 GiB headroom for + compile overhead and image decode. +- Key flags: `--step-execution` enables the step-wise runtime; `--max-num-seqs` + controls how many compatible requests may stay active together. +- Keep `--max-num-seqs 1` when you want the more conservative path, when traffic + is mostly single-request, or when you are debugging correctness before + measuring throughput. +- Current batching is still shape-sensitive: different step progress can + co-batch, but different resolutions do not yet co-batch. + +### 2x B200 ModelOpt mixed FP8/NVFP4 + +#### Environment + +- OS: Linux +- Python: 3.12 +- Driver / runtime: NVIDIA CUDA environment with two B200 GPUs +- vLLM version: Match the vLLM-Omni checkout used for deployment +- vLLM-Omni version or commit: Use the commit that contains ModelOpt mixed + FP8/NVFP4 checkpoint loading for diffusion models + +#### Command + +This configuration uses a pre-quantized ModelOpt mixed checkpoint for +Qwen-Image-2512. The validated checkpoint keeps boundary, modulation, +normalization, and output layers in BF16, uses FP8 for sensitive attention +projections, and uses NVFP4 for heavier MLP / attention-output linear layers. + +Validated checkpoint: + +```text +feizhai123/qwen-image-2512-modelopt-mixed-fp8-sensitive-nvfp4-heavy +``` + +Start the server with the CUTLASS quantized linear backend: + +```bash +CUDA_VISIBLE_DEVICES=2,3 \ +vllm serve feizhai123/qwen-image-2512-modelopt-mixed-fp8-sensitive-nvfp4-heavy \ + --omni \ + --host 0.0.0.0 \ + --port 8091 \ + --trust-remote-code \ + --tensor-parallel-size 2 \ + --linear-backend cutlass \ + --force-cutlass-fp8 +``` + +#### Verification + +Run a direct image generation request after the server is ready: + +```bash +curl http://localhost:8091/v1/images/generations \ + -H "Content-Type: application/json" \ + -d '{ + "model": "feizhai123/qwen-image-2512-modelopt-mixed-fp8-sensitive-nvfp4-heavy", + "prompt": "A cinematic Warhammer 40,000 Ultramarine warrior in blue power armor, armor stained with alien ichor, swinging a chainsword at a Tyranid creature, gothic battlefield, dramatic lighting, high detail, no text, no watermark", + "negative_prompt": "blurry, low quality, text, watermark, logo, malformed armor, bad hands", + "size": "1024x1024", + "num_inference_steps": 20, + "true_cfg_scale": 4.0, + "guidance_scale": 4.0, + "seed": 48 + }' +``` + +Check that: + +- The server responds on `http://localhost:8091/health`. +- The response contains one generated image. +- Logs show CUTLASS selection for ModelOpt FP8 / mixed-precision linear + layers instead of falling back to the default automatic backend choice. + +#### Benchmark + +The numbers below are loaded-once request-level measurements, not an HTTP +concurrency benchmark. Each case loads the model once, runs warmup requests, +then measures 100 sequential requests. + +```text +Hardware: 2x B200 +Tensor parallel size: 2 +Requests: 100 +Concurrency: 1 +Resolution: 1024x1024 +Denoising steps: 20 +``` + +| Config | Mean | P90 | P99 | Peak VRAM | +| --- | ---: | ---: | ---: | ---: | +| BF16 | 2.696s | 2.700s | 2.707s | 99,000 MiB | +| ModelOpt FP8 CUTLASS | 2.663s | 2.671s | 2.676s | 85,760 MiB | +| Mixed FP8/NVFP4 CUTLASS | 2.584s | 2.590s | 2.594s | 84,412 MiB | + +The same 2-GPU setup was also used to sweep NVFP4 linear backends. Direct +CUTLASS was the fastest validated backend for this checkpoint. + +| Backend | Mean | P90 | P99 | Peak VRAM | +| --- | ---: | ---: | ---: | ---: | +| auto | 2.927s | 2.948s | 3.535s | 85,340 MiB | +| flashinfer-cudnn | 3.523s | 3.528s | 3.766s | 84,500 MiB | +| flashinfer-trtllm | 2.711s | 2.727s | 3.010s | 84,996 MiB | +| flashinfer-cutlass | 2.935s | 2.950s | 3.011s | 85,340 MiB | +| cutlass | 2.578s | 2.584s | 2.592s | 84,412 MiB | + +#### Notes + +- Use `--linear-backend cutlass` to select the validated quantized linear + backend from the CLI. +- `--force-cutlass-fp8` keeps ModelOpt FP8 layers on the CUTLASS FP8 path. +- The benchmark table intentionally reports Qwen-Image-2512 as + concurrency-1 request-level data; do not compare it directly with an online + HTTP concurrency benchmark. diff --git a/recipes/README.md b/recipes/README.md index 94cbcdbd5..a1c6973d5 100644 --- a/recipes/README.md +++ b/recipes/README.md @@ -42,7 +42,7 @@ recipes/ | [`OpenBMB/MiniCPM-o-4_5.md`](./OpenBMB/MiniCPM-o-4_5.md) | Online serving for omni multimodal chat (text / image / audio / video → text + 24 kHz speech) | 2x A100/H100 80GB / 3x mid-tier GPU / 8x RTX 4090 24GB | | [`OpenBMB/VoxCPM2.md`](./OpenBMB/VoxCPM2.md) | Online + offline TTS with native AR pipeline (48 kHz, 30+ languages) | 1x RTX 4090 24GB | | [`Qwen/Qwen-Image.md`](./Qwen/Qwen-Image.md) | Text-to-image serving with step-wise continuous batching replay and ModelOpt mixed FP8/NVFP4 | 1x A100 80GB / 2x B200 | -| [`Qwen/Qwen-Image.md`](./Qwen/Qwen-Image.md) | Text-to-image serving with step-wise continuous batching replay | 1x A100 80GB | +| [`Qwen/Qwen-Image-2512.md`](./Qwen/Qwen-Image-2512.md) | Text-to-image serving with step-wise continuous batching replay and ModelOpt FP8 / mixed FP8/NVFP4 | 1x A800 80GB / 2x B200 | | [`Qwen/Qwen-Image-Edit.md`](./Qwen/Qwen-Image-Edit.md) | Text-guided single-image editing | 1x or 2x H200 141GB | | [`Qwen/Qwen3-Omni.md`](./Qwen/Qwen3-Omni.md) | Online serving for multimodal chat | 1x A100 80GB | | [`Qwen/Qwen3-TTS.md`](./Qwen/Qwen3-TTS.md) | Text-to-speech serving (CustomVoice / VoiceDesign / Base) | 1x H100/A100 80GB |