vllm-omni/recipes/Qwen/Qwen-Image.md

Qwen-Image for text-to-image serving with optional continuous batching on 1x A100 80GB

Summary

• Vendor: Qwen
• Model: Qwen/Qwen-Image
• Task: Text-to-image generation
• Mode: Online serving with optional step-wise continuous batching
• Maintainer: Community

When to use this recipe

Use this recipe when you want a known-good starting point for serving Qwen/Qwen-Image on a single 80 GB A100, validate the normal online-serving path, and optionally replay the benefit of step-wise continuous batching with the benchmark assets already bundled in this repository.

References

• Upstream or canonical docs: docs/user_guide/examples/online_serving/text_to_image.md
• Related example under examples/: examples/online_serving/text_to_image/README.md
• Related benchmark: benchmarks/diffusion/diffusion_benchmark_serving.py

Hardware Support

This recipe currently documents one CUDA GPU serving configuration. Extend it with more hardware sections as community validation lands.

GPU

1x A100 80GB

Environment

• OS: Linux
• Python: 3.10+
• Driver / runtime: NVIDIA CUDA environment with an A100 80 GB GPU
• vLLM version: Match the repository requirements for your checkout
• vLLM-Omni version or commit: Use the commit you are deploying from

Command

Start the baseline server:

vllm serve Qwen/Qwen-Image --omni --port 8091

To enable the step-wise runtime without batching:

vllm serve Qwen/Qwen-Image --omni \
  --port 8091 \
  --step-execution \
  --max-num-seqs 1

To enable experimental compatible-request batching:

vllm serve Qwen/Qwen-Image --omni \
  --port 8091 \
  --step-execution \
  --max-num-seqs 8

You can also use the example launcher and pass the extra flags through:

bash examples/online_serving/text_to_image/run_server.sh --step-execution --max-num-seqs 8

Verification

Run the existing client example after the server is ready:

python examples/online_serving/text_to_image/openai_chat_client.py \
  --server http://localhost:8091 \
  --prompt "A ceramic teapot on a wooden table" \
  --output /tmp/qwen_image_recipe.png

For a direct API smoke test:

curl http://localhost:8091/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen-Image",
    "prompt": "A ceramic teapot on a wooden table",
    "size": "1024x1024",
    "num_inference_steps": 20,
    "seed": 42
  }'

To replay the batching benefit with matched warmup:

python benchmarks/diffusion/diffusion_benchmark_serving.py \
  --backend vllm-omni \
  --dataset vbench \
  --task t2i \
  --model Qwen/Qwen-Image \
  --num-prompts 10 \
  --height 1024 \
  --width 1024 \
  --num-inference-steps 50 \
  --seed 42 \
  --port 8089 \
  --max-concurrency 8 \
  --warmup-requests 8 \
  --warmup-concurrency 8 \
  --warmup-num-inference-steps 3 \
  --disable-tqdm

Run that once against --max-num-seqs 1, then rerun it against --max-num-seqs 8 and compare the output JSON or terminal metrics.

Notes

• Memory usage: keep headroom for first-request compile and image decode overhead.
• Key flags: --step-execution enables the step-wise runtime; --max-num-seqs controls how many compatible requests may stay active together.
• Keep --max-num-seqs 1 when you want the more conservative path, when traffic is mostly single-request, or when you are debugging correctness before measuring throughput.
• Current batching is still shape-sensitive: different step progress can co-batch, but different resolutions do not yet co-batch.