vllm-omni/examples/offline_inference/text_to_image

Text-To-Image

This folder provides several entrypoints for experimenting with Qwen/Qwen-Image Qwen/Qwen-Image-2512 Tongyi-MAI/Z-Image-Turbo stepfun-ai/NextStep-1.1 using vLLM-Omni, note that NextStep-1.1 has different architecture so we treat it differently regarding running arguments and pipeline.

• text_to_image.py: command-line script for single image generation with advanced options.
• web_demo.py: lightweight Gradio UI for interactive prompt/seed/CFG exploration.

Note that when you pass in multiple independent prompts, they will be processed sequentially. Batching requests is currently not supported.

Basic Usage

from vllm_omni.entrypoints.omni import Omni

if __name__ == "__main__":
    omni = Omni(model="Qwen/Qwen-Image")
    prompt = "a cup of coffee on the table"
    outputs = omni.generate(prompt)
    images = outputs[0].request_output.images
    images[0].save("coffee.png")

Or put more than one prompt in a request.

from vllm_omni.entrypoints.omni import Omni

if __name__ == "__main__":
    omni = Omni(model="Qwen/Qwen-Image")
    prompts = [
      "a cup of coffee on a table",
      "a toy dinosaur on a sandy beach",
      "a fox waking up in bed and yawning",
    ]
    outputs = omni.generate(prompts)
    for i, output in enumerate(outputs):
      image = output.request_output.images[0].save(f"{i}.jpg")

!!! info

However, it is not currently recommended to do so
because not all models support batch inference,
and batch requesting mostly does not provide significant performance improvement (despite the impression that it does).
This feature is primarily for the sake of interface compatibility with vLLM and to allow for future improvements.

!!! info

For diffusion pipelines, the stage config field `stage_args.[].engine_args.max_num_seqs` is 1 by default, and the input
list is sliced into single-item requests before feeding into the diffusion pipeline. For models that do internally support
batched inputs, you can [modify this configuration](../../../configuration/stage_configs.md) to let the model accept a longer batch of prompts.

Apart from string prompt, vLLM-Omni also supports dictionary prompts in the same style as vLLM. This is useful for models that support negative prompts.

from vllm_omni.entrypoints.omni import Omni

if __name__ == "__main__":
    omni = Omni(model="Qwen/Qwen-Image")
    outputs = omni.generate([
      {
        "prompt": "a cup of coffee on a table"，
        "negative_prompt": "low resolution"
      },
      {
        "prompt": "a toy dinosaur on a sandy beach"，
        "negative_prompt": "cinematic, realistic"
      }
    ])
    for i, output in enumerate(outputs):
      image = output.request_output.images[0].save(f"{i}.jpg")

Local CLI Usage

Qwen/Tongyi Models

python text_to_image.py \
  --model Tongyi-MAI/Z-Image-Turbo \
  --prompt "a cup of coffee on the table" \
  --seed 42 \
  --cfg-scale 4.0 \
  --num-images-per-prompt 1 \
  --num-inference-steps 50 \
  --height 1024 \
  --width 1024 \
  --output outputs/coffee.png

NextStep Models

NextStep-1.1 can have extra arguments

python text_to_image.py \
  --model stepfun-ai/NextStep-1.1 \
  --prompt "A baby panda wearing an Iron Man mask, holding a board with 'NextStep-1' written on it" \
  --height 512 \
  --width 512 \
  --num-inference-steps 28 \
  --guidance-scale 7.5 \
  --guidance-scale-2 1.0 \
  --cfg-schedule constant \
  --output nextstep_output.png \
  --seed 42

Flux.2-dev Models

To start Flux.2-dev with a single GPU, cpu-offload must be enabled because the total size of its weights exceeds the 80GB memory capacity of the GPU.

python examples/offline_inference/text_to_image/text_to_image.py \
  --model black-forest-labs/FLUX.2-dev \
  --prompt "a lovely bunny holding a sign that says 'vllm-omni'" \
  --seed 42 \
  --tensor-parallel-size 1 \
  --num-images-per-prompt 1 \
  --num-inference-steps 50 \
  --guidance-scale 4.0 \
  --height 1024 \
  --width 1024 \
  --enable-cpu-offload \
  --output flux2-dev.png

Key Arguments

Common arguments:

• --prompt: text description (string).
• --seed: integer seed for deterministic sampling.
• --cfg-scale: true CFG scale (model-specific guidance strength).
• --num-images-per-prompt: number of images to generate per prompt (saves as output, output_1, ...).
• --num-inference-steps: diffusion sampling steps (more steps = higher quality, slower).
• --height/--width: output resolution (defaults 1024x1024).
• --output: path to save the generated PNG.
• --vae-use-slicing: enable VAE slicing for memory optimization.
• --vae-use-tiling: enable VAE tiling for memory optimization.
• --cfg-parallel-size: set it to 2 to enable CFG Parallel. See more examples in user_guide.
• --enable-cpu-offload: enable CPU offloading for diffusion models.
• --guidance-scale: classifier-free guidance scale.

NextStep-1.1 specific:

• --guidance-scale-2: secondary guidance scale, e.g. image-level CFG (default: 1.0).
• --timesteps-shift: timesteps shift parameter for sampling (default: 1.0).
• --cfg-schedule: CFG schedule type, "constant" or "linear" (default: "constant").
• --use-norm: apply layer normalization to sampled tokens.

ℹ️ If you encounter OOM errors, try using --vae-use-slicing and --vae-use-tiling to reduce memory usage.

ℹ️ Qwen-Image currently publishes best-effort presets at 1328x1328, 1664x928, 928x1664, 1472x1140, 1140x1472, 1584x1056, and 1056x1584. Adjust --height/--width accordingly for the most reliable outcomes.

LoRA

This example supports Peft-compatible LoRA (Low-Rank Adaptation) adapters for diffusion models. Pass --lora-path to use a LoRA adapter and optionally --lora-scale (default 1.0); omit it to use the base model only.

Basic usage with LoRA

python text_to_image.py \
  --model Tongyi-MAI/Z-Image-Turbo \
  --prompt "A piece of cheesecake" \
  --lora-path /path/to/lora/ \
  --lora-scale 1.0 \
  --output output.png

LoRA parameters

• --lora-path: Path to LoRA adapter folder (PEFT format). Loaded at initialization and used for generation.
• --lora-scale: Scale factor for LoRA weights (default: 1.0). Higher values increase the influence of the LoRA adapter.

LoRA adapter format

LoRA adapters must be in PEFT (Parameter-Efficient Fine-Tuning) format. A typical LoRA adapter directory structure:

lora_adapter/
├── adapter_config.json
└── adapter_model.safetensors

Web UI Demo

Launch the gradio demo:

python gradio_demo.py --port 7862

Then open http://localhost:7862/ on your local browser to interact with the web UI.