vllm-omni/examples/online_serving/text_to_image/README.md

# Text-To-Image

This example demonstrates online text-to-image generation with vLLM-Omni. The
startup script and Python clients default to Qwen-Image, but `MODEL` can select
any supported text-to-image model. The existing curl helper remains a concrete
Qwen-Image request through the standard Images API.

## Start Server

### Basic Start

```bash
vllm serve Qwen/Qwen-Image --omni --port 8091
```
!!! note
    If you encounter Out-of-Memory (OOM) issues or have limited GPU memory, you can enable VAE slicing and tiling to reduce memory usage, --vae-use-slicing --vae-use-tiling

### Start with Parameters

Or use the startup script:

```bash
bash run_server.sh
```

### Start with Parallelism Acceleration

Enable Tensor Parallelism and VAE Patch Parallelism for faster inference:

```bash
# With Tensor Parallelism (requires >= 2 GPUs)
vllm serve Qwen/Qwen-Image --omni --port 8091 --tensor-parallel-size 2

# With Tensor Parallelism and VAE Patch Parallelism (requires >= 2 GPUs)
vllm serve Qwen/Qwen-Image --omni --port 8091 --tensor-parallel-size 2 --vae-patch-parallel-size 2 --vae-use-tiling

# With Sequence Parallelism (Ulysses-SP, requires >= 2 GPUs)
vllm serve Qwen/Qwen-Image --omni --port 8091 --usp 2

# With Ring-Attention (requires >= 2 GPUs)
vllm serve Qwen/Qwen-Image --omni --port 8091 --ring 2

# Combined: Ulysses + Ring (requires >= 4 GPUs)
vllm serve Qwen/Qwen-Image --omni --port 8091 --usp 2 --ring 2
```

For more details on parallelism acceleration, see the [Parallelism Acceleration Guide](../../diffusion/parallelism_acceleration.md).

## API Calls

### Method 1: Using curl

```bash
# Basic text-to-image generation
bash run_curl_text_to_image.sh

# Or execute directly
curl -s http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "A beautiful landscape painting"}
    ],
    "extra_body": {
      "height": 1024,
      "width": 1024,
      "num_inference_steps": 50,
      "true_cfg_scale": 4.0,
      "seed": 42
    }
  }' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
```

### Method 2: Using OpenAI Python SDK

```python
from openai import OpenAI
import base64

client = OpenAI(base_url="http://localhost:8091/v1", api_key="none")

response = client.chat.completions.create(
    model="Qwen/Qwen-Image",
    messages=[{"role": "user", "content": "A beautiful landscape painting"}],
    extra_body={
        "height": 1024,
        "width": 1024,
        "num_inference_steps": 50,
        "true_cfg_scale": 4.0,
        "seed": 42,
    },
)

img_url = response.choices[0].message.content[0].image_url.url
_, b64_data = img_url.split(",", 1)
with open("output.png", "wb") as f:
    f.write(base64.b64decode(b64_data))
```

!!! note
    The OpenAI SDK's `extra_body` keyword argument merges parameters into the
    top-level request body automatically. When using curl or Python `requests`,
    wrap generation parameters inside a literal `"extra_body"` key in the JSON
    instead (as shown in the curl example above).

### Method 3: Using Python Client Script

```bash
python openai_chat_client.py --prompt "A beautiful landscape painting" --output output.png
```

### Method 4: Using Gradio Demo

```bash
python gradio_demo.py
# Visit http://localhost:7860
```

## LoRA

This example supports both startup-time distilled LoRAs and request-time
PEFT-compatible adapters. The LoRA path must be readable on the **server**
machine (usually a local path or a mounted directory).

### Start Server with a Distilled LoRA

```bash
vllm serve Qwen/Qwen-Image-2512 \
  --omni \
  --port 8091 \
  --lora-backend distill \
  --lora-path /path/to/Qwen-Image-2512-Lightning-4steps.safetensors
```

The distilled LoRA is fused once during server initialization and applies to
every request. Use the checkpoint's few-step sampling settings, for example
`num_inference_steps=4` and `true_cfg_scale=1.0`.

### Using Python Client with LoRA

```bash
python openai_chat_client.py \
  --prompt "A piece of cheesecake" \
  --lora-path /path/to/lora_adapter \
  --lora-name my_lora \
  --lora-scale 1.0 \
  --output output.png
```

### Using curl with LoRA (Images API)

The `/v1/images/generations` endpoint supports a `lora` field in the request body:

```bash
curl -X POST http://localhost:8091/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "A piece of cheesecake",
    "size": "1024x1024",
    "seed": 42,
    "lora": {
      "name": "my_lora",
      "local_path": "/path/to/lora_adapter",
      "scale": 1.0
    }
  }' | jq -r '.data[0].b64_json' | base64 -d > output.png
```

### LoRA Parameters

| Parameter    | Type  | Description                                                               |
| ------------ | ----- | ------------------------------------------------------------------------- |
| `name`       | str   | LoRA adapter name (optional, defaults to path stem)                       |
| `local_path` | str   | Server-local path to LoRA adapter folder (PEFT format, required)          |
| `scale`      | float | LoRA scale factor (default: 1.0)                                          |
| `int_id`     | int   | LoRA integer ID for caching (optional, derived from path if not provided) |

### LoRA Adapter Format

LoRA adapters must be in PEFT (Parameter-Efficient Fine-Tuning) format. A typical LoRA adapter directory structure:

```
lora_adapter/
├── adapter_config.json
└── adapter_model.safetensors
```

## Request Format

### Simple Text Generation

```json
{
  "messages": [
    {"role": "user", "content": "A beautiful landscape painting"}
  ]
}
```

### Generation with Parameters

Use `extra_body` to pass generation parameters:

```json
{
  "messages": [
    {"role": "user", "content": "A beautiful landscape painting"}
  ],
  "extra_body": {
    "height": 1024,
    "width": 1024,
    "num_inference_steps": 50,
    "true_cfg_scale": 4.0,
    "seed": 42
  }
}
```

### Multimodal Input (Text + Structured Content)

```json
{
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "A beautiful landscape painting"}
      ]
    }
  ]
}
```

## Generation Parameters

When using `/v1/chat/completions`, pass these inside `extra_body` in the curl
JSON, or via the `extra_body` keyword argument in the OpenAI Python SDK.
When using the dedicated `/v1/images/generations` endpoint, pass the supported
generation controls as top-level JSON fields directly. For image dimensions and
count, use `size` and `n` rather than `height`, `width`, or
`num_outputs_per_prompt`.

| Parameter                | Type  | Default | Description                    |
| ------------------------ | ----- | ------- | ------------------------------ |
| `height`                 | int   | None    | Image height in pixels         |
| `width`                  | int   | None    | Image width in pixels          |
| `size`                   | str   | None    | Image size (e.g., "1024x1024") |
| `num_inference_steps`    | int   | model   | Number of denoising steps      |
| `true_cfg_scale`         | float | model   | True CFG scale when supported  |
| `seed`                   | int   | None    | Random seed (reproducible)     |
| `negative_prompt`        | str   | None    | Negative prompt                |
| `num_outputs_per_prompt` | int   | 1       | Number of images to generate   |
| `use_system_prompt` | str | None | System prompt preset: `en_unified`, `en_vanilla`, `en_recaption`, `en_think_recaption`, `dynamic`, `None`, or custom text string. Only for HunyuanImage-3.0. |
| `system_prompt` | str | None | Custom system prompt text. Only used when `use_system_prompt` is set to `custom`. Only for HunyuanImage-3.0. |

## Response Format

```json
{
  "id": "chatcmpl-xxx",
  "created": 1234567890,
  "model": "Qwen/Qwen-Image",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": [{
        "type": "image_url",
        "image_url": {
          "url": "data:image/png;base64,..."
        }
      }]
    },
    "finish_reason": "stop"
  }],
  "usage": {...}
}
```

## Extract Image

```bash
# Extract base64 from response and decode to image
cat response.json | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
```

## File Description

| File                        | Description                  |
| --------------------------- | ---------------------------- |
| `run_server.sh`             | Server startup script        |
| `run_curl_text_to_image.sh` | curl example                 |
| `openai_chat_client.py`     | Python client                |
| `gradio_demo.py`            | Gradio interactive interface |