vllm-omni/recipes/Boogu/Boogu-Image.md

# Boogu-Image

> Text-to-image and image-editing online serving
(Boogu-Image-0.1-Base / -Turbo / -Edit / -Edit-Turbo)

## Summary

- Vendor: Boogu
- Model: `Boogu/Boogu-Image-0.1-Base` (text-to-image),
  `Boogu/Boogu-Image-0.1-Turbo` (four-step text-to-image),
  `Boogu/Boogu-Image-0.1-Edit` (image editing), and
  `Boogu/Boogu-Image-0.1-Edit-Turbo` (four-step image editing)
- Task: Text-to-image generation and text-guided image editing (TI2I)
- Mode: Online serving with the OpenAI-compatible API
- Maintainer: Community

## When to use this recipe

Use this recipe when you want a known-good starting point for serving
`Boogu/Boogu-Image-0.1-Base`, `Boogu/Boogu-Image-0.1-Turbo`,
`Boogu/Boogu-Image-0.1-Edit`, or `Boogu/Boogu-Image-0.1-Edit-Turbo` with
vLLM-Omni's native pipeline (no
`--diffusion-load-format diffusers`, and the upstream `boogu` package is not
required).

Boogu-Image-0.1 is an Apache-2.0 unified image generation and editing model
family. The Base text-to-image checkpoint pairs a Qwen3-VL multimodal encoder
with a Diffusion Transformer (DiT) and a flow-match Euler scheduler with
time-shift. It handles photorealistic generation and Chinese/English text
rendering. The Edit checkpoint uses the native `BooguImagePipeline`, while the
distilled Turbo and Edit-Turbo checkpoints run the four-step DMD path through
`BooguImageTurboPipeline`.

## References

- Upstream model card: <https://huggingface.co/Boogu/Boogu-Image-0.1-Base>
- Turbo model card: <https://huggingface.co/Boogu/Boogu-Image-0.1-Turbo>
- Edit model card: <https://huggingface.co/Boogu/Boogu-Image-0.1-Edit>
- Edit-Turbo 1K hotfix: <https://huggingface.co/Boogu/Boogu-Image-0.1-Edit-Turbo/tree/hotfix-1k-20260708>
- Project page: <https://boogu.org>
- GitHub: <https://github.com/boogu-project/Boogu-Image>
- Related example: [`examples/online_serving/text_to_image/`](../../examples/online_serving/text_to_image/README.md)

## Hardware Support

This recipe documents supported configurations for CUDA GPU serving. The native
pipeline supports single-GPU inference and classifier-free-guidance (CFG)
parallelism. CPU offload, cache acceleration, and other multi-GPU dimensions
remain unsupported (see Notes).

## GPU

### 1 x A100/H100 (Single GPU, 40GB+ VRAM)

The model footprint is roughly 34.6 GiB on GPU, so a 40GB+ card is recommended.

#### Command

```bash
vllm serve Boogu/Boogu-Image-0.1-Base --omni --port 8091
```

!!! note
    If you hit Out-of-Memory (OOM) on a smaller card, enable VAE slicing and
    tiling to reduce peak memory: `--vae-use-slicing --vae-use-tiling`.

#### Verification

After the server is ready, test with a simple request:

```bash
curl -X POST http://localhost:8091/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Boogu/Boogu-Image-0.1-Base",
    "prompt": "A mountain lake at sunset, photorealistic, cinematic lighting",
    "size": "1024x1024",
    "num_inference_steps": 28,
    "guidance_scale": 4.0,
    "seed": 42
  }' | jq -r '.data[0].b64_json' | base64 -d > output.png
```

Or via the chat-completions endpoint (parameters go in `extra_body`):

```bash
curl -s http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "A mountain lake at sunset, photorealistic, cinematic lighting"}
    ],
    "extra_body": {
      "height": 1024,
      "width": 1024,
      "num_inference_steps": 28,
      "guidance_scale": 4.0,
      "seed": 42
    }
  }' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
```

#### Notes

- **Memory usage:** ~34.6 GiB on GPU. Use `--vae-use-slicing --vae-use-tiling`
  to trim peak VRAM if needed.
- **Key flags:**
    - `--omni` — enables vLLM-Omni diffusion serving.
- **Guidance:** Boogu-Image uses `guidance_scale` (mapped to the upstream
  `text_guidance_scale`); the default is `4.0`. Classifier-free guidance is
  active whenever `guidance_scale > 1.0`.
- **Recommended settings:** `num_inference_steps=28`-`50`, `guidance_scale=4.0`.
  The model's maximum native resolution is 2K.
- **Known limitations (not yet supported):** CPU offload
  (`--enable-cpu-offload` / `--enable-layerwise-offload`), Cache-DiT
  (`--cache-backend cache_dit`), and TP / SP / HSDP multi-GPU parallelism.

### 2 x H100 (CFG parallel, Base T2I)

Base T2I has two predictions per guided denoising step, so two CFG ranks map
one-to-one to the positive and negative-text branches:

```bash
vllm serve Boogu/Boogu-Image-0.1-Base \
  --omni \
  --cfg-parallel-size 2 \
  --port 8091
```

Use `guidance_scale > 1.0` to activate the parallel two-branch path. Requests
with `guidance_scale=1.0` remain valid on the same server: every rank evaluates
only the positive branch, no negative embeddings are built, and no guidance is
applied.

### 1 x RTX 4090 48 GB (FP8)

Boogu-Image supports two ways to quantize or load the DiT transformer:

| Path               | Base model                       | Edit model                       | Method                       | Scheme                  |
| ------------------ | -------------------------------- | -------------------------------- | ---------------------------- | ----------------------- |
| Serialized TorchAO | `Boogu/Boogu-Image-0.1-Base-fp8` | `Boogu/Boogu-Image-0.1-Edit-fp8` | `torchao_float8_weight_only` | FP8 weight-only (W8A16) |
| Native online FP8  | `Boogu/Boogu-Image-0.1-Base`     | `Boogu/Boogu-Image-0.1-Edit`     | `fp8`                        | Dynamic W8A8            |

#### Environment

The following configuration was validated:

- OS: Linux
- Python: 3.12
- Driver / runtime: NVIDIA CUDA environment with one RTX 4090 48 GB GPU
- vLLM version: Match the vLLM-Omni checkout used for deployment
- vLLM-Omni version or commit: Use a commit that contains TorchAO FP8
  checkpoint loading for diffusion models
- TorchAO version: 0.17.0
- `kernels` / `kernels-data`: 0.16.1 / matching kernels-data

#### Command

Serve the official Base FP8 checkpoint using its pre-quantized TorchAO FP8
weight-only transformer weights:

```bash
vllm serve Boogu/Boogu-Image-0.1-Base-fp8 \
  --omni \
  --port 8091 \
  --diffusion-quantization-config \
  '{"transformer":{"method":"torchao_float8_weight_only"}}'
```

Native online FP8 (currently applied only to eligible vLLM-quantizable linear
layers in the DiT transformer):

```bash
vllm serve Boogu/Boogu-Image-0.1-Base \
  --omni \
  --port 8091 \
  --diffusion-quantization-config \
  '{"transformer":{"method":"fp8"}}'
```

For image editing, use the matching Edit model ID from the table and keep the
corresponding quantization configuration.

#### Verification

The request format is shared by both FP8 paths. Set `<MODEL_ID>` to the model
ID passed to `vllm serve`. For example, use
`Boogu/Boogu-Image-0.1-Base-fp8` for serialized TorchAO FP8 or
`Boogu/Boogu-Image-0.1-Base` for native online FP8. For image editing, use the
corresponding Edit model.

Base text-to-image:
## Fast text-to-image (Boogu-Image-0.1-Turbo)

Turbo is the distilled text-to-image checkpoint. Its `model_index.json`
declares `BooguImageTurboPipeline`, which runs the four-step DMD path instead
of the scheduler-driven path used by Base.

### Command

```bash
vllm serve Boogu/Boogu-Image-0.1-Turbo --omni --port 8091
```

### Verification

```bash
curl -X POST http://localhost:8091/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<MODEL_ID>",
    "prompt": "A mountain lake at sunset, photorealistic, cinematic lighting",
    "size": "1024x1024",
    "num_inference_steps": 28,
    "guidance_scale": 4.0,
    "seed": 42
  }' | jq -r '.data[0].b64_json' | base64 -d > output_fp8.png
```

Image editing:

```bash
curl -X POST http://localhost:8091/v1/images/edits \
  -F model="<MODEL_ID>" \
  -F image="@input.png" \
  -F prompt="Change the style to a colored pencil drawing." \
  -F num_inference_steps=28 \
  -F guidance_scale=4.0 \
  -F guidance_scale_2=1.0 \
  -F seed=42 \
  | jq -r '.data[0].b64_json' | base64 -d > edited_fp8.png
```

#### Notes

- **Memory usage:**
    - The serialized TorchAO FP8 path uses approximately 21.1 GiB at 512x512
      and 24.2-24.6 GiB at 1024x1024.
    - The native online FP8 path uses approximately 31.5 GiB for Base and
      30.7 GiB for Edit at 1024x1024. It uses more memory because only eligible
      DiT linear layers are quantized online, while the MLLM from the regular
      Base/Edit checkpoint remains in BF16. Most of the observed difference
      comes from the MLLM precision, although differences in the two DiT FP8
      representations and kernels may also contribute.
- **Quantization scope:** The transformer-scoped configuration applies to the
  DiT. With the serialized `-fp8` checkpoints, the MLLM follows its own
  checkpoint-declared FP8 configuration. With the regular checkpoints used for
  online FP8, the MLLM remains in BF16. The VAE and scheduler are outside this
  quantization routing.
    "model": "Boogu/Boogu-Image-0.1-Turbo",
    "prompt": "A mountain lake at sunset, photorealistic, cinematic lighting",
    "size": "1024x1024",
    "seed": 42
  }' | jq -r '.data[0].b64_json' | base64 -d > output-turbo.png
```

### Notes

- **Required settings:** `guidance_scale=1.0` and `guidance_scale_2=1.0`; any
  other value is rejected. `num_inference_steps` defaults to 4. All three are
  pipeline defaults, so the request above omits them.
- **Resolution:** upstream constrains Turbo to 1K, so `1024x1024` is the
  supported working resolution. The shared pipeline still clamps at 2K; larger
  sizes are not validated for this checkpoint.
- **Known limitations:** the same single-GPU limitations as Base apply; CPU
  offload, Cache-DiT, and multi-GPU parallelism are not yet validated.

## Image editing (Boogu-Image-0.1-Edit)

The Edit checkpoint is served by the same native pipeline (`BooguImagePipeline`);
the image-editing (TI2I) path activates automatically when a request carries a
reference image. The Base text-to-image path is unaffected (no reference image
is sent).

### Command

```bash
vllm serve Boogu/Boogu-Image-0.1-Edit --omni --port 8091
```

For text-only or image-only guidance, use two CFG ranks:

```bash
vllm serve Boogu/Boogu-Image-0.1-Edit \
  --omni \
  --cfg-parallel-size 2 \
  --port 8091
```

Double guidance has three predictions per denoising step. It supports either
two ranks (round-robin assignment: rank 0 runs branches 0 and 2) or three ranks
(one branch per rank):

```bash
# Practical two-GPU configuration
vllm serve Boogu/Boogu-Image-0.1-Edit \
  --omni \
  --cfg-parallel-size 2 \
  --port 8091

# Maximum branch parallelism
vllm serve Boogu/Boogu-Image-0.1-Edit \
  --omni \
  --cfg-parallel-size 3 \
  --port 8091
```

### Verification

Edit an image with `/v1/images/edits` (the model-card example — change a photo
to a colored-pencil drawing). Diffusion parameters are plain multipart form
fields; add `guidance_scale_2` to enable image guidance (double-guidance path):

```bash
curl -s http://localhost:8091/v1/images/edits \
  -F model="Boogu/Boogu-Image-0.1-Edit" \
  -F image="@input.png" \
  -F prompt="Change the style to a colored pencil drawing." \
  -F num_inference_steps=28 \
  -F guidance_scale=5.0 \
  -F guidance_scale_2=2.0 \
  -F seed=42 \
  | jq -r '.data[0].b64_json' | base64 -d > edited.png
```

Or via chat completions (attach the image as a data URL; parameters go in
`extra_body`):

```bash
curl -s http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64>"}},
        {"type": "text", "text": "Change the style to a colored pencil drawing."}
      ]}
    ],
    "extra_body": {
      "num_inference_steps": 28,
      "guidance_scale": 5.0,
      "guidance_scale_2": 1.0,
      "seed": 42
    }
  }' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > edited.png
```

### Notes

- **Single reference image:** only one input image is supported for now (the
  upstream "Only support 1 reference image for now" limit).
- **Guidance semantics:**
    - `guidance_scale` = text guidance (upstream `text_guidance_scale`, default
    `4.0`); `> 1.0` enables text CFG. Editing typically uses `5.0`.
    - `guidance_scale_2` = image guidance (upstream `image_guidance_scale`,
    default `1.0` = off). Setting it `> 1.0` enables the double-guidance path
    (3 model predictions per step), steering more strongly toward the reference
    image.
- **CFG-size recommendations:**

  | Request mode | Scales | Predictions per step | Recommended CFG size |
  | --- | --- | ---: | ---: |
  | Base T2I text guidance | `guidance_scale > 1`, no reference | 2 | 2 |
  | Edit text-only guidance | `guidance_scale > 1`, `guidance_scale_2 = 1` | 2 | 2 |
  | Edit image-only guidance | `guidance_scale = 1`, `guidance_scale_2 > 1` | 2 | 2 |
  | Edit double guidance | both scales `> 1` | 3 | 2 (practical) or 3 (maximum parallelism) |
  | CFG off | both scales `= 1` | 1 | Reuse a size-2 server safely; no speedup expected |

  Double guidance preserves Boogu's original combination:

  ```text
  pred = positive_with_reference
       + (text_scale - 1) * (positive_with_reference - negative_with_reference)
       + (image_scale - 1) * (negative_with_reference - uncond)
  ```
- **Output resolution:** the output size follows the reference image (upstream
  `align_res`, on by default for a single reference), so `height`/`width` are
  derived from the input and requested sizes are not applied for edits.
- **Same limitations** as the Base checkpoint apply except that CFG parallelism
  is supported as described above; CPU offload, Cache-DiT, TP, SP, and HSDP are
  still unsupported.
- **Offline editing:** the shared offline example
  [`examples/offline_inference/image_to_image/image_edit.py`](../../examples/offline_inference/image_to_image/image_to_image.md)
  supports Boogu-Image-Edit directly
  (`--model Boogu/Boogu-Image-0.1-Edit --guidance-scale 5.0`, optional
  `--guidance-scale-2`).

## Fast image editing (Boogu-Image-0.1-Edit-Turbo)

Edit-Turbo is the distilled image-editing checkpoint. Its `model_index.json`
declares `BooguImagePipeline`, so `--model-class-name BooguImageTurboPipeline`
is required to reach the four-step DMD path; without it the checkpoint is
served on the dense Edit path. The upstream 1K hotfix is recommended for
stable results and is pinned below to avoid silently loading the older
checkpoint from the repository's default revision.

### Command

```bash
vllm serve Boogu/Boogu-Image-0.1-Edit-Turbo \
  --omni \
  --model-class-name BooguImageTurboPipeline \
  --revision hotfix-1k-20260708 \
  --port 8091
```

### Verification

Edit an image with the distilled four-step settings:

```bash
curl -s http://localhost:8091/v1/images/edits \
  -F model="Boogu/Boogu-Image-0.1-Edit-Turbo" \
  -F image="@input.png" \
  -F prompt="Change the style to a colored pencil drawing." \
  -F num_inference_steps=4 \
  -F guidance_scale=1.0 \
  -F seed=42 \
  | jq -r '.data[0].b64_json' | base64 -d > edited-turbo.png
```

Or via chat completions:

```bash
curl -s http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64>"}},
        {"type": "text", "text": "Change the style to a colored pencil drawing."}
      ]}
    ],
    "extra_body": {
      "num_inference_steps": 4,
      "guidance_scale": 1.0,
      "seed": 42
    }
  }' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > edited-turbo.png
```

### Notes

- **Pinned revision:** use `hotfix-1k-20260708`; upstream recommends the 1K
  hotfix over the 1.5K variant for more stable results.
- **Required settings:** `guidance_scale=1.0` and `guidance_scale_2=1.0`; the
  Turbo pipeline rejects any other value, so the `guidance_scale_2=2.0` shown
  for the regular Edit checkpoint does not apply here. `num_inference_steps`
  defaults to 4.
- **Reference images:** the same single-reference and `align_res` behavior as
  the regular Edit checkpoint applies.
- **Known limitations:** the same single-GPU limitations as Base and Edit apply;
  CPU offload, Cache-DiT, and multi-GPU parallelism are not yet validated.

## Output format and compression

Image requests accept `output_format` (`png`, `jpeg`, or `webp`) and
`output_compression` (0-100). The default is `png` with the fastest,
least-compressed PNG encode, which is lossless but produces multi-megabyte
payloads. For latency-sensitive clients, `output_format: "jpeg"` removes
almost the entire response-encoding overhead: on a 1024x1024 Turbo request
the client-observed e2e latency drops from ~900 ms to ~766 ms and the
payload shrinks from ~3.1 MB to ~0.8 MB. Use `output_compression` to trade
encode time against payload size when PNG must stay lossless: lower values
(e.g. 1) compress harder and produce smaller payloads at proportionally
longer encode times.

## Attention backend

The diffusion attention backend defaults to `FLASH_ATTN` (FlashAttention-2).
On Hopper-class GPUs (SM90) pass `--diffusion-attention-backend CUDNN_ATTN`
for a measured ~12% engine-time reduction on Boogu-Image workloads:

```bash
vllm serve Boogu/Boogu-Image-0.1-Turbo \
  --omni \
  --diffusion-attention-backend CUDNN_ATTN \
  --port 8091
```

Measured on a 143 GB SM90 card (1024x1024, 4 steps, 5-run mean): stage time
673 ms with `CUDNN_ATTN` vs 763 ms with the default. Outputs are numerically
different from FLASH_ATTN (different attention kernel) but visually
equivalent, deterministic per seed, and decode identically. On datacenter
Blackwell (SM100 / SM103) use `--diffusion-attention-backend TRTLLM_ATTN`
instead; it is rejected on SM90.

## Performance validation

The checked-in single-device A40 measurements remain the current Boogu
baseline: Base T2I is about 16.37 s request latency at concurrency 1 and Edit
text-only guidance is about 29.64 s for 512x512, 28-step requests, with roughly
36.8 GB peak memory. CFG-parallel A/B numbers must be collected on the same
hardware and software revision before reporting a speedup; do not compare a
multi-GPU result against these A40 numbers as if it were a controlled A/B.

Run the dedicated perf rows in
`tests/dfx/perf/tests/test_boogu_image_vllm_omni.json` and
`tests/dfx/perf/tests/test_boogu_image_edit_vllm_omni.json` on 2x/3x H100, then
record latency, throughput, and per-GPU peak memory beside the single-device
run. The expected shape is close to the general CFG-parallel range (roughly
1.5x-1.8x for balanced two-branch workloads); the three-branch Edit result will
depend on whether two-rank round-robin or full three-rank execution is used and
must be reported from measurement rather than assumed.