vllm-omni/recipes/Boogu/Boogu-Image.md

Boogu-Image

Text-to-image and image-editing online serving (Boogu-Image-0.1-Base / -Turbo / -Edit / -Edit-Turbo)

Summary

• Vendor: Boogu
• Model: Boogu/Boogu-Image-0.1-Base (text-to-image), Boogu/Boogu-Image-0.1-Turbo (four-step text-to-image), Boogu/Boogu-Image-0.1-Edit (image editing), and Boogu/Boogu-Image-0.1-Edit-Turbo (four-step image editing)
• Task: Text-to-image generation and text-guided image editing (TI2I)
• Mode: Online serving with the OpenAI-compatible API
• Maintainer: Community

When to use this recipe

Use this recipe when you want a known-good starting point for serving Boogu/Boogu-Image-0.1-Base, Boogu/Boogu-Image-0.1-Turbo, Boogu/Boogu-Image-0.1-Edit, or Boogu/Boogu-Image-0.1-Edit-Turbo with vLLM-Omni's native pipeline (no --diffusion-load-format diffusers, and the upstream boogu package is not required).

Boogu-Image-0.1 is an Apache-2.0 unified image generation and editing model family. The Base text-to-image checkpoint pairs a Qwen3-VL multimodal encoder with a Diffusion Transformer (DiT) and a flow-match Euler scheduler with time-shift. It handles photorealistic generation and Chinese/English text rendering. The Edit checkpoint uses the native BooguImagePipeline, while the distilled Turbo and Edit-Turbo checkpoints run the four-step DMD path through BooguImageTurboPipeline.

References

• Upstream model card: https://huggingface.co/Boogu/Boogu-Image-0.1-Base
• Turbo model card: https://huggingface.co/Boogu/Boogu-Image-0.1-Turbo
• Edit model card: https://huggingface.co/Boogu/Boogu-Image-0.1-Edit
• Edit-Turbo 1K hotfix: https://huggingface.co/Boogu/Boogu-Image-0.1-Edit-Turbo/tree/hotfix-1k-20260708
• Project page: https://boogu.org
• GitHub: https://github.com/boogu-project/Boogu-Image
• Related example: examples/online_serving/text_to_image/

Hardware Support

This recipe documents supported configurations for CUDA GPU serving. The native pipeline supports single-GPU inference and classifier-free-guidance (CFG) parallelism. CPU offload, cache acceleration, and other multi-GPU dimensions remain unsupported (see Notes).

GPU

1 x A100/H100 (Single GPU, 40GB+ VRAM)

The model footprint is roughly 34.6 GiB on GPU, so a 40GB+ card is recommended.

Command

vllm serve Boogu/Boogu-Image-0.1-Base --omni --port 8091

!!! note If you hit Out-of-Memory (OOM) on a smaller card, enable VAE slicing and tiling to reduce peak memory: --vae-use-slicing --vae-use-tiling.

Verification

After the server is ready, test with a simple request:

curl -X POST http://localhost:8091/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Boogu/Boogu-Image-0.1-Base",
    "prompt": "A mountain lake at sunset, photorealistic, cinematic lighting",
    "size": "1024x1024",
    "num_inference_steps": 28,
    "guidance_scale": 4.0,
    "seed": 42
  }' | jq -r '.data[0].b64_json' | base64 -d > output.png

Or via the chat-completions endpoint (parameters go in extra_body):

curl -s http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": "A mountain lake at sunset, photorealistic, cinematic lighting"}
    ],
    "extra_body": {
      "height": 1024,
      "width": 1024,
      "num_inference_steps": 28,
      "guidance_scale": 4.0,
      "seed": 42
    }
  }' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png

Notes

• Memory usage: ~34.6 GiB on GPU. Use --vae-use-slicing --vae-use-tiling to trim peak VRAM if needed.
• Key flags:
  • --omni — enables vLLM-Omni diffusion serving.
• Guidance: Boogu-Image uses guidance_scale (mapped to the upstream text_guidance_scale); the default is 4.0. Classifier-free guidance is active whenever guidance_scale > 1.0.
• Recommended settings: num_inference_steps=28-50, guidance_scale=4.0. The model's maximum native resolution is 2K.
• Known limitations (not yet supported): CPU offload (--enable-cpu-offload / --enable-layerwise-offload), Cache-DiT (--cache-backend cache_dit), and TP / SP / HSDP multi-GPU parallelism.

2 x H100 (CFG parallel, Base T2I)

Base T2I has two predictions per guided denoising step, so two CFG ranks map one-to-one to the positive and negative-text branches:

vllm serve Boogu/Boogu-Image-0.1-Base \
  --omni \
  --cfg-parallel-size 2 \
  --port 8091

Use guidance_scale > 1.0 to activate the parallel two-branch path. Requests with guidance_scale=1.0 remain valid on the same server: every rank evaluates only the positive branch, no negative embeddings are built, and no guidance is applied.

1 x RTX 4090 24 GB / 48 GB (FP8)

Boogu-Image supports the following FP8 loading paths:

Path	Checkpoints	MLLM	DiT
Pre-quantized FP8	Boogu/Boogu-Image-0.1-{Base,Edit}-fp8	HF checkpoint FP8 configuration	TorchAO FP8 weight-only (W8A16)
Online FP8	Boogu/Boogu-Image-0.1-{Base,Edit}	Optional 128x128 block FP8: Triton W8A8 (vLLM 0.28.0); Marlin W8A16 (vLLM 0.29.0)	Native dynamic FP8 (CUTLASS W8A8)

Environment

The following isolated environments were used for Base T2I and Edit I2I measurements at 512x512:

• OS: Linux
• Python: 3.12
• Driver / runtime: NVIDIA CUDA environment with one RTX 4090 48 GB GPU
• vLLM version: Match the vLLM-Omni checkout used for deployment
• vLLM-Omni version or commit: Use a commit that contains TorchAO FP8 checkpoint loading for diffusion models
• TorchAO version: 0.17.0
• kernels / kernels-data: 0.16.1 / 0.16.1
• Transformers: 5.17.0
• Tokenizers: 0.23.1

Command

Serve the official Base FP8 checkpoint using its pre-quantized TorchAO FP8 weight-only transformer weights:

vllm serve Boogu/Boogu-Image-0.1-Base-fp8 \
  --omni \
  --port 8091 \
  --diffusion-quantization-config \
  '{"transformer":{"method":"torchao_float8_weight_only"}}'

Native online FP8:

# DiT only
vllm serve Boogu/Boogu-Image-0.1-Base \
  --omni \
  --port 8091 \
  --diffusion-quantization-config \
  '{"mllm":null,"transformer":"fp8","vae":null}'

# MLLM + DiT
vllm serve Boogu/Boogu-Image-0.1-Base \
  --omni \
  --port 8091 \
  --quantization fp8

# MLLM + DiT (using --diffusion-quantization-config)
vllm serve Boogu/Boogu-Image-0.1-Base \
  --omni \
  --port 8091 \
  --diffusion-quantization-config \
  '{"mllm":"fp8","transformer":"fp8","vae":null}'

For image editing, replace Base with Edit and keep the corresponding quantization configuration.

Notes

• Memory usage:

  • The serialized TorchAO FP8 path uses approximately 21.1 GiB at 512x512 and 24.2-24.6 GiB at 1024x1024.
  • The DiT-only online FP8 path uses approximately 26.9 GiB at 512x512 and 29.9-30.9 GiB at 1024x1024.
  • The MLLM + DiT online FP8 path uses approximately 20.5-20.6 GiB at 512x512 and 23.6-24.5 GiB at 1024x1024.

• Quantization scope: For original Base/Edit checkpoints, --quantization fp8 quantizes the MLLM's language-model linear layers through per-block FP8 and the DiT's linear layers through native online FP8. The vision encoder, embeddings, normalization layers and VAE remain unquantized.

• Performance: In the 48 GB Base/Edit benchmarks at 512x512, both DiT-only and MLLM + DiT online FP8 reduced E2E latency by over 10 seconds compared with official FP8 and vLLM-Omni pre-quantized FP8. The main runtime difference lies in the DiT quantization implementation, which switches from TorchAO weight-only FP8 (W8A16) to native CUTLASS W8A8 kernels. This reduces additional dequantization overhead and enables FP8 Tensor Core acceleration.

• Known Limitation:

  • Output consistency: With MLLM + DiT online FP8, same-seed outputs may visibly differ from BF16 for some Base prompts, but these differences do not mean lower image quality.

  • Import compatibility conflict between TorchAO and Diffusers:: Loading pre-quantized checkpoints requires torchao>=0.17.0, which has a compatibility conflict with diffusers==0.40.0. This produces a warning when Diffusers is imported, but currently does not affect Boogu-Image weight loading.

  • Conflict between the default kernels and Transformers versions: The default Transformers version range (>=5.10.1,<5.15) conflicts with kernels==0.16.1 when loading pre-quantized FP8 MLLM weights through Hugging Face FP8Linear. When using this loading path with kernels==0.16.1, please upgrade Transformers to 5.17.0.

Verification

The request format is shared by both FP8 paths. Set <MODEL_ID> to the model ID passed to vllm serve. For example, use Boogu/Boogu-Image-0.1-Base-fp8 for serialized TorchAO FP8 or Boogu/Boogu-Image-0.1-Base for native online FP8. For image editing, use the corresponding Edit model.

Base text-to-image:

curl -X POST http://localhost:8091/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<MODEL_ID>",
    "prompt": "A mountain lake at sunset, photorealistic, cinematic lighting",
    "size": "1024x1024",
    "num_inference_steps": 28,
    "guidance_scale": 4.0,
    "seed": 42
  }' | jq -r '.data[0].b64_json' | base64 -d > output_fp8.png

Image editing:

curl -X POST http://localhost:8091/v1/images/edits \
  -F model="<MODEL_ID>" \
  -F image="@input.png" \
  -F prompt="Change the style to a colored pencil drawing." \
  -F num_inference_steps=28 \
  -F guidance_scale=4.0 \
  -F guidance_scale_2=1.0 \
  -F seed=42 \
  | jq -r '.data[0].b64_json' | base64 -d > edited_fp8.png

Fast text-to-image (Boogu-Image-0.1-Turbo)

Turbo is the distilled text-to-image checkpoint. Its model_index.json declares BooguImageTurboPipeline, which runs the four-step DMD path instead of the scheduler-driven path used by Base.

Command

vllm serve Boogu/Boogu-Image-0.1-Turbo --omni --port 8091

Verification

curl -X POST http://localhost:8091/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Boogu/Boogu-Image-0.1-Turbo",
    "prompt": "A mountain lake at sunset, photorealistic, cinematic lighting",
    "size": "1024x1024",
    "seed": 42
  }' | jq -r '.data[0].b64_json' | base64 -d > output-turbo.png

### Notes

- **Required settings:** `guidance_scale=1.0` and `guidance_scale_2=1.0`; any
  other value is rejected. `num_inference_steps` defaults to 4. All three are
  pipeline defaults, so the request above omits them.
- **Resolution:** upstream constrains Turbo to 1K, so `1024x1024` is the
  supported working resolution. The shared pipeline still clamps at 2K; larger
  sizes are not validated for this checkpoint.
- **Known limitations:** the same single-GPU limitations as Base apply; CPU
  offload, Cache-DiT, and multi-GPU parallelism are not yet validated.

## Image editing (Boogu-Image-0.1-Edit)

The Edit checkpoint is served by the same native pipeline (`BooguImagePipeline`);
the image-editing (TI2I) path activates automatically when a request carries a
reference image. The Base text-to-image path is unaffected (no reference image
is sent).

### Command

```bash
vllm serve Boogu/Boogu-Image-0.1-Edit --omni --port 8091

For text-only or image-only guidance, use two CFG ranks:

vllm serve Boogu/Boogu-Image-0.1-Edit \
  --omni \
  --cfg-parallel-size 2 \
  --port 8091

Double guidance has three predictions per denoising step. It supports either two ranks (round-robin assignment: rank 0 runs branches 0 and 2) or three ranks (one branch per rank):

# Practical two-GPU configuration
vllm serve Boogu/Boogu-Image-0.1-Edit \
  --omni \
  --cfg-parallel-size 2 \
  --port 8091

# Maximum branch parallelism
vllm serve Boogu/Boogu-Image-0.1-Edit \
  --omni \
  --cfg-parallel-size 3 \
  --port 8091

Verification

Edit an image with /v1/images/edits (the model-card example — change a photo to a colored-pencil drawing). Diffusion parameters are plain multipart form fields; add guidance_scale_2 to enable image guidance (double-guidance path):

curl -s http://localhost:8091/v1/images/edits \
  -F model="Boogu/Boogu-Image-0.1-Edit" \
  -F image="@input.png" \
  -F prompt="Change the style to a colored pencil drawing." \
  -F num_inference_steps=28 \
  -F guidance_scale=5.0 \
  -F guidance_scale_2=2.0 \
  -F seed=42 \
  | jq -r '.data[0].b64_json' | base64 -d > edited.png

Or via chat completions (attach the image as a data URL; parameters go in extra_body):

curl -s http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64>"}},
        {"type": "text", "text": "Change the style to a colored pencil drawing."}
      ]}
    ],
    "extra_body": {
      "num_inference_steps": 28,
      "guidance_scale": 5.0,
      "guidance_scale_2": 1.0,
      "seed": 42
    }
  }' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > edited.png

Notes

• Single reference image: only one input image is supported for now (the upstream "Only support 1 reference image for now" limit).

• Guidance semantics:

  • guidance_scale = text guidance (upstream text_guidance_scale, default 4.0); > 1.0 enables text CFG. Editing typically uses 5.0.
  • guidance_scale_2 = image guidance (upstream image_guidance_scale, default 1.0 = off). Setting it > 1.0 enables the double-guidance path (3 model predictions per step), steering more strongly toward the reference image.

• CFG-size recommendations:

  Request mode	Scales	Predictions per step	Recommended CFG size
  Base T2I text guidance	guidance_scale > 1, no reference	2	2
  Edit text-only guidance	guidance_scale > 1, guidance_scale_2 = 1	2	2
  Edit image-only guidance	guidance_scale = 1, guidance_scale_2 > 1	2	2
  Edit double guidance	both scales > 1	3	2 (practical) or 3 (maximum parallelism)
  CFG off	both scales = 1	1	Reuse a size-2 server safely; no speedup expected

  Double guidance preserves Boogu's original combination:

  pred = positive_with_reference
       + (text_scale - 1) * (positive_with_reference - negative_with_reference)
       + (image_scale - 1) * (negative_with_reference - uncond)
  

• Output resolution: the output size follows the reference image (upstream align_res, on by default for a single reference), so height/width are derived from the input and requested sizes are not applied for edits.

• Same limitations as the Base checkpoint apply except that CFG parallelism is supported as described above; CPU offload, Cache-DiT, TP, SP, and HSDP are still unsupported.

• Offline editing: the shared offline example examples/offline_inference/image_to_image/image_edit.py supports Boogu-Image-Edit directly (--model Boogu/Boogu-Image-0.1-Edit --guidance-scale 5.0, optional --guidance-scale-2).

• Request batching (--max-num-seqs > 1) is not bit-exact vs. serial. Compatible TI2I requests (same task type, same guidance_scale_2 explicitness) can share one forward pass. The instruction encoder batches with padding="longest", so a request's padded batch shape depends on whichever partner request(s) happen to co-arrive, even though production RoPE derives each row's own effective position count from that row's own attention_mask.sum() rather than from the batch's padded width. In the measurements below, --max-num-seqs 1 and --max-num-seqs > 1 do not produce pixel-identical output for the same seed; the exact mechanism (padding-shape-dependent kernel/reduction-order numerics vs. some other batch-shape effect) has not been isolated, so treat this as an observed drift under the specific configs below, not a diagnosed root cause, and do not extrapolate the numbers below as a general upper bound on other configs. Two distinct regimes have been measured so far, and they are not interchangeable tolerances:

  • Fixed-shape co-batch (partner instructions encode to the same length): Boogu-Image-0.1-Edit, 512x512, 20 steps, guidance 5/2, one fixed reference image and instruction pair, seeds 42-73, B=2 vs. serial -- mean pixel MAE 1.70/255, max per-image MAE 2.68/255, min PSNR 34.35 dB (2/32 image pairs were pixel-exact).
  • Heterogeneous partner length (co-batched instructions encode to different lengths): MAD 2.6-6.0 in the fixed-length partner-mixing check on this PR's test head -- larger than the fixed-shape case above in that check. A targeted current-head A/B run observed a comparable range: Boogu-Image-0.1-Edit, 512x512, 20 steps, guidance 5/2, one fixed A request (5-token instruction) co-batched in turn with a 2-token and a 49-token partner instruction (both also differing in reference image and content, not length alone; max_num_seqs=2, co-batching verified via request tracing, not assumed) -- solo vs. A+2-token partner MAD 4.06/255 (PSNR 29.9 dB), solo vs. A+49-token partner MAD 3.87/255 (PSNR 29.7 dB), 2-token vs. 49-token partner MAD 4.05/255 (PSNR 30.2 dB). A controlled ablation (same partner content/reference, varying only instruction length) would be needed to isolate a pure length effect, and has not been run. Pin --max-num-seqs 1 for any workload that requires strict serial-reproducibility (e.g. golden-image regression tests); use --max-num-seqs > 1 only when this drift is acceptable for the throughput gain.

Fast image editing (Boogu-Image-0.1-Edit-Turbo)

Edit-Turbo is the distilled image-editing checkpoint. Its model_index.json declares BooguImagePipeline, so --model-class-name BooguImageTurboPipeline is required to reach the four-step DMD path; without it the checkpoint is served on the dense Edit path. The upstream 1K hotfix is recommended for stable results and is pinned below to avoid silently loading the older checkpoint from the repository's default revision.

Command

vllm serve Boogu/Boogu-Image-0.1-Edit-Turbo \
  --omni \
  --model-class-name BooguImageTurboPipeline \
  --revision hotfix-1k-20260708 \
  --port 8091

Verification

Edit an image with the distilled four-step settings:

curl -s http://localhost:8091/v1/images/edits \
  -F model="Boogu/Boogu-Image-0.1-Edit-Turbo" \
  -F image="@input.png" \
  -F prompt="Change the style to a colored pencil drawing." \
  -F num_inference_steps=4 \
  -F guidance_scale=1.0 \
  -F seed=42 \
  | jq -r '.data[0].b64_json' | base64 -d > edited-turbo.png

Or via chat completions:

curl -s http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "user", "content": [
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64>"}},
        {"type": "text", "text": "Change the style to a colored pencil drawing."}
      ]}
    ],
    "extra_body": {
      "num_inference_steps": 4,
      "guidance_scale": 1.0,
      "seed": 42
    }
  }' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > edited-turbo.png

Notes

• Pinned revision: use hotfix-1k-20260708; upstream recommends the 1K hotfix over the 1.5K variant for more stable results.
• Required settings: guidance_scale=1.0 and guidance_scale_2=1.0; the Turbo pipeline rejects any other value, so the guidance_scale_2=2.0 shown for the regular Edit checkpoint does not apply here. num_inference_steps defaults to 4.
• Reference images: the same single-reference and align_res behavior as the regular Edit checkpoint applies.
• Known limitations: the same single-GPU limitations as Base and Edit apply; CPU offload, Cache-DiT, and multi-GPU parallelism are not yet validated.

Output format and compression

Image requests accept output_format (png, jpeg, or webp) and output_compression (0-100). The default is png with the fastest, least-compressed PNG encode, which is lossless but produces multi-megabyte payloads. For latency-sensitive clients, output_format: "jpeg" removes almost the entire response-encoding overhead: on a 1024x1024 Turbo request the client-observed e2e latency drops from ~900 ms to ~766 ms and the payload shrinks from ~3.1 MB to ~0.8 MB. Use output_compression to trade encode time against payload size when PNG must stay lossless: lower values (e.g. 1) compress harder and produce smaller payloads at proportionally longer encode times.

Attention backend

The diffusion attention backend defaults to FLASH_ATTN (FlashAttention-2). On Hopper-class GPUs (SM90) pass --diffusion-attention-backend CUDNN_ATTN for a measured ~12% engine-time reduction on Boogu-Image workloads:

vllm serve Boogu/Boogu-Image-0.1-Turbo \
  --omni \
  --diffusion-attention-backend CUDNN_ATTN \
  --port 8091

Measured on a 143 GB SM90 card (1024x1024, 4 steps, 5-run mean): stage time 673 ms with CUDNN_ATTN vs 763 ms with the default. Outputs are numerically different from FLASH_ATTN (different attention kernel) but visually equivalent, deterministic per seed, and decode identically. On datacenter Blackwell (SM100 / SM103) use --diffusion-attention-backend TRTLLM_ATTN instead; it is rejected on SM90.

Performance validation

The checked-in single-device A40 measurements remain the current Boogu baseline: Base T2I is about 16.37 s request latency at concurrency 1 and Edit text-only guidance is about 29.64 s for 512x512, 28-step requests, with roughly 36.8 GB peak memory. CFG-parallel A/B numbers must be collected on the same hardware and software revision before reporting a speedup; do not compare a multi-GPU result against these A40 numbers as if it were a controlled A/B.

Run the dedicated perf rows in tests/dfx/perf/tests/test_boogu_image_vllm_omni.json and tests/dfx/perf/tests/test_boogu_image_edit_vllm_omni.json on 2x/3x H100, then record latency, throughput, and per-GPU peak memory beside the single-device run. The expected shape is close to the general CFG-parallel range (roughly 1.5x-1.8x for balanced two-branch workloads); the three-branch Edit result will depend on whether two-rank round-robin or full three-rank execution is used and must be reported from measurement rather than assumed.