vllm-omni/recipes/Bagel/BAGEL-7B-MoT.md

BAGEL-7B-MoT

BAGEL image generation through the shared online and offline image examples

Summary

• Vendor: ByteDance Seed
• Model: ByteDance-Seed/BAGEL-7B-MoT
• Task: Text-to-image, image-to-image, text-to-text, and image-to-text
• Mode: Offline inference and OpenAI-compatible online serving
• Maintainer: Community

When to use this recipe

Use this recipe when you want to run BAGEL through the shared image examples instead of model-specific example clients. The generic examples can format BAGEL text-to-image and image-to-image prompts, select the image output modality, attach reference images, and forward BAGEL-specific generation parameters through the pipeline-declared extra_args contract.

References

• Upstream model: ByteDance-Seed/BAGEL-7B-MoT
• Related offline example: examples/offline_inference/text_to_image/text_to_image.py
• Related offline image-to-image example: examples/offline_inference/image_to_image/image_edit.py
• Related offline understanding example: examples/offline_inference/x_to_text/x_to_text.py
• Related online example: examples/online_serving/text_to_image/openai_chat_client.py
• Related online image-to-image example: examples/online_serving/image_to_image/openai_chat_client.py
• Default deploy configs: vllm_omni/deploy/bagel.yaml, vllm_omni/deploy/bagel_single_stage.yaml

Hardware Support

This recipe documents the CUDA layouts used by the in-repo BAGEL deploy configs. The default two-stage config shares one 80 GB GPU; for more headroom, move the diffusion stage to a second GPU in a custom deploy config.

GPU

1x A100 80GB

Environment

• OS: Linux
• Python: Match the repository requirements for your checkout
• Driver / runtime: NVIDIA CUDA environment with one A100 80 GB GPU
• vLLM version: Match the repository requirements for your checkout
• vLLM-Omni version or commit: Use the commit you are deploying from

Offline Commands

Run text-to-image with the shared offline example from the repository root. Pick the topology with --deploy-config and forward BAGEL-specific generation parameters as a JSON object through --extra-body:

# Two-stage (Thinker + DiT), shares one GPU by default
python examples/offline_inference/text_to_image/text_to_image.py \
  --model ByteDance-Seed/BAGEL-7B-MoT \
  --deploy-config vllm_omni/deploy/bagel.yaml \
  --prompt "A beautiful sunset over mountains" \
  --height 1024 \
  --width 1024 \
  --num-inference-steps 50 \
  --negative-prompt "blurry, low quality" \
  --seed 42 \
  --extra-body '{"timestep_shift": 3.0, "cfg_text_scale": 4.0, "cfg_img_scale": 1.5, "cfg_interval": [0.4, 1.0], "cfg_renorm_type": "global", "cfg_renorm_min": 0.0, "think": false}' \
  --output /tmp/bagel_text2img.png

# Single-stage (DiT only, with internal LLM/ViT/VAE)
python examples/offline_inference/text_to_image/text_to_image.py \
  --model ByteDance-Seed/BAGEL-7B-MoT \
  --deploy-config vllm_omni/deploy/bagel_single_stage.yaml \
  --prompt "A beautiful sunset over mountains" \
  --height 1024 \
  --width 1024 \
  --num-inference-steps 50 \
  --seed 42 \
  --extra-body '{"timestep_shift": 3.0, "cfg_text_scale": 4.0, "cfg_interval": [0.4, 1.0], "cfg_renorm_type": "global", "cfg_renorm_min": 0.0}' \
  --output /tmp/bagel_text2img_single.png

The --extra-body JSON forwards BAGEL-specific parameters into OmniDiffusionSamplingParams.extra_args. Keys are filtered against the model's declared extra_body_params (see vllm_omni/model_extras/bagel.py), so unknown keys for BAGEL are silently dropped. This is the preferred way to pass model-specific params — timestep_shift (use 3.0 for 1024×1024; the deprecated --timesteps-shift flag defaults to 1.0 and is NextStep-oriented), cfg_text_scale, cfg_img_scale, cfg_interval, cfg_renorm_type, cfg_renorm_min, and think. --seed makes generation reproducible.

Note

: the two-stage bagel.yaml runs both stages on GPU 0. If you hit CUDA OOM during warmup, lower stage-0 gpu_memory_utilization in the YAML or split the stages across two GPUs (see the 2x CUDA GPUs section).

Run image-to-image with the shared offline image-to-image example:

python examples/offline_inference/image_to_image/image_edit.py \
  --model ByteDance-Seed/BAGEL-7B-MoT \
  --prompt "Make the scene look like a watercolor painting" \
  --image /tmp/bagel_text2img.png \
  --height 512 \
  --width 512 \
  --extra-args '{"cfg_text_scale": 4.0, "cfg_img_scale": 1.5}' \
  --output /tmp/bagel_img2img.png

The --extra-args JSON forwards BAGEL-specific parameters (e.g. cfg_text_scale, cfg_img_scale, cfg_interval, cfg_renorm_type) into OmniDiffusionSamplingParams.extra_args via the model-extras registry.

Run text-to-text with the shared understanding example. BAGEL's default bagel.yaml deploy config is discovered from the checkpoint, so no model-specific example or explicit deploy config is required:

python examples/offline_inference/x_to_text/x_to_text.py \
  --model ByteDance-Seed/BAGEL-7B-MoT \
  --prompt "Where is the capital of France?"

Add --image to run image-to-text with the same example:

python examples/offline_inference/x_to_text/x_to_text.py \
  --model ByteDance-Seed/BAGEL-7B-MoT \
  --image /path/to/input.jpg \
  --prompt "Please describe this image in detail."

Online Commands

Start the OpenAI-compatible server:

vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni --port 8091

To use the single-stage topology online:

vllm serve ByteDance-Seed/BAGEL-7B-MoT \
  --omni \
  --port 8091 \
  --deploy-config vllm_omni/deploy/bagel_single_stage.yaml

Send a text-to-image request with BAGEL-specific generation parameters:

curl http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "<|im_start|>A beautiful sunset over mountains<|im_end|>"}
        ]
      }
    ],
    "modalities": ["image"],
    "extra_body": {
      "height": 512,
      "width": 512,
      "num_inference_steps": 50,
      "cfg_text_scale": 4.0,
      "cfg_img_scale": 1.5,
      "negative_prompt": "blurry, low quality",
      "seed": 42
    }
  }'

The important part is that BAGEL-specific keys such as cfg_text_scale, cfg_img_scale, cfg_interval, cfg_renorm_type, and cfg_renorm_min belong in extra_body. The serving layer routes the declared keys into OmniDiffusionSamplingParams.extra_args for the BAGEL pipeline.

Verification

Decode the returned data URL into an image:

curl http://localhost:8091/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "<|im_start|>A ceramic teapot on a wooden table<|im_end|>"}
        ]
      }
    ],
    "modalities": ["image"],
    "extra_body": {
      "height": 512,
      "width": 512,
      "num_inference_steps": 25,
      "cfg_text_scale": 4.0,
      "seed": 42
    }
  }' | jq -r '.choices[]?.message.content[]? | select(.image_url.url) | .image_url.url' \
    | head -n 1 \
    | cut -d',' -f2- \
    | base64 -d > /tmp/bagel_online.png

ls -lh /tmp/bagel_online.png

2x CUDA GPUs

Create a custom deploy config from vllm_omni/deploy/bagel.yaml and move the diffusion stage to GPU 1:

stages:
  - stage_id: 0
    devices: "0"
    # keep the remaining stage-0 settings from bagel.yaml
  - stage_id: 1
    devices: "1"
    # keep the remaining stage-1 settings from bagel.yaml

Then start serving with that config:

vllm serve ByteDance-Seed/BAGEL-7B-MoT \
  --omni \
  --port 8091 \
  --deploy-config /path/to/custom_bagel_2gpu.yaml

Use the online curl request from the 1x A100 80GB section to verify that the server returns an image.