[Diffusion] Add Boogu Edit-Turbo support docs and tests (#6701)
Signed-off-by: xRay2016 <1150722393@qq.com>
This commit is contained in:
1 parent
eb11446b7f
commit
8e26dce1f4
8 files changed
+291
-59
No files matched your search
@@ -21,7 +21,7 @@ th {
|
||||
|--------------|--------|-------------------|------------|---------|-----|-----------|--------|
|
||||
| `Qwen3OmniMoeForConditionalGeneration` | Qwen3-Omni | `Qwen/Qwen3-Omni-30B-A3B-Instruct` | ✅︎ | | | | [Repository](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Qwen/Qwen3-Omni.md) |
|
||||
| `Qwen2_5OmniForConditionalGeneration` | Qwen2.5-Omni | `Qwen/Qwen2.5-Omni-7B`, `Qwen/Qwen2.5-Omni-3B` | ✅︎ | ✅︎ | ✅︎ | ✅︎ | — |
|
||||
| `MingFlashOmniForConditionalGeneration` + `MingImagePipeline` | Ming-flash-omni-2.0 (omni-speech + imagegen<sup>1</sup>) | `Jonathan1909/Ming-flash-omni-2.0` | ✅︎ | | | | — |
|
||||
| `MingFlashOmniForConditionalGeneration` + `MingImagePipeline` | Ming-flash-omni-2.0 (omni-speech + imagegen<sup>1</sup>) | `Jonathan1909/Ming-flash-omni-2.0` | ✅︎ | | | | — |
|
||||
| `BagelForConditionalGeneration` | BAGEL (DiT-only) | `ByteDance-Seed/BAGEL-7B-MoT` | ✅︎ | ✅︎ | | ✅︎ | — |
|
||||
| `InternVLAA1Pipeline` | InternVLA-A1 | `InternRobotics/InternVLA-A1-3B` | ✅︎ | ✅︎ | | | — |
|
||||
| `Gr00tN1d7Pipeline` | GR00T N1.7 | `nvidia/GR00T-N1.7-3B` | ✅︎ | | | | — |
|
||||
@@ -62,7 +62,7 @@ th {
|
||||
| `LongcatImagePipeline` | LongCat-Image | `meituan-longcat/LongCat-Image` | ✅︎ | ✅︎ | ✅︎ | ✅︎ | — |
|
||||
| `LongCatImageEditPipeline` | LongCat-Image-Edit | `meituan-longcat/LongCat-Image-Edit` | ✅︎ | ✅︎ | ✅︎ | ✅︎ | — |
|
||||
| `LongCatVideoAvatarPipeline` | LongCat-Video-Avatar-1.5 A2V/AI2V (native single-speaker and multi-speaker AI2V/AVC) | `meituan-longcat/LongCat-Video-Avatar-1.5` | ✅︎ | | | | [Repository](https://github.com/vllm-project/vllm-omni/blob/main/recipes/meituan-longcat/LongCat-Video-Avatar-1.5.md) |
|
||||
| `BooguImagePipeline` | Boogu-Image | `Boogu/Boogu-Image-0.1-Base`, `Boogu/Boogu-Image-0.1-Edit` | ✅︎ | | | ✅︎ | — |
|
||||
| `BooguImagePipeline` | Boogu-Image | `Boogu/Boogu-Image-0.1-Base`, `Boogu/Boogu-Image-0.1-Edit`, `Boogu/Boogu-Image-0.1-Edit-Turbo` | ✅︎ | | | ✅︎ | [Repository](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Boogu/Boogu-Image.md) |
|
||||
| `StableDiffusionXLPipeline` | Stable-Diffusion-XL | `stabilityai/stable-diffusion-xl-base-1.0` | ✅︎ | ✅︎ | ✅︎ | ✅︎ | — |
|
||||
| `StableDiffusion3Pipeline` | Stable-Diffusion-3 | `stabilityai/stable-diffusion-3.5-medium` | ✅︎ | ✅︎ | | ✅︎ | — |
|
||||
| `CosyVoice3Model` | CosyVoice3 | `FunAudioLLM/Fun-CosyVoice3-0.5B-2512` | ✅︎ | ✅︎ | | ✅︎ | — |
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
# Boogu-Image
|
||||
|
||||
> Text-to-image and image-editing online serving (Boogu-Image-0.1-Base / -Edit)
|
||||
> Text-to-image and image-editing online serving
|
||||
(Boogu-Image-0.1-Base / -Edit / -Edit-Turbo)
|
||||
|
||||
## Summary
|
||||
|
||||
- Vendor: Boogu
|
||||
- Model: `Boogu/Boogu-Image-0.1-Base` (text-to-image), `Boogu/Boogu-Image-0.1-Edit` (image editing)
|
||||
- Model: `Boogu/Boogu-Image-0.1-Base` (text-to-image),
|
||||
`Boogu/Boogu-Image-0.1-Edit` (image editing), and
|
||||
`Boogu/Boogu-Image-0.1-Edit-Turbo` (four-step image editing)
|
||||
- Task: Text-to-image generation and text-guided image editing (TI2I)
|
||||
- Mode: Online serving with the OpenAI-compatible API
|
||||
- Maintainer: Community
|
||||
@@ -13,19 +16,23 @@
|
||||
## When to use this recipe
|
||||
|
||||
Use this recipe when you want a known-good starting point for serving
|
||||
`Boogu/Boogu-Image-0.1-Base` with vLLM-Omni's native pipeline (no
|
||||
`Boogu/Boogu-Image-0.1-Base`, `Boogu/Boogu-Image-0.1-Edit`, or
|
||||
`Boogu/Boogu-Image-0.1-Edit-Turbo` with vLLM-Omni's native pipeline (no
|
||||
`--diffusion-load-format diffusers`, and the upstream `boogu` package is not
|
||||
required).
|
||||
|
||||
Boogu-Image-0.1 is an Apache-2.0 unified image generation and editing model
|
||||
family. This recipe covers the Base text-to-image checkpoint, which pairs a
|
||||
Qwen3-VL multimodal encoder with a Diffusion Transformer (DiT) and a flow-match
|
||||
Euler scheduler with time-shift. It handles photorealistic generation and
|
||||
Chinese/English text rendering.
|
||||
family. The Base text-to-image checkpoint pairs a Qwen3-VL multimodal encoder
|
||||
with a Diffusion Transformer (DiT) and a flow-match Euler scheduler with
|
||||
time-shift. It handles photorealistic generation and Chinese/English text
|
||||
rendering. The Edit and Edit-Turbo checkpoints use the same native
|
||||
`BooguImagePipeline` for image editing.
|
||||
|
||||
## References
|
||||
|
||||
- Upstream model card: <https://huggingface.co/Boogu/Boogu-Image-0.1-Base>
|
||||
- Edit model card: <https://huggingface.co/Boogu/Boogu-Image-0.1-Edit>
|
||||
- Edit-Turbo 1K hotfix: <https://huggingface.co/Boogu/Boogu-Image-0.1-Edit-Turbo/tree/hotfix-1k-20260708>
|
||||
- Project page: <https://boogu.org>
|
||||
- GitHub: <https://github.com/boogu-project/Boogu-Image>
|
||||
- Related example: [`examples/online_serving/text_to_image/`](../../examples/online_serving/text_to_image/README.md)
|
||||
@@ -93,7 +100,7 @@ curl -s http://localhost:8091/v1/chat/completions \
|
||||
- **Memory usage:** ~34.6 GiB on GPU. Use `--vae-use-slicing --vae-use-tiling`
|
||||
to trim peak VRAM if needed.
|
||||
- **Key flags:**
|
||||
- `--omni` — enables vLLM-Omni diffusion serving.
|
||||
- `--omni` — enables vLLM-Omni diffusion serving.
|
||||
- **Guidance:** Boogu-Image uses `guidance_scale` (mapped to the upstream
|
||||
`text_guidance_scale`); the default is `4.0`. Classifier-free guidance is
|
||||
active whenever `guidance_scale > 1.0`.
|
||||
@@ -111,13 +118,13 @@ the image-editing (TI2I) path activates automatically when a request carries a
|
||||
reference image. The Base text-to-image path is unaffected (no reference image
|
||||
is sent).
|
||||
|
||||
#### Command
|
||||
### Command
|
||||
|
||||
```bash
|
||||
vllm serve Boogu/Boogu-Image-0.1-Edit --omni --port 8091
|
||||
```
|
||||
|
||||
#### Verification
|
||||
### Verification
|
||||
|
||||
Edit an image with `/v1/images/edits` (the model-card example — change a photo
|
||||
to a colored-pencil drawing). Diffusion parameters are plain multipart form
|
||||
@@ -157,14 +164,14 @@ curl -s http://localhost:8091/v1/chat/completions \
|
||||
}' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > edited.png
|
||||
```
|
||||
|
||||
#### Notes
|
||||
### Notes
|
||||
|
||||
- **Single reference image:** only one input image is supported for now (the
|
||||
upstream "Only support 1 reference image for now" limit).
|
||||
- **Guidance semantics:**
|
||||
- `guidance_scale` = text guidance (upstream `text_guidance_scale`, default
|
||||
- `guidance_scale` = text guidance (upstream `text_guidance_scale`, default
|
||||
`4.0`); `> 1.0` enables text CFG. Editing typically uses `5.0`.
|
||||
- `guidance_scale_2` = image guidance (upstream `image_guidance_scale`,
|
||||
- `guidance_scale_2` = image guidance (upstream `image_guidance_scale`,
|
||||
default `1.0` = off). Setting it `> 1.0` enables the double-guidance path
|
||||
(3 model predictions per step), steering more strongly toward the reference
|
||||
image.
|
||||
@@ -178,3 +185,69 @@ curl -s http://localhost:8091/v1/chat/completions \
|
||||
supports Boogu-Image-Edit directly
|
||||
(`--model Boogu/Boogu-Image-0.1-Edit --guidance-scale 5.0`, optional
|
||||
`--guidance-scale-2`).
|
||||
|
||||
## Fast image editing (Boogu-Image-0.1-Edit-Turbo)
|
||||
|
||||
Edit-Turbo is the distilled image-editing checkpoint. Its `model_index.json`
|
||||
declares `BooguImagePipeline`, so it uses the same native TI2I path as Edit.
|
||||
The upstream 1K hotfix is recommended for stable results and is pinned below
|
||||
to avoid silently loading the older checkpoint from the repository's default
|
||||
revision.
|
||||
|
||||
### Command
|
||||
|
||||
```bash
|
||||
vllm serve Boogu/Boogu-Image-0.1-Edit-Turbo \
|
||||
--omni \
|
||||
--revision hotfix-1k-20260708 \
|
||||
--port 8091
|
||||
```
|
||||
|
||||
### Verification
|
||||
|
||||
Edit an image with the distilled four-step settings:
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8091/v1/images/edits \
|
||||
-F model="Boogu/Boogu-Image-0.1-Edit-Turbo" \
|
||||
-F image="@input.png" \
|
||||
-F prompt="Change the style to a colored pencil drawing." \
|
||||
-F num_inference_steps=4 \
|
||||
-F guidance_scale=1.0 \
|
||||
-F seed=42 \
|
||||
| jq -r '.data[0].b64_json' | base64 -d > edited-turbo.png
|
||||
```
|
||||
|
||||
Or via chat completions:
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8091/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"messages": [
|
||||
{"role": "user", "content": [
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64>"}},
|
||||
{"type": "text", "text": "Change the style to a colored pencil drawing."}
|
||||
]}
|
||||
],
|
||||
"extra_body": {
|
||||
"num_inference_steps": 4,
|
||||
"guidance_scale": 1.0,
|
||||
"seed": 42
|
||||
}
|
||||
}' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > edited-turbo.png
|
||||
```
|
||||
|
||||
### Notes
|
||||
|
||||
- **Pinned revision:** use `hotfix-1k-20260708`; upstream recommends the 1K
|
||||
hotfix over the 1.5K variant for more stable results.
|
||||
- **Recommended settings:** `num_inference_steps=4` and
|
||||
`guidance_scale=1.0`. Edit-Turbo is guidance-distilled, meaning the guidance
|
||||
behavior is already baked into the distilled checkpoint. A scale of `1.0`
|
||||
avoids applying additional classifier-free guidance; unlike the regular Edit
|
||||
checkpoint, it should not be increased to `5.0`.
|
||||
- **Reference images:** the same single-reference and `align_res` behavior as
|
||||
the regular Edit checkpoint applies.
|
||||
- **Known limitations:** the same single-GPU limitations as Base and Edit apply;
|
||||
CPU offload, Cache-DiT, and multi-GPU parallelism are not yet validated.
|
||||
@@ -1,5 +1,5 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
|
||||
"""L1 unit tests for the native Boogu-Image pipeline.
|
||||
|
||||
@@ -133,6 +133,51 @@ def test_constructor_strips_mllm_lm_head(boogu_pipeline, mock_dependencies):
|
||||
assert boogu_pipeline.mllm is mock_dependencies["inner_encoder"]
|
||||
|
||||
|
||||
def test_constructor_forwards_revision_to_all_component_loaders(mock_dependencies, mocker):
|
||||
from vllm_omni.diffusion.models.boogu_image.pipeline_boogu_image import (
|
||||
BooguImagePipeline,
|
||||
)
|
||||
|
||||
revision = "hotfix-1k-20260708"
|
||||
prefetch = mocker.patch(f"{_MODULE}.prefetch_subfolders")
|
||||
scheduler_loader = mocker.patch(
|
||||
f"{_MODULE}.FlowMatchEulerDiscreteScheduler.from_pretrained",
|
||||
return_value=mock_dependencies["scheduler"],
|
||||
)
|
||||
mllm_loader = mocker.patch(
|
||||
f"{_MODULE}.Qwen3VLForConditionalGeneration.from_pretrained",
|
||||
return_value=mock_dependencies["mllm_wrapper"],
|
||||
)
|
||||
processor_loader = mocker.patch(
|
||||
f"{_MODULE}.Qwen3VLProcessor.from_pretrained",
|
||||
return_value=mock_dependencies["processor"],
|
||||
)
|
||||
vae_loader = mocker.patch(
|
||||
f"{_MODULE}.AutoencoderKL.from_pretrained",
|
||||
return_value=mock_dependencies["vae"],
|
||||
)
|
||||
od_config = OmniDiffusionConfig(
|
||||
model="dummy-boogu",
|
||||
revision=revision,
|
||||
tf_model_config=TransformerConfig(params={}),
|
||||
dtype=torch.float32,
|
||||
num_gpus=1,
|
||||
)
|
||||
|
||||
pipeline = BooguImagePipeline(od_config=od_config)
|
||||
|
||||
(source,) = pipeline.weights_sources
|
||||
assert source.revision == revision
|
||||
prefetch.assert_called_once_with(
|
||||
"dummy-boogu",
|
||||
["scheduler", "vae", "mllm", "processor"],
|
||||
local_files_only=True,
|
||||
revision=revision,
|
||||
)
|
||||
for loader in (scheduler_loader, mllm_loader, processor_loader, vae_loader):
|
||||
assert loader.call_args.kwargs["revision"] == revision
|
||||
|
||||
|
||||
def test_constructor_weights_sources(boogu_pipeline):
|
||||
(source,) = boogu_pipeline.weights_sources
|
||||
assert source.model_or_path == "dummy-boogu"
|
||||
|
||||
@@ -1,15 +1,15 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
|
||||
"""
|
||||
Online serving tests for Boogu-Image-0.1-Edit (image-to-image via chat completions).
|
||||
Online serving tests for Boogu-Image-0.1-Edit and Edit-Turbo.
|
||||
|
||||
The Edit checkpoint shares the ``BooguImagePipeline`` class with the Base
|
||||
Both Edit checkpoints share the ``BooguImagePipeline`` class with the Base
|
||||
text-to-image checkpoint; the editing (TI2I) path activates when the request
|
||||
carries a reference image. Text guidance rides on ``guidance_scale`` (upstream
|
||||
default 4.0), and image guidance rides on ``guidance_scale_2`` (upstream default
|
||||
1.0 = off; a value > 1 enables the double-guidance path). Only a single
|
||||
reference image is supported.
|
||||
carries a reference image. The regular Edit checkpoint uses text guidance via
|
||||
``guidance_scale`` and optional image guidance via ``guidance_scale_2``.
|
||||
Edit-Turbo is guidance-distilled and uses four steps with guidance scale 1.0.
|
||||
Only a single reference image is supported.
|
||||
|
||||
- ``test_single_image_to_image_001``: one reference image, text guidance only.
|
||||
- ``test_double_guidance_001``: one reference image, text + image guidance.
|
||||
@@ -40,6 +40,10 @@ pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
|
||||
EDIT_PROMPT = "Change the style to a colored pencil drawing."
|
||||
SINGLE_CARD_FEATURE_MARKS = hardware_marks(res={"cuda": "H100"})
|
||||
|
||||
EDIT_MODEL = "Boogu/Boogu-Image-0.1-Edit"
|
||||
EDIT_TURBO_MODEL = "Boogu/Boogu-Image-0.1-Edit-Turbo"
|
||||
EDIT_TURBO_REVISION = "hotfix-1k-20260708"
|
||||
|
||||
|
||||
def _get_diffusion_feature_cases(model: str):
|
||||
"""Return one ``default`` ``OmniServerParams`` row for ``model`` (no extra ``server_args``)."""
|
||||
@@ -54,13 +58,44 @@ def _get_diffusion_feature_cases(model: str):
|
||||
]
|
||||
|
||||
|
||||
def _get_single_edit_cases():
|
||||
"""Return the regular and distilled Edit server/request configurations."""
|
||||
return [
|
||||
pytest.param(
|
||||
OmniServerParams(model=EDIT_MODEL),
|
||||
{
|
||||
"num_inference_steps": 2,
|
||||
"guidance_scale": 5.0,
|
||||
},
|
||||
id="edit",
|
||||
marks=SINGLE_CARD_FEATURE_MARKS,
|
||||
),
|
||||
pytest.param(
|
||||
OmniServerParams(
|
||||
model=EDIT_TURBO_MODEL,
|
||||
server_args=["--revision", EDIT_TURBO_REVISION],
|
||||
),
|
||||
{
|
||||
"num_inference_steps": 4,
|
||||
"guidance_scale": 1.0,
|
||||
},
|
||||
id="edit-turbo-hotfix-1k",
|
||||
marks=SINGLE_CARD_FEATURE_MARKS,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"omni_server",
|
||||
_get_diffusion_feature_cases("Boogu/Boogu-Image-0.1-Edit"),
|
||||
indirect=True,
|
||||
("omni_server", "sampling_params"),
|
||||
_get_single_edit_cases(),
|
||||
indirect=["omni_server"],
|
||||
)
|
||||
def test_single_image_to_image_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
|
||||
"""Single-reference text-guided edit smoke for ``Boogu/Boogu-Image-0.1-Edit``."""
|
||||
def test_single_image_to_image_001(
|
||||
omni_server: OmniServer,
|
||||
sampling_params: dict,
|
||||
openai_client: OpenAIClientHandler,
|
||||
):
|
||||
"""Single-reference text-guided edit smoke for regular and Turbo Edit."""
|
||||
image_data_url = f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}"
|
||||
|
||||
messages = dummy_messages_from_mix_data(image_data_url=image_data_url, content_text=EDIT_PROMPT)
|
||||
@@ -69,9 +104,7 @@ def test_single_image_to_image_001(omni_server: OmniServer, openai_client: OpenA
|
||||
"model": omni_server.model,
|
||||
"messages": messages,
|
||||
"extra_body": {
|
||||
"num_inference_steps": 2,
|
||||
# Boogu text guidance (upstream default 4.0); > 1 enables CFG.
|
||||
"guidance_scale": 5.0,
|
||||
**sampling_params,
|
||||
"seed": 42,
|
||||
},
|
||||
}
|
||||
@@ -81,7 +114,7 @@ def test_single_image_to_image_001(omni_server: OmniServer, openai_client: OpenA
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"omni_server",
|
||||
_get_diffusion_feature_cases("Boogu/Boogu-Image-0.1-Edit"),
|
||||
_get_diffusion_feature_cases(EDIT_MODEL),
|
||||
indirect=True,
|
||||
)
|
||||
def test_double_guidance_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
|
||||
@@ -105,15 +138,20 @@ def test_double_guidance_001(omni_server: OmniServer, openai_client: OpenAIClien
|
||||
openai_client.send_diffusion_request(request_config)
|
||||
|
||||
|
||||
def _edits_request_config(model: str, *, guidance_scale_2: float | None = None) -> dict:
|
||||
def _edits_request_config(
|
||||
model: str,
|
||||
*,
|
||||
num_inference_steps: int = 2,
|
||||
guidance_scale: float = 5.0,
|
||||
guidance_scale_2: float | None = None,
|
||||
) -> dict:
|
||||
"""Multipart ``/v1/images/edits`` config: one synthetic reference + smoke-depth params."""
|
||||
image_bytes = base64.b64decode(generate_synthetic_image(512, 512)["base64"])
|
||||
data = {
|
||||
"model": model,
|
||||
"prompt": EDIT_PROMPT,
|
||||
"num_inference_steps": 2,
|
||||
# Boogu text guidance (upstream default 4.0); > 1 enables CFG.
|
||||
"guidance_scale": 5.0,
|
||||
"num_inference_steps": num_inference_steps,
|
||||
"guidance_scale": guidance_scale,
|
||||
"seed": 42,
|
||||
}
|
||||
if guidance_scale_2 is not None:
|
||||
@@ -133,19 +171,25 @@ def _assert_edits_response_has_image(responses) -> None:
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"omni_server",
|
||||
_get_diffusion_feature_cases("Boogu/Boogu-Image-0.1-Edit"),
|
||||
indirect=True,
|
||||
("omni_server", "sampling_params"),
|
||||
_get_single_edit_cases(),
|
||||
indirect=["omni_server"],
|
||||
)
|
||||
def test_images_edits_endpoint_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
|
||||
"""Single-reference text-guided edit via the OpenAI-compatible ``/v1/images/edits`` endpoint."""
|
||||
responses = openai_client.send_images_edits_http_request(_edits_request_config(omni_server.model))
|
||||
def test_images_edits_endpoint_001(
|
||||
omni_server: OmniServer,
|
||||
sampling_params: dict,
|
||||
openai_client: OpenAIClientHandler,
|
||||
):
|
||||
"""Single-reference regular and Turbo Edit via ``/v1/images/edits``."""
|
||||
responses = openai_client.send_images_edits_http_request(
|
||||
_edits_request_config(omni_server.model, **sampling_params)
|
||||
)
|
||||
_assert_edits_response_has_image(responses)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"omni_server",
|
||||
_get_diffusion_feature_cases("Boogu/Boogu-Image-0.1-Edit"),
|
||||
_get_diffusion_feature_cases(EDIT_MODEL),
|
||||
indirect=True,
|
||||
)
|
||||
def test_images_edits_endpoint_double_guidance_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
|
||||
"""
|
||||
L4 expansion coverage for ``Boogu/Boogu-Image-0.1-Edit`` (image editing / TI2I).
|
||||
L4 expansion coverage for Boogu-Image Edit and Edit-Turbo (image editing / TI2I).
|
||||
|
||||
The nightly smoke module (``test_boogu_image_edit.py``) already covers single-image
|
||||
text-guided editing and the text+image double-guidance path. This expansion
|
||||
@@ -16,11 +16,12 @@ module adds the two edit paths smoke does not exercise:
|
||||
``height`` / ``width``: asserts the output resolution follows the reference
|
||||
(upstream ``align_res``, on by default for a single reference).
|
||||
|
||||
The Edit checkpoint shares ``BooguImagePipeline`` with the Base checkpoint; the
|
||||
TI2I path activates when the request carries a reference image. Only a single
|
||||
reference image is supported. Cases stay smoke-depth (``num_inference_steps=2``).
|
||||
CPU offload, Cache-DiT, and multi-GPU parallelism are not yet supported for this
|
||||
model and are intentionally omitted.
|
||||
Both Edit checkpoints share ``BooguImagePipeline`` with the Base checkpoint;
|
||||
the TI2I path activates when the request carries a reference image. Only a
|
||||
single reference image is supported. The regular Edit cases stay smoke-depth
|
||||
(``num_inference_steps=2``), while Edit-Turbo uses its tuned four-step
|
||||
configuration. CPU offload, Cache-DiT, and multi-GPU parallelism are not yet
|
||||
supported for these models and are intentionally omitted.
|
||||
|
||||
From ``tests/``::
|
||||
|
||||
@@ -41,6 +42,8 @@ os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
|
||||
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
|
||||
|
||||
MODEL = "Boogu/Boogu-Image-0.1-Edit"
|
||||
EDIT_TURBO_MODEL = "Boogu/Boogu-Image-0.1-Edit-Turbo"
|
||||
EDIT_TURBO_REVISION = "hotfix-1k-20260708"
|
||||
EDIT_PROMPT = "Change the style to a colored pencil drawing."
|
||||
|
||||
SINGLE_CARD_FEATURE_MARKS = hardware_marks(res={"cuda": "H100"})
|
||||
@@ -57,6 +60,33 @@ def _get_diffusion_feature_cases(model: str):
|
||||
]
|
||||
|
||||
|
||||
def _get_align_res_cases():
|
||||
"""Return regular and Turbo Edit configurations for align-res coverage."""
|
||||
return [
|
||||
pytest.param(
|
||||
OmniServerParams(model=MODEL),
|
||||
{
|
||||
"num_inference_steps": 2,
|
||||
"guidance_scale": 5.0,
|
||||
},
|
||||
id="edit",
|
||||
marks=SINGLE_CARD_FEATURE_MARKS,
|
||||
),
|
||||
pytest.param(
|
||||
OmniServerParams(
|
||||
model=EDIT_TURBO_MODEL,
|
||||
server_args=["--revision", EDIT_TURBO_REVISION],
|
||||
),
|
||||
{
|
||||
"num_inference_steps": 4,
|
||||
"guidance_scale": 1.0,
|
||||
},
|
||||
id="edit-turbo-hotfix-1k",
|
||||
marks=SINGLE_CARD_FEATURE_MARKS,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("omni_server", _get_diffusion_feature_cases(MODEL), indirect=True)
|
||||
def test_image_only_guidance(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
|
||||
"""Image-only guidance: text CFG off (1.0), image CFG on (2.0) -> image-only CFG branch."""
|
||||
@@ -76,8 +106,16 @@ def test_image_only_guidance(omni_server: OmniServer, openai_client: OpenAIClien
|
||||
openai_client.send_diffusion_request(request_config)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("omni_server", _get_diffusion_feature_cases(MODEL), indirect=True)
|
||||
def test_align_res_output_size(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
|
||||
@pytest.mark.parametrize(
|
||||
("omni_server", "sampling_params"),
|
||||
_get_align_res_cases(),
|
||||
indirect=["omni_server"],
|
||||
)
|
||||
def test_align_res_output_size(
|
||||
omni_server: OmniServer,
|
||||
sampling_params: dict,
|
||||
openai_client: OpenAIClientHandler,
|
||||
) -> None:
|
||||
"""Output resolution follows the reference image (upstream ``align_res``).
|
||||
|
||||
No ``height`` / ``width`` is requested. The reference is 512x384 — both
|
||||
@@ -92,8 +130,7 @@ def test_align_res_output_size(omni_server: OmniServer, openai_client: OpenAICli
|
||||
"model": omni_server.model,
|
||||
"messages": messages,
|
||||
"extra_body": {
|
||||
"num_inference_steps": 2,
|
||||
"guidance_scale": 5.0,
|
||||
**sampling_params,
|
||||
"seed": 42,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
|
||||
"""Unit tests for vllm_omni.entrypoints.utils module."""
|
||||
|
||||
import os
|
||||
@@ -420,6 +423,7 @@ class TestLoadAndResolveStageConfigs:
|
||||
"""Ensure that kwargs survive default stage creation."""
|
||||
engine_backend = "vllm_omni.experimental.ar_diffusion.engine.ARDiffusionEngine"
|
||||
kwargs = {"dtype": torch.float32, "engine_backend": engine_backend}
|
||||
kwargs["revision"] = "pinned-revision"
|
||||
mocker.patch("vllm_omni.entrypoints.utils.resolve_model_config_path", return_value=None)
|
||||
mocker.patch("vllm_omni.entrypoints.utils.load_stage_configs_from_model", return_value=([], None))
|
||||
|
||||
@@ -433,6 +437,7 @@ class TestLoadAndResolveStageConfigs:
|
||||
assert len(stage_configs) == 1
|
||||
assert "dtype" in stage_configs[0]["engine_args"]
|
||||
assert stage_configs[0]["engine_args"]["engine_backend"] == engine_backend
|
||||
assert stage_configs[0]["engine_args"]["revision"] == "pinned-revision"
|
||||
|
||||
def test_deploy_config_preserves_cli_overrides_and_replicas(self, tmp_path, mocker: MockerFixture):
|
||||
deploy_path = tmp_path / "qwen3_multi.yaml"
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
|
||||
"""Native vLLM-Omni pipeline for Boogu-Image-0.1.
|
||||
|
||||
@@ -220,7 +220,7 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
|
||||
DiffusersPipelineLoader.ComponentSource(
|
||||
model_or_path=od_config.model,
|
||||
subfolder="transformer",
|
||||
revision=None,
|
||||
revision=od_config.revision,
|
||||
prefix="transformer.",
|
||||
fall_back_to_pt=True,
|
||||
)
|
||||
@@ -233,10 +233,18 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
|
||||
# See ``hub_prefetch.py`` for the transformers v5 multi-worker subfolder
|
||||
# race; prefetch the whole component set before any from_pretrained.
|
||||
boogu_subfolders = ["scheduler", "vae", "mllm", "processor"]
|
||||
prefetch_subfolders(model, boogu_subfolders, local_files_only=local_files_only)
|
||||
prefetch_subfolders(
|
||||
model,
|
||||
boogu_subfolders,
|
||||
local_files_only=local_files_only,
|
||||
revision=od_config.revision,
|
||||
)
|
||||
|
||||
self.scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
|
||||
model, subfolder="scheduler", local_files_only=local_files_only
|
||||
model,
|
||||
subfolder="scheduler",
|
||||
local_files_only=local_files_only,
|
||||
revision=od_config.revision,
|
||||
)
|
||||
|
||||
mllm = from_pretrained_with_prefetch(
|
||||
@@ -246,6 +254,7 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
|
||||
prefetch_list=boogu_subfolders,
|
||||
local_files_only=local_files_only,
|
||||
torch_dtype=od_config.dtype,
|
||||
revision=od_config.revision,
|
||||
)
|
||||
# Upstream reuses the full VLM as an optional instruction rewriter and
|
||||
# encodes with its inner model (no ``lm_head``); the rewriter is not
|
||||
@@ -255,7 +264,10 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
|
||||
self.mllm = mllm.to(self._execution_device)
|
||||
|
||||
self.processor = Qwen3VLProcessor.from_pretrained(
|
||||
model, subfolder="processor", local_files_only=local_files_only
|
||||
model,
|
||||
subfolder="processor",
|
||||
local_files_only=local_files_only,
|
||||
revision=od_config.revision,
|
||||
)
|
||||
|
||||
self.vae = from_pretrained_with_prefetch(
|
||||
@@ -264,6 +276,7 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
|
||||
subfolder="vae",
|
||||
prefetch_list=boogu_subfolders,
|
||||
local_files_only=local_files_only,
|
||||
revision=od_config.revision,
|
||||
).to(self._execution_device)
|
||||
|
||||
self.transformer = BooguImageTransformer2DModel(od_config=od_config)
|
||||
|
||||
@@ -1,3 +1,6 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
|
||||
|
||||
import json
|
||||
import os
|
||||
import types
|
||||
@@ -600,6 +603,18 @@ def load_and_resolve_stage_configs(
|
||||
stage_configs = []
|
||||
|
||||
stage_configs = filter_stages(config_path, stage_configs, kwargs)
|
||||
|
||||
# ``revision`` selects the model source for every stage. Keep the global
|
||||
# CLI pin on each stage's engine args so diffusion config construction can
|
||||
# pass it through to all component loaders. The registry-to-legacy-stage
|
||||
# conversion currently drops this inherited vLLM EngineArgs field.
|
||||
revision = (kwargs or {}).get("revision")
|
||||
if revision is not None:
|
||||
for stage_config in stage_configs:
|
||||
engine_args = stage_config.get("engine_args") if hasattr(stage_config, "get") else None
|
||||
if engine_args is not None and engine_args.get("revision") is None:
|
||||
engine_args["revision"] = revision
|
||||
|
||||
logger.debug(f"stage_configs: {stage_configs}")
|
||||
|
||||
return config_path, stage_configs, omni_lb_policy
|
||||
|
||||
Reference in new issue
Block a user