[Diffusion] Add Boogu Edit-Turbo support docs and tests (#6701)

Signed-off-by: xRay2016 <1150722393@qq.com>
This commit is contained in:
XRay2016 authored and GitHub committed 2026-09-01 08:41:23 +08:00
1 parent eb11446b7f
commit 8e26dce1f4
8 files changed
+291 -59

No files matched your search

+2 -2
View File
@@ -21,7 +21,7 @@ th {
|--------------|--------|-------------------|------------|---------|-----|-----------|--------|
| `Qwen3OmniMoeForConditionalGeneration` | Qwen3-Omni | `Qwen/Qwen3-Omni-30B-A3B-Instruct` | ✅︎ | | | | [Repository](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Qwen/Qwen3-Omni.md) |
| `Qwen2_5OmniForConditionalGeneration` | Qwen2.5-Omni | `Qwen/Qwen2.5-Omni-7B`, `Qwen/Qwen2.5-Omni-3B` | ✅︎ | ✅︎ | ✅︎ | ✅︎ | — |
| `MingFlashOmniForConditionalGeneration` + `MingImagePipeline` | Ming-flash-omni-2.0 (omni-speech + imagegen<sup>1</sup>) | `Jonathan1909/Ming-flash-omni-2.0` | ✅︎ | | | | — |
| `MingFlashOmniForConditionalGeneration` + `MingImagePipeline` | Ming-flash-omni-2.0 (omni-speech + imagegen<sup>1</sup>) | `Jonathan1909/Ming-flash-omni-2.0` | ✅︎ | | | | — |
| `BagelForConditionalGeneration` | BAGEL (DiT-only) | `ByteDance-Seed/BAGEL-7B-MoT` | ✅︎ | ✅︎ | | ✅︎ | — |
| `InternVLAA1Pipeline` | InternVLA-A1 | `InternRobotics/InternVLA-A1-3B` | ✅︎ | ✅︎ | | | — |
| `Gr00tN1d7Pipeline` | GR00T N1.7 | `nvidia/GR00T-N1.7-3B` | ✅︎ | | | | — |
@@ -62,7 +62,7 @@ th {
| `LongcatImagePipeline` | LongCat-Image | `meituan-longcat/LongCat-Image` | ✅︎ | ✅︎ | ✅︎ | ✅︎ | — |
| `LongCatImageEditPipeline` | LongCat-Image-Edit | `meituan-longcat/LongCat-Image-Edit` | ✅︎ | ✅︎ | ✅︎ | ✅︎ | — |
| `LongCatVideoAvatarPipeline` | LongCat-Video-Avatar-1.5 A2V/AI2V (native single-speaker and multi-speaker AI2V/AVC) | `meituan-longcat/LongCat-Video-Avatar-1.5` | ✅︎ | | | | [Repository](https://github.com/vllm-project/vllm-omni/blob/main/recipes/meituan-longcat/LongCat-Video-Avatar-1.5.md) |
| `BooguImagePipeline` | Boogu-Image | `Boogu/Boogu-Image-0.1-Base`, `Boogu/Boogu-Image-0.1-Edit` | ✅︎ | | | ✅︎ | — |
| `BooguImagePipeline` | Boogu-Image | `Boogu/Boogu-Image-0.1-Base`, `Boogu/Boogu-Image-0.1-Edit`, `Boogu/Boogu-Image-0.1-Edit-Turbo` | ✅︎ | | | ✅︎ | [Repository](https://github.com/vllm-project/vllm-omni/blob/main/recipes/Boogu/Boogu-Image.md) |
| `StableDiffusionXLPipeline` | Stable-Diffusion-XL | `stabilityai/stable-diffusion-xl-base-1.0` | ✅︎ | ✅︎ | ✅︎ | ✅︎ | — |
| `StableDiffusion3Pipeline` | Stable-Diffusion-3 | `stabilityai/stable-diffusion-3.5-medium` | ✅︎ | ✅︎ | | ✅︎ | — |
| `CosyVoice3Model` | CosyVoice3 | `FunAudioLLM/Fun-CosyVoice3-0.5B-2512` | ✅︎ | ✅︎ | | ✅︎ | — |
+86 -13
View File
@@ -1,11 +1,14 @@
# Boogu-Image
> Text-to-image and image-editing online serving (Boogu-Image-0.1-Base / -Edit)
> Text-to-image and image-editing online serving
(Boogu-Image-0.1-Base / -Edit / -Edit-Turbo)
## Summary
- Vendor: Boogu
- Model: `Boogu/Boogu-Image-0.1-Base` (text-to-image), `Boogu/Boogu-Image-0.1-Edit` (image editing)
- Model: `Boogu/Boogu-Image-0.1-Base` (text-to-image),
`Boogu/Boogu-Image-0.1-Edit` (image editing), and
`Boogu/Boogu-Image-0.1-Edit-Turbo` (four-step image editing)
- Task: Text-to-image generation and text-guided image editing (TI2I)
- Mode: Online serving with the OpenAI-compatible API
- Maintainer: Community
@@ -13,19 +16,23 @@
## When to use this recipe
Use this recipe when you want a known-good starting point for serving
`Boogu/Boogu-Image-0.1-Base` with vLLM-Omni's native pipeline (no
`Boogu/Boogu-Image-0.1-Base`, `Boogu/Boogu-Image-0.1-Edit`, or
`Boogu/Boogu-Image-0.1-Edit-Turbo` with vLLM-Omni's native pipeline (no
`--diffusion-load-format diffusers`, and the upstream `boogu` package is not
required).
Boogu-Image-0.1 is an Apache-2.0 unified image generation and editing model
family. This recipe covers the Base text-to-image checkpoint, which pairs a
Qwen3-VL multimodal encoder with a Diffusion Transformer (DiT) and a flow-match
Euler scheduler with time-shift. It handles photorealistic generation and
Chinese/English text rendering.
family. The Base text-to-image checkpoint pairs a Qwen3-VL multimodal encoder
with a Diffusion Transformer (DiT) and a flow-match Euler scheduler with
time-shift. It handles photorealistic generation and Chinese/English text
rendering. The Edit and Edit-Turbo checkpoints use the same native
`BooguImagePipeline` for image editing.
## References
- Upstream model card: <https://huggingface.co/Boogu/Boogu-Image-0.1-Base>
- Edit model card: <https://huggingface.co/Boogu/Boogu-Image-0.1-Edit>
- Edit-Turbo 1K hotfix: <https://huggingface.co/Boogu/Boogu-Image-0.1-Edit-Turbo/tree/hotfix-1k-20260708>
- Project page: <https://boogu.org>
- GitHub: <https://github.com/boogu-project/Boogu-Image>
- Related example: [`examples/online_serving/text_to_image/`](../../examples/online_serving/text_to_image/README.md)
@@ -93,7 +100,7 @@ curl -s http://localhost:8091/v1/chat/completions \
- **Memory usage:** ~34.6 GiB on GPU. Use `--vae-use-slicing --vae-use-tiling`
to trim peak VRAM if needed.
- **Key flags:**
- `--omni` — enables vLLM-Omni diffusion serving.
- `--omni` — enables vLLM-Omni diffusion serving.
- **Guidance:** Boogu-Image uses `guidance_scale` (mapped to the upstream
`text_guidance_scale`); the default is `4.0`. Classifier-free guidance is
active whenever `guidance_scale > 1.0`.
@@ -111,13 +118,13 @@ the image-editing (TI2I) path activates automatically when a request carries a
reference image. The Base text-to-image path is unaffected (no reference image
is sent).
#### Command
### Command
```bash
vllm serve Boogu/Boogu-Image-0.1-Edit --omni --port 8091
```
#### Verification
### Verification
Edit an image with `/v1/images/edits` (the model-card example — change a photo
to a colored-pencil drawing). Diffusion parameters are plain multipart form
@@ -157,14 +164,14 @@ curl -s http://localhost:8091/v1/chat/completions \
}' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > edited.png
```
#### Notes
### Notes
- **Single reference image:** only one input image is supported for now (the
upstream "Only support 1 reference image for now" limit).
- **Guidance semantics:**
- `guidance_scale` = text guidance (upstream `text_guidance_scale`, default
- `guidance_scale` = text guidance (upstream `text_guidance_scale`, default
`4.0`); `> 1.0` enables text CFG. Editing typically uses `5.0`.
- `guidance_scale_2` = image guidance (upstream `image_guidance_scale`,
- `guidance_scale_2` = image guidance (upstream `image_guidance_scale`,
default `1.0` = off). Setting it `> 1.0` enables the double-guidance path
(3 model predictions per step), steering more strongly toward the reference
image.
@@ -178,3 +185,69 @@ curl -s http://localhost:8091/v1/chat/completions \
supports Boogu-Image-Edit directly
(`--model Boogu/Boogu-Image-0.1-Edit --guidance-scale 5.0`, optional
`--guidance-scale-2`).
## Fast image editing (Boogu-Image-0.1-Edit-Turbo)
Edit-Turbo is the distilled image-editing checkpoint. Its `model_index.json`
declares `BooguImagePipeline`, so it uses the same native TI2I path as Edit.
The upstream 1K hotfix is recommended for stable results and is pinned below
to avoid silently loading the older checkpoint from the repository's default
revision.
### Command
```bash
vllm serve Boogu/Boogu-Image-0.1-Edit-Turbo \
--omni \
--revision hotfix-1k-20260708 \
--port 8091
```
### Verification
Edit an image with the distilled four-step settings:
```bash
curl -s http://localhost:8091/v1/images/edits \
-F model="Boogu/Boogu-Image-0.1-Edit-Turbo" \
-F image="@input.png" \
-F prompt="Change the style to a colored pencil drawing." \
-F num_inference_steps=4 \
-F guidance_scale=1.0 \
-F seed=42 \
| jq -r '.data[0].b64_json' | base64 -d > edited-turbo.png
```
Or via chat completions:
```bash
curl -s http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": "data:image/png;base64,<BASE64>"}},
{"type": "text", "text": "Change the style to a colored pencil drawing."}
]}
],
"extra_body": {
"num_inference_steps": 4,
"guidance_scale": 1.0,
"seed": 42
}
}' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > edited-turbo.png
```
### Notes
- **Pinned revision:** use `hotfix-1k-20260708`; upstream recommends the 1K
hotfix over the 1.5K variant for more stable results.
- **Recommended settings:** `num_inference_steps=4` and
`guidance_scale=1.0`. Edit-Turbo is guidance-distilled, meaning the guidance
behavior is already baked into the distilled checkpoint. A scale of `1.0`
avoids applying additional classifier-free guidance; unlike the regular Edit
checkpoint, it should not be increased to `5.0`.
- **Reference images:** the same single-reference and `align_res` behavior as
the regular Edit checkpoint applies.
- **Known limitations:** the same single-GPU limitations as Base and Edit apply;
CPU offload, Cache-DiT, and multi-GPU parallelism are not yet validated.
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""L1 unit tests for the native Boogu-Image pipeline.
@@ -133,6 +133,51 @@ def test_constructor_strips_mllm_lm_head(boogu_pipeline, mock_dependencies):
assert boogu_pipeline.mllm is mock_dependencies["inner_encoder"]
def test_constructor_forwards_revision_to_all_component_loaders(mock_dependencies, mocker):
from vllm_omni.diffusion.models.boogu_image.pipeline_boogu_image import (
BooguImagePipeline,
)
revision = "hotfix-1k-20260708"
prefetch = mocker.patch(f"{_MODULE}.prefetch_subfolders")
scheduler_loader = mocker.patch(
f"{_MODULE}.FlowMatchEulerDiscreteScheduler.from_pretrained",
return_value=mock_dependencies["scheduler"],
)
mllm_loader = mocker.patch(
f"{_MODULE}.Qwen3VLForConditionalGeneration.from_pretrained",
return_value=mock_dependencies["mllm_wrapper"],
)
processor_loader = mocker.patch(
f"{_MODULE}.Qwen3VLProcessor.from_pretrained",
return_value=mock_dependencies["processor"],
)
vae_loader = mocker.patch(
f"{_MODULE}.AutoencoderKL.from_pretrained",
return_value=mock_dependencies["vae"],
)
od_config = OmniDiffusionConfig(
model="dummy-boogu",
revision=revision,
tf_model_config=TransformerConfig(params={}),
dtype=torch.float32,
num_gpus=1,
)
pipeline = BooguImagePipeline(od_config=od_config)
(source,) = pipeline.weights_sources
assert source.revision == revision
prefetch.assert_called_once_with(
"dummy-boogu",
["scheduler", "vae", "mllm", "processor"],
local_files_only=True,
revision=revision,
)
for loader in (scheduler_loader, mllm_loader, processor_loader, vae_loader):
assert loader.call_args.kwargs["revision"] == revision
def test_constructor_weights_sources(boogu_pipeline):
(source,) = boogu_pipeline.weights_sources
assert source.model_or_path == "dummy-boogu"
@@ -1,15 +1,15 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
Online serving tests for Boogu-Image-0.1-Edit (image-to-image via chat completions).
Online serving tests for Boogu-Image-0.1-Edit and Edit-Turbo.
The Edit checkpoint shares the ``BooguImagePipeline`` class with the Base
Both Edit checkpoints share the ``BooguImagePipeline`` class with the Base
text-to-image checkpoint; the editing (TI2I) path activates when the request
carries a reference image. Text guidance rides on ``guidance_scale`` (upstream
default 4.0), and image guidance rides on ``guidance_scale_2`` (upstream default
1.0 = off; a value > 1 enables the double-guidance path). Only a single
reference image is supported.
carries a reference image. The regular Edit checkpoint uses text guidance via
``guidance_scale`` and optional image guidance via ``guidance_scale_2``.
Edit-Turbo is guidance-distilled and uses four steps with guidance scale 1.0.
Only a single reference image is supported.
- ``test_single_image_to_image_001``: one reference image, text guidance only.
- ``test_double_guidance_001``: one reference image, text + image guidance.
@@ -40,6 +40,10 @@ pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
EDIT_PROMPT = "Change the style to a colored pencil drawing."
SINGLE_CARD_FEATURE_MARKS = hardware_marks(res={"cuda": "H100"})
EDIT_MODEL = "Boogu/Boogu-Image-0.1-Edit"
EDIT_TURBO_MODEL = "Boogu/Boogu-Image-0.1-Edit-Turbo"
EDIT_TURBO_REVISION = "hotfix-1k-20260708"
def _get_diffusion_feature_cases(model: str):
"""Return one ``default`` ``OmniServerParams`` row for ``model`` (no extra ``server_args``)."""
@@ -54,13 +58,44 @@ def _get_diffusion_feature_cases(model: str):
]
def _get_single_edit_cases():
"""Return the regular and distilled Edit server/request configurations."""
return [
pytest.param(
OmniServerParams(model=EDIT_MODEL),
{
"num_inference_steps": 2,
"guidance_scale": 5.0,
},
id="edit",
marks=SINGLE_CARD_FEATURE_MARKS,
),
pytest.param(
OmniServerParams(
model=EDIT_TURBO_MODEL,
server_args=["--revision", EDIT_TURBO_REVISION],
),
{
"num_inference_steps": 4,
"guidance_scale": 1.0,
},
id="edit-turbo-hotfix-1k",
marks=SINGLE_CARD_FEATURE_MARKS,
),
]
@pytest.mark.parametrize(
"omni_server",
_get_diffusion_feature_cases("Boogu/Boogu-Image-0.1-Edit"),
indirect=True,
("omni_server", "sampling_params"),
_get_single_edit_cases(),
indirect=["omni_server"],
)
def test_single_image_to_image_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
"""Single-reference text-guided edit smoke for ``Boogu/Boogu-Image-0.1-Edit``."""
def test_single_image_to_image_001(
omni_server: OmniServer,
sampling_params: dict,
openai_client: OpenAIClientHandler,
):
"""Single-reference text-guided edit smoke for regular and Turbo Edit."""
image_data_url = f"data:image/jpeg;base64,{generate_synthetic_image(512, 512)['base64']}"
messages = dummy_messages_from_mix_data(image_data_url=image_data_url, content_text=EDIT_PROMPT)
@@ -69,9 +104,7 @@ def test_single_image_to_image_001(omni_server: OmniServer, openai_client: OpenA
"model": omni_server.model,
"messages": messages,
"extra_body": {
"num_inference_steps": 2,
# Boogu text guidance (upstream default 4.0); > 1 enables CFG.
"guidance_scale": 5.0,
**sampling_params,
"seed": 42,
},
}
@@ -81,7 +114,7 @@ def test_single_image_to_image_001(omni_server: OmniServer, openai_client: OpenA
@pytest.mark.parametrize(
"omni_server",
_get_diffusion_feature_cases("Boogu/Boogu-Image-0.1-Edit"),
_get_diffusion_feature_cases(EDIT_MODEL),
indirect=True,
)
def test_double_guidance_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
@@ -105,15 +138,20 @@ def test_double_guidance_001(omni_server: OmniServer, openai_client: OpenAIClien
openai_client.send_diffusion_request(request_config)
def _edits_request_config(model: str, *, guidance_scale_2: float | None = None) -> dict:
def _edits_request_config(
model: str,
*,
num_inference_steps: int = 2,
guidance_scale: float = 5.0,
guidance_scale_2: float | None = None,
) -> dict:
"""Multipart ``/v1/images/edits`` config: one synthetic reference + smoke-depth params."""
image_bytes = base64.b64decode(generate_synthetic_image(512, 512)["base64"])
data = {
"model": model,
"prompt": EDIT_PROMPT,
"num_inference_steps": 2,
# Boogu text guidance (upstream default 4.0); > 1 enables CFG.
"guidance_scale": 5.0,
"num_inference_steps": num_inference_steps,
"guidance_scale": guidance_scale,
"seed": 42,
}
if guidance_scale_2 is not None:
@@ -133,19 +171,25 @@ def _assert_edits_response_has_image(responses) -> None:
@pytest.mark.parametrize(
"omni_server",
_get_diffusion_feature_cases("Boogu/Boogu-Image-0.1-Edit"),
indirect=True,
("omni_server", "sampling_params"),
_get_single_edit_cases(),
indirect=["omni_server"],
)
def test_images_edits_endpoint_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
"""Single-reference text-guided edit via the OpenAI-compatible ``/v1/images/edits`` endpoint."""
responses = openai_client.send_images_edits_http_request(_edits_request_config(omni_server.model))
def test_images_edits_endpoint_001(
omni_server: OmniServer,
sampling_params: dict,
openai_client: OpenAIClientHandler,
):
"""Single-reference regular and Turbo Edit via ``/v1/images/edits``."""
responses = openai_client.send_images_edits_http_request(
_edits_request_config(omni_server.model, **sampling_params)
)
_assert_edits_response_has_image(responses)
@pytest.mark.parametrize(
"omni_server",
_get_diffusion_feature_cases("Boogu/Boogu-Image-0.1-Edit"),
_get_diffusion_feature_cases(EDIT_MODEL),
indirect=True,
)
def test_images_edits_endpoint_double_guidance_001(omni_server: OmniServer, openai_client: OpenAIClientHandler):
@@ -1,8 +1,8 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""
L4 expansion coverage for ``Boogu/Boogu-Image-0.1-Edit`` (image editing / TI2I).
L4 expansion coverage for Boogu-Image Edit and Edit-Turbo (image editing / TI2I).
The nightly smoke module (``test_boogu_image_edit.py``) already covers single-image
text-guided editing and the text+image double-guidance path. This expansion
@@ -16,11 +16,12 @@ module adds the two edit paths smoke does not exercise:
``height`` / ``width``: asserts the output resolution follows the reference
(upstream ``align_res``, on by default for a single reference).
The Edit checkpoint shares ``BooguImagePipeline`` with the Base checkpoint; the
TI2I path activates when the request carries a reference image. Only a single
reference image is supported. Cases stay smoke-depth (``num_inference_steps=2``).
CPU offload, Cache-DiT, and multi-GPU parallelism are not yet supported for this
model and are intentionally omitted.
Both Edit checkpoints share ``BooguImagePipeline`` with the Base checkpoint;
the TI2I path activates when the request carries a reference image. Only a
single reference image is supported. The regular Edit cases stay smoke-depth
(``num_inference_steps=2``), while Edit-Turbo uses its tuned four-step
configuration. CPU offload, Cache-DiT, and multi-GPU parallelism are not yet
supported for these models and are intentionally omitted.
From ``tests/``::
@@ -41,6 +42,8 @@ os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
pytestmark = [pytest.mark.diffusion, pytest.mark.slow]
MODEL = "Boogu/Boogu-Image-0.1-Edit"
EDIT_TURBO_MODEL = "Boogu/Boogu-Image-0.1-Edit-Turbo"
EDIT_TURBO_REVISION = "hotfix-1k-20260708"
EDIT_PROMPT = "Change the style to a colored pencil drawing."
SINGLE_CARD_FEATURE_MARKS = hardware_marks(res={"cuda": "H100"})
@@ -57,6 +60,33 @@ def _get_diffusion_feature_cases(model: str):
]
def _get_align_res_cases():
"""Return regular and Turbo Edit configurations for align-res coverage."""
return [
pytest.param(
OmniServerParams(model=MODEL),
{
"num_inference_steps": 2,
"guidance_scale": 5.0,
},
id="edit",
marks=SINGLE_CARD_FEATURE_MARKS,
),
pytest.param(
OmniServerParams(
model=EDIT_TURBO_MODEL,
server_args=["--revision", EDIT_TURBO_REVISION],
),
{
"num_inference_steps": 4,
"guidance_scale": 1.0,
},
id="edit-turbo-hotfix-1k",
marks=SINGLE_CARD_FEATURE_MARKS,
),
]
@pytest.mark.parametrize("omni_server", _get_diffusion_feature_cases(MODEL), indirect=True)
def test_image_only_guidance(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
"""Image-only guidance: text CFG off (1.0), image CFG on (2.0) -> image-only CFG branch."""
@@ -76,8 +106,16 @@ def test_image_only_guidance(omni_server: OmniServer, openai_client: OpenAIClien
openai_client.send_diffusion_request(request_config)
@pytest.mark.parametrize("omni_server", _get_diffusion_feature_cases(MODEL), indirect=True)
def test_align_res_output_size(omni_server: OmniServer, openai_client: OpenAIClientHandler) -> None:
@pytest.mark.parametrize(
("omni_server", "sampling_params"),
_get_align_res_cases(),
indirect=["omni_server"],
)
def test_align_res_output_size(
omni_server: OmniServer,
sampling_params: dict,
openai_client: OpenAIClientHandler,
) -> None:
"""Output resolution follows the reference image (upstream ``align_res``).
No ``height`` / ``width`` is requested. The reference is 512x384 — both
@@ -92,8 +130,7 @@ def test_align_res_output_size(omni_server: OmniServer, openai_client: OpenAICli
"model": omni_server.model,
"messages": messages,
"extra_body": {
"num_inference_steps": 2,
"guidance_scale": 5.0,
**sampling_params,
"seed": 42,
},
}
+5
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Unit tests for vllm_omni.entrypoints.utils module."""
import os
@@ -420,6 +423,7 @@ class TestLoadAndResolveStageConfigs:
"""Ensure that kwargs survive default stage creation."""
engine_backend = "vllm_omni.experimental.ar_diffusion.engine.ARDiffusionEngine"
kwargs = {"dtype": torch.float32, "engine_backend": engine_backend}
kwargs["revision"] = "pinned-revision"
mocker.patch("vllm_omni.entrypoints.utils.resolve_model_config_path", return_value=None)
mocker.patch("vllm_omni.entrypoints.utils.load_stage_configs_from_model", return_value=([], None))
@@ -433,6 +437,7 @@ class TestLoadAndResolveStageConfigs:
assert len(stage_configs) == 1
assert "dtype" in stage_configs[0]["engine_args"]
assert stage_configs[0]["engine_args"]["engine_backend"] == engine_backend
assert stage_configs[0]["engine_args"]["revision"] == "pinned-revision"
def test_deploy_config_preserves_cli_overrides_and_replicas(self, tmp_path, mocker: MockerFixture):
deploy_path = tmp_path / "qwen3_multi.yaml"
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Native vLLM-Omni pipeline for Boogu-Image-0.1.
@@ -220,7 +220,7 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
DiffusersPipelineLoader.ComponentSource(
model_or_path=od_config.model,
subfolder="transformer",
revision=None,
revision=od_config.revision,
prefix="transformer.",
fall_back_to_pt=True,
)
@@ -233,10 +233,18 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
# See ``hub_prefetch.py`` for the transformers v5 multi-worker subfolder
# race; prefetch the whole component set before any from_pretrained.
boogu_subfolders = ["scheduler", "vae", "mllm", "processor"]
prefetch_subfolders(model, boogu_subfolders, local_files_only=local_files_only)
prefetch_subfolders(
model,
boogu_subfolders,
local_files_only=local_files_only,
revision=od_config.revision,
)
self.scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
model, subfolder="scheduler", local_files_only=local_files_only
model,
subfolder="scheduler",
local_files_only=local_files_only,
revision=od_config.revision,
)
mllm = from_pretrained_with_prefetch(
@@ -246,6 +254,7 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
prefetch_list=boogu_subfolders,
local_files_only=local_files_only,
torch_dtype=od_config.dtype,
revision=od_config.revision,
)
# Upstream reuses the full VLM as an optional instruction rewriter and
# encodes with its inner model (no ``lm_head``); the rewriter is not
@@ -255,7 +264,10 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
self.mllm = mllm.to(self._execution_device)
self.processor = Qwen3VLProcessor.from_pretrained(
model, subfolder="processor", local_files_only=local_files_only
model,
subfolder="processor",
local_files_only=local_files_only,
revision=od_config.revision,
)
self.vae = from_pretrained_with_prefetch(
@@ -264,6 +276,7 @@ class BooguImagePipeline(nn.Module, ProgressBarMixin, SupportsComponentDiscovery
subfolder="vae",
prefetch_list=boogu_subfolders,
local_files_only=local_files_only,
revision=od_config.revision,
).to(self._execution_device)
self.transformer = BooguImageTransformer2DModel(od_config=od_config)
+15
View File
@@ -1,3 +1,6 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
import json
import os
import types
@@ -600,6 +603,18 @@ def load_and_resolve_stage_configs(
stage_configs = []
stage_configs = filter_stages(config_path, stage_configs, kwargs)
# ``revision`` selects the model source for every stage. Keep the global
# CLI pin on each stage's engine args so diffusion config construction can
# pass it through to all component loaders. The registry-to-legacy-stage
# conversion currently drops this inherited vLLM EngineArgs field.
revision = (kwargs or {}).get("revision")
if revision is not None:
for stage_config in stage_configs:
engine_args = stage_config.get("engine_args") if hasattr(stage_config, "get") else None
if engine_args is not None and engine_args.get("revision") is None:
engine_args["revision"] = revision
logger.debug(f"stage_configs: {stage_configs}")
return config_path, stage_configs, omni_lb_policy