[Model] Add Krea 2 text-to-image diffusion model (#4730)
Signed-off-by: Abhinay1997 <devarintinagasaiabhinay@gmail.com>
This commit is contained in:
Naga Sai Abhinay Devarinti authored and GitHub committed
1 parent
b95905deb1
commit
6581e2afa9
14 files changed
+1666
No files matched your search
@@ -650,6 +650,8 @@ steps:
|
||||
tests/e2e/online_serving/test_qwen_image_edit_expansion.py
|
||||
tests/e2e/online_serving/test_qwen_image_layered_expansion.py
|
||||
tests/e2e/online_serving/test_bagel_expansion.py
|
||||
tests/e2e/online_serving/test_krea2_expansion.py
|
||||
tests/e2e/offline_inference/test_krea2_expansion.py
|
||||
-m "full_model and diffusion and H100 and not distributed_cuda"
|
||||
--run-level "full_model"
|
||||
agents:
|
||||
@@ -693,6 +695,7 @@ steps:
|
||||
pytest -sv
|
||||
tests/e2e/online_serving/test_qwen_image_expansion.py
|
||||
tests/e2e/online_serving/test_qwen_image_edit_expansion.py
|
||||
tests/e2e/offline_inference/test_krea2_expansion.py
|
||||
-m "full_model and diffusion and H100 and distributed_cuda"
|
||||
--run-level "full_model"
|
||||
agents:
|
||||
|
||||
@@ -34,6 +34,7 @@ th {
|
||||
| `QwenImageEditPlusPipeline` | Qwen-Image-Edit-2511 | `Qwen/Qwen-Image-Edit-2511` | ✅︎ | ✅︎ | ✅︎ | ✅︎ |
|
||||
| `GlmImagePipeline` | GLM-Image | `zai-org/GLM-Image` | ✅︎ | ✅︎ | | |
|
||||
| `ZImagePipeline` | Z-Image | `Tongyi-MAI/Z-Image-Turbo` | ✅︎ | ✅︎ | ✅︎ | ✅︎ |
|
||||
| `Krea2Pipeline` | Krea 2 (Raw + Turbo) | `krea/Krea-2-Raw`, `krea/Krea-2-Turbo` | ✅︎ | | | |
|
||||
| `WanPipeline` | Wan2.1-T2V, Wan2.2-T2V, Wan2.2-TI2V | `Wan-AI/Wan2.1-T2V-1.3B-Diffusers`, `Wan-AI/Wan2.1-T2V-14B-Diffusers`, `Wan-AI/Wan2.2-T2V-A14B-Diffusers`, `Wan-AI/Wan2.2-TI2V-5B-Diffusers` | ✅︎ | ✅︎ | ✅︎ | ✅︎ |
|
||||
| `WanImageToVideoPipeline` | Wan2.2-I2V | `Wan-AI/Wan2.2-I2V-A14B-Diffusers` | ✅︎ | ✅︎ | ✅︎ | ✅︎ |
|
||||
| `Cosmos3OmniDiffusersPipeline` | Cosmos3 T2I, T2V, I2V, V2V, T2V with sound, action policy | `nvidia/Cosmos3-Nano`, `nvidia/Cosmos3-Super` | ✅︎ | ✅︎ | ✅︎ | ✅︎ |
|
||||
|
||||
@@ -121,6 +121,7 @@ The following tables show which models support each feature:
|
||||
| **GLM-Image** | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
|
||||
| **Hidream-I1-Full** | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
||||
| **HunyuanImage3** | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ |
|
||||
| **Krea 2** | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ (decode) | ❌ | ❌ |
|
||||
| **LongCat-Image** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
|
||||
| **LongCat-Image-Edit** | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
|
||||
| **MagiHuman** | ❌ | ❌ | ❌ | ❓ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
|
||||
@@ -144,6 +145,7 @@ The following tables show which models support each feature:
|
||||
> 1. Nextstep_1(T2I) does not support cache acceleration methods such as TeaCache or Cache-DiT.
|
||||
> 2. `Tongyi-MAI/Z-Image-Turbo` and `SII-GAIR/daVinci-MagiHuman-Base-1080p` are distilled models with minimal NFEs; CFG-Parallel is not necessary.
|
||||
> 3. Cosmos3 T2I uses `Cosmos3OmniDiffusersPipeline` with `modalities=["image"]`. Model-level CPU offload is not supported; use layerwise offload.
|
||||
> 4. Krea 2 currently supports single-GPU inference plus LoRA, Cache-DiT, HSDP, CPU/layerwise offload, and VAE-patch-parallel (decode). TP/SP/CFG-Parallel are not yet wired. The few-step distilled (Turbo) checkpoint uses `is_distilled=true` (fixed timestep shift `mu=1.15`); generate at 2048x2048 by default with `num_inference_steps≈8` and `guidance_scale=0`. The Raw checkpoint uses 1024x1024, `num_inference_steps=28`, and `guidance_scale=4.5`.
|
||||
|
||||
### VideoGen
|
||||
|
||||
|
||||
@@ -36,6 +36,7 @@ This folder provides several entrypoints for experimenting with text-to-image di
|
||||
| `black-forest-labs/FLUX.2-dev` | 1024 x 1024 | 65.7 | >80 (CPU offload required) |
|
||||
| `HunyuanImage-3.0` | 1024 x 1024 | 80.0 (TP≥3) | 160 |
|
||||
| `HiDream-I1-Full` | 1024 x 1024 | 63.7 | 57.7 |
|
||||
| `krea/Krea-2-Raw`, `krea/Krea-2-Turbo` | 1024 (Raw) / 2048 (Turbo) | — | ~30 |
|
||||
|
||||
!!! info
|
||||
*Peak VRAM: based on basic single-card usage, batch size =1, without any acceleration/optimization features. FLUX.2-dev requires `--enable-cpu-offload` on a single 80 GiB GPU.
|
||||
@@ -196,6 +197,37 @@ python examples/offline_inference/text_to_image/text_to_image.py \
|
||||
--output /output.png
|
||||
```
|
||||
|
||||
### Krea 2
|
||||
|
||||
Krea 2 is published as diffusers-format checkpoints: `krea/Krea-2-Turbo` (few-step distilled) and `krea/Krea-2-Raw`.
|
||||
The pipeline class (`Krea2Pipeline`) is auto-detected from `model_index.json`. The distilled Turbo checkpoint sets
|
||||
`is_distilled=true` and is best run at 2048x2048 with few steps and guidance disabled:
|
||||
|
||||
```bash
|
||||
# Few-step distilled (Turbo) checkpoint
|
||||
python examples/offline_inference/text_to_image/text_to_image.py \
|
||||
--model krea/Krea-2-Turbo \
|
||||
--prompt "a fox in the snow" \
|
||||
--num-inference-steps 8 \
|
||||
--guidance-scale 0.0 \
|
||||
--height 2048 --width 2048 \
|
||||
--output krea2.png
|
||||
```
|
||||
|
||||
For the Raw checkpoint, use more steps with CFG enabled:
|
||||
|
||||
```bash
|
||||
python examples/offline_inference/text_to_image/text_to_image.py \
|
||||
--model krea/Krea-2-Raw \
|
||||
--prompt "a fox in the snow" \
|
||||
--num-inference-steps 28 \
|
||||
--guidance-scale 4.5 \
|
||||
--output krea2.png
|
||||
```
|
||||
|
||||
`guidance-scale` follows the Krea 2 convention (`velocity = cond + guidance_scale * (cond - uncond)`); CFG is enabled
|
||||
whenever `guidance-scale > 0`.
|
||||
|
||||
### Batch Requests (Multiple Prompts)
|
||||
|
||||
You can pass multiple prompts in a single `Omni.generate` call. `Omni`
|
||||
|
||||
@@ -44,6 +44,7 @@ recipes/
|
||||
| [`inclusionAI/Ming-flash-omni-2.0.md`](./inclusionAI/Ming-flash-omni-2.0.md) | Online serving for multimodal chat + standalone TTS | 4x H100 / 1x H100 80GB |
|
||||
| [`inclusionAI/Ming-omni-tts.md`](./inclusionAI/Ming-omni-tts.md) | Offline + online dense Ming TTS/audio generation | 1x H100 80GB / 1x AMD MI300X (ROCm 7.2) |
|
||||
| [`IndexTeam/IndexTTS-2.md`](./IndexTeam/IndexTTS-2.md) | Online serving for voice-cloned TTS with optional emotion control | 1x L4 24GB or larger CUDA GPU |
|
||||
| [`krea/Krea-2.md`](./krea/Krea-2.md) | Text-to-image (Turbo + Raw), offline + online, with LoRA | 1x H100 80GB |
|
||||
| [`LTX/LTX-2.md`](./LTX/LTX-2.md) | Text-to-video and image-to-video serving | 1x H200 141GB |
|
||||
| [`LTX/LTX-2.3.md`](./LTX/LTX-2.3.md) | Text-to-video with audio generation (22B) | 1x GPU (96GB VRAM) |
|
||||
| [`MammothModa2/MammothModa2-Preview.md`](./MammothModa2/MammothModa2-Preview.md) | Text-to-image with the shared offline image example (AR → DiT) | 1x L40S 48GB / 1x ≥40GB GPU |
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
# Krea 2 for text-to-image (base + LoRA)
|
||||
|
||||
## Summary
|
||||
|
||||
- Vendor: Krea
|
||||
- Model: `krea/Krea-2-Turbo` (few-step distilled) and `krea/Krea-2-Raw`
|
||||
- Task: Text-to-image (T2I)
|
||||
- Mode: Offline engine and online OpenAI-compatible serving
|
||||
- Maintainer: Community
|
||||
|
||||
## When to use this recipe
|
||||
|
||||
Use this recipe for a known-good starting point to run the Krea 2 single-stream
|
||||
MMDiT with vLLM-Omni on a **single 80 GB NVIDIA H100** — both the offline
|
||||
`text_to_image.py` entrypoint and the online `/v1/images/generations` route,
|
||||
including PEFT LoRA adapters. `Krea2Pipeline` is auto-detected from
|
||||
`model_index.json`; guidance is checkpoint-aware (distilled Turbo runs no-CFG,
|
||||
Raw runs CFG).
|
||||
|
||||
## References
|
||||
|
||||
- Related example under `examples/`:
|
||||
[`examples/offline_inference/text_to_image/README.md`](../../examples/offline_inference/text_to_image/README.md)
|
||||
(see the "Krea 2" section)
|
||||
- Feature support:
|
||||
[`docs/user_guide/diffusion_features.md`](../../docs/user_guide/diffusion_features.md)
|
||||
- Related PR: [#4730](https://github.com/vllm-project/vllm-omni/pull/4730)
|
||||
|
||||
## Hardware Support
|
||||
|
||||
This recipe documents a **single-GPU** CUDA layout on H100 80 GB. Add more
|
||||
platforms (ROCm / NPU) or multi-GPU layouts as community validation lands.
|
||||
|
||||
## GPU
|
||||
|
||||
### 1× H100 80GB
|
||||
|
||||
#### Environment
|
||||
|
||||
Versions from a working editable install (activate `vllm-omni/.venv` or your equivalent):
|
||||
|
||||
- OS: Linux
|
||||
- Python: 3.12
|
||||
- Driver / runtime: NVIDIA driver **595.71.05** (CUDA 13.x)
|
||||
- torch: **2.11.0+cu130**
|
||||
- vLLM: **0.24.0**
|
||||
- vLLM-Omni: editable install from this repo (Git **`960ce8f6`**)
|
||||
- Transformers: **5.12.1**
|
||||
|
||||
Krea 2 weights are ~30 GB and fit comfortably on a single 80 GB H100.
|
||||
|
||||
#### Command
|
||||
|
||||
Offline — few-step distilled (Turbo) checkpoint (2048×2048, no CFG):
|
||||
|
||||
```bash
|
||||
python examples/offline_inference/text_to_image/text_to_image.py \
|
||||
--model krea/Krea-2-Turbo \
|
||||
--prompt "a fox in the snow" \
|
||||
--num-inference-steps 8 \
|
||||
--guidance-scale 0.0 \
|
||||
--height 2048 --width 2048 \
|
||||
--output krea2_turbo.png
|
||||
```
|
||||
|
||||
Offline — Raw checkpoint (1024×1024, CFG enabled):
|
||||
|
||||
```bash
|
||||
python examples/offline_inference/text_to_image/text_to_image.py \
|
||||
--model krea/Krea-2-Raw \
|
||||
--prompt "a fox in the snow" \
|
||||
--num-inference-steps 28 \
|
||||
--guidance-scale 4.5 \
|
||||
--height 1024 --width 1024 \
|
||||
--output krea2_raw.png
|
||||
```
|
||||
|
||||
LoRA (offline) — pass a PEFT adapter with `--lora-path` / `--lora-scale`.
|
||||
`NagaSaiAbhinay/Krea-2-vllm-darkbrush-LoRA` is a vLLM-Omni-compatible repackaging
|
||||
of `krea/Krea-2-LoRA-darkbrush`; its trigger word is `monochrome ink wash style`:
|
||||
|
||||
```bash
|
||||
python examples/offline_inference/text_to_image/text_to_image.py \
|
||||
--model krea/Krea-2-Turbo \
|
||||
--prompt "a fox in the snow, monochrome ink wash style" \
|
||||
--num-inference-steps 8 --guidance-scale 0.0 \
|
||||
--lora-path NagaSaiAbhinay/Krea-2-vllm-darkbrush-LoRA --lora-scale 1.0 \
|
||||
--output krea2_darkbrush.png
|
||||
```
|
||||
|
||||
Online — serve and query the OpenAI-compatible image route:
|
||||
|
||||
```bash
|
||||
vllm serve krea/Krea-2-Turbo --omni --enforce-eager --port 8091
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8091/v1/images/generations \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"prompt": "a fox in the snow, monochrome ink wash style",
|
||||
"size": "1024x1024",
|
||||
"num_inference_steps": 8,
|
||||
"guidance_scale": 0.0,
|
||||
"seed": 42,
|
||||
"lora": {"name": "darkbrush", "path": "NagaSaiAbhinay/Krea-2-vllm-darkbrush-LoRA", "scale": 1.0}
|
||||
}' | jq -r '.data[0].b64_json' | base64 -d > krea2_online.png
|
||||
```
|
||||
|
||||
Cache-DiT acceleration (offline) — add `--cache-backend cache_dit`:
|
||||
|
||||
```bash
|
||||
python examples/offline_inference/text_to_image/text_to_image.py \
|
||||
--model krea/Krea-2-Turbo --prompt "a fox in the snow" \
|
||||
--num-inference-steps 8 --guidance-scale 0.0 \
|
||||
--cache-backend cache_dit --output krea2_cachedit.png
|
||||
```
|
||||
|
||||
#### Verification
|
||||
|
||||
```bash
|
||||
# Offline: the script prints "Saved generated image to <path>"; confirm the file exists.
|
||||
ls -lh krea2_turbo.png
|
||||
```
|
||||
|
||||
For the online route, decode the returned `b64_json` (see the `curl` above) and
|
||||
confirm a valid 1024×1024 PNG is written.
|
||||
|
||||
#### Notes
|
||||
|
||||
- Key flags: distilled Turbo → few steps (~8) with `guidance_scale=0.0`; Raw →
|
||||
more steps (~28) with `guidance_scale>0` (CFG). `guidance-scale` follows the
|
||||
Krea 2 convention `velocity = cond + guidance_scale * (cond - uncond)`.
|
||||
- LoRA: adapters must be PEFT format (`adapter_config.json` +
|
||||
`adapter_model.safetensors`); the projections are `ReplicatedLinear`
|
||||
(264 modules). Offline uses `--lora-path` (which sets
|
||||
`OmniDiffusionSamplingParams.lora_request` / `lora_scale`); online uses the
|
||||
top-level `lora` object. Style LoRAs like darkbrush need their trigger word in
|
||||
the prompt.
|
||||
- Cache-DiT is supported; `has_separate_cfg` is checkpoint-aware (False for Turbo
|
||||
no-CFG, True for Raw CFG).
|
||||
- Known limitations: TP / SP / CFG-Parallel are not yet wired for Krea 2. HSDP,
|
||||
layerwise CPU offload, and VAE-patch-parallel (decode) are supported.
|
||||
@@ -0,0 +1,155 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
|
||||
"""L4 functionality tests for the Krea 2 text-to-image diffusion pipeline (``Krea2Pipeline``).
|
||||
|
||||
Runs against the public few-step distilled checkpoint ``krea/Krea-2-Turbo`` by default (override with the
|
||||
``KREA2_MODEL`` environment variable, e.g. ``krea/Krea-2-Raw`` for the Raw checkpoint or a local diffusers
|
||||
directory). They cover a basic functional smoke plus the layerwise-CPU-offload path that the pipeline declares
|
||||
via ``SupportsComponentDiscovery`` / ``_layerwise_offload_blocks_attrs``.
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from tests.helpers.mark import hardware_test
|
||||
from tests.helpers.runtime import OmniRunnerHandler
|
||||
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
|
||||
from vllm_omni.lora.request import LoRARequest
|
||||
from vllm_omni.lora.utils import stable_lora_int_id
|
||||
|
||||
MODEL = os.environ.get("KREA2_MODEL", "krea/Krea-2-Turbo")
|
||||
# vLLM-Omni-compatible PEFT repackaging of krea/Krea-2-LoRA-darkbrush (264 modules, r=alpha=32).
|
||||
LORA = os.environ.get("KREA2_LORA", "NagaSaiAbhinay/Krea-2-vllm-darkbrush-LoRA")
|
||||
PROMPT = "a fox in the snow, photorealistic"
|
||||
# darkbrush is a trigger-word LoRA: the trained monochrome ink-wash style only manifests when
|
||||
# its trigger phrase is present, so the LoRA test prompt appends it for a strong, stable signal.
|
||||
LORA_TRIGGER = "monochrome ink wash style"
|
||||
LORA_PROMPT = f"a fox in the snow, {LORA_TRIGGER}"
|
||||
|
||||
pytestmark = [
|
||||
pytest.mark.diffusion,
|
||||
pytest.mark.full_model,
|
||||
]
|
||||
|
||||
|
||||
def _sampling() -> OmniDiffusionSamplingParams:
|
||||
# Small resolution + few steps to keep the L4 case light. guidance_scale resolution is checkpoint-aware inside
|
||||
# the pipeline (distilled -> no-CFG, Raw -> CFG), so this stays agnostic to which checkpoint KREA2_MODEL points at.
|
||||
return OmniDiffusionSamplingParams(
|
||||
height=512,
|
||||
width=512,
|
||||
num_inference_steps=8,
|
||||
guidance_scale=0.0,
|
||||
seed=42,
|
||||
)
|
||||
|
||||
|
||||
@hardware_test(res={"cuda": "H100"})
|
||||
@pytest.mark.parametrize("omni_runner", [(MODEL, None)], indirect=True)
|
||||
def test_krea2_text_to_image_001(omni_runner_handler: OmniRunnerHandler) -> None:
|
||||
"""Basic functional smoke: a single prompt produces a decoded image."""
|
||||
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
|
||||
|
||||
|
||||
@hardware_test(res={"cuda": "H100"})
|
||||
@pytest.mark.parametrize(
|
||||
"omni_runner",
|
||||
[(MODEL, None, {"enable_layerwise_offload": True})],
|
||||
indirect=True,
|
||||
)
|
||||
def test_krea2_layerwise_offload(omni_runner_handler: OmniRunnerHandler) -> None:
|
||||
"""Exercise layerwise CPU offload on the DiT (SupportsComponentDiscovery + _layerwise_offload_blocks_attrs)."""
|
||||
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
|
||||
|
||||
|
||||
@hardware_test(res={"cuda": "H100"})
|
||||
@pytest.mark.cache
|
||||
@pytest.mark.parametrize("omni_runner", [(MODEL, None, {"cache_backend": "cache_dit"})], indirect=True)
|
||||
def test_krea2_cache_dit(omni_runner_handler: OmniRunnerHandler) -> None:
|
||||
"""Exercise Cache-DiT on Krea 2 via the custom Krea2Pipeline enabler (``enable_cache_for_krea2``).
|
||||
|
||||
Validates the docs' Cache-DiT support claim for Krea 2. ``has_separate_cfg`` is checkpoint-aware
|
||||
(False for the distilled Turbo no-CFG path, True for the Raw CFG path); the default Turbo checkpoint
|
||||
exercises the no-CFG branch.
|
||||
"""
|
||||
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
|
||||
|
||||
|
||||
@hardware_test(res={"cuda": "H100"}, num_cards=2)
|
||||
@pytest.mark.parametrize(
|
||||
"omni_runner",
|
||||
# Standalone HSDP: use_hsdp=True with an explicit hsdp_shard_size shards the DiT weights
|
||||
# across 2 ranks (replicate_size defaults to 1 -> world_size = 1 * 2 = 2). HSDP must keep
|
||||
# tensor_parallel_size/data_parallel_size at 1 (see vllm_omni/diffusion/data.py __post_init__).
|
||||
# Also validated locally at 8 GPUs (hsdp_shard_size=8); CI shard provisions 2.
|
||||
[(MODEL, None, {"use_hsdp": True, "hsdp_shard_size": 2})],
|
||||
indirect=True,
|
||||
)
|
||||
def test_krea2_hsdp(omni_runner_handler: OmniRunnerHandler) -> None:
|
||||
"""Exercise Hybrid Sharded Data Parallel (HSDP) weight sharding on the DiT across 2 GPUs.
|
||||
|
||||
Validates the docs' HSDP-support claim for Krea 2. Scheduled on the multi-GPU (distributed_cuda)
|
||||
shard by ``num_cards=2``; skipped automatically on boxes with fewer than 2 CUDA devices.
|
||||
"""
|
||||
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
|
||||
|
||||
|
||||
@hardware_test(res={"cuda": "H100"}, num_cards=2)
|
||||
@pytest.mark.parametrize(
|
||||
"omni_runner",
|
||||
# VAE patch-parallel decode: partition the VAE decode across the TP world. Mirrors the online
|
||||
# "tp_vae_patch" case (tensor_parallel_size=2 + vae_patch_parallel_size=2) from the wan2.2/z-image
|
||||
# expansion suites, so vae_patch_parallel_size must equal the world size established by TP.
|
||||
# Also validated locally at 8 GPUs (tp=8, vae_patch=8); CI shard provisions 2.
|
||||
[(MODEL, None, {"tensor_parallel_size": 2, "vae_patch_parallel_size": 2})],
|
||||
indirect=True,
|
||||
)
|
||||
def test_krea2_vae_patch_parallel(omni_runner_handler: OmniRunnerHandler) -> None:
|
||||
"""Exercise VAE patch-parallel (decode) across 2 GPUs.
|
||||
|
||||
Validates the docs' VAE-patch-parallel (decode) support claim for Krea 2. Scheduled on the
|
||||
multi-GPU (distributed_cuda) shard by ``num_cards=2``; skipped automatically on boxes with
|
||||
fewer than 2 CUDA devices.
|
||||
"""
|
||||
omni_runner_handler.send_diffusion_request({"model": MODEL, "prompt": PROMPT, "sampling_params": _sampling()})
|
||||
|
||||
|
||||
def _generate(handler: OmniRunnerHandler, lora_request: LoRARequest | None, lora_scale: float = 1.0) -> np.ndarray:
|
||||
sp = _sampling()
|
||||
sp.lora_request = lora_request
|
||||
sp.lora_scale = lora_scale
|
||||
resp = handler.send_diffusion_request({"model": MODEL, "prompt": LORA_PROMPT, "sampling_params": sp})
|
||||
return np.asarray(resp.images[0].convert("RGB"), dtype=np.int16)
|
||||
|
||||
|
||||
@hardware_test(res={"cuda": "H100"})
|
||||
@pytest.mark.parametrize("omni_runner", [(MODEL, None)], indirect=True)
|
||||
def test_krea2_lora(omni_runner_handler: OmniRunnerHandler) -> None:
|
||||
"""Validate diffusion LoRA on Krea 2: visible effect, scale sensitivity, clean deactivation.
|
||||
|
||||
Uses ``LORA`` (a vLLM-Omni-compatible PEFT repackaging of ``krea/Krea-2-LoRA-darkbrush``,
|
||||
264 modules matched via the ReplicatedLinear projections) passed per-request through
|
||||
``OmniDiffusionSamplingParams.lora_request`` / ``lora_scale`` — the same fields the
|
||||
diffusion LoRA manager reads to activate/deactivate the adapter.
|
||||
"""
|
||||
lora_request = LoRARequest(lora_name="darkbrush", lora_int_id=stable_lora_int_id(LORA), lora_path=LORA)
|
||||
|
||||
baseline = _generate(omni_runner_handler, None)
|
||||
img_1x = _generate(omni_runner_handler, lora_request, lora_scale=1.0)
|
||||
img_2x = _generate(omni_runner_handler, lora_request, lora_scale=2.0)
|
||||
restored = _generate(omni_runner_handler, None)
|
||||
|
||||
diff_1x = np.abs(baseline - img_1x).mean()
|
||||
diff_2x = np.abs(baseline - img_2x).mean()
|
||||
diff_restored = np.abs(baseline - restored).mean()
|
||||
|
||||
# (a) Adapter has a visible effect at both scales.
|
||||
assert diff_1x > 0.5, f"LoRA scale=1.0 had no visible effect: diff={diff_1x}"
|
||||
assert diff_2x > 0.5, f"LoRA scale=2.0 had no visible effect: diff={diff_2x}"
|
||||
# (b) Scale changes the output (stronger adapter -> larger perturbation).
|
||||
assert not np.isclose(diff_1x, diff_2x, atol=1.0), f"LoRA scale had no effect: 1x={diff_1x:.2f}, 2x={diff_2x:.2f}"
|
||||
# (c) Passing lora_request=None cleanly deactivates: same seed -> back to baseline (modulo fp drift).
|
||||
assert diff_restored < 5.0, f"LoRA did not deactivate cleanly: diff_restored={diff_restored:.2f}"
|
||||
@@ -0,0 +1,152 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
|
||||
"""L4 online-serving tests for the Krea 2 text-to-image diffusion pipeline.
|
||||
|
||||
Exercises the OpenAI-compatible ``/v1/images/generations`` route (served via
|
||||
``vllm serve <model> --omni --enforce-eager``) rather than the offline engine path
|
||||
covered by ``tests/e2e/offline_inference/test_krea2_expansion.py``.
|
||||
|
||||
Runs against the public few-step distilled checkpoint ``krea/Krea-2-Turbo`` by default
|
||||
(override with the ``KREA2_MODEL`` environment variable, e.g. ``krea/Krea-2-Raw`` for the
|
||||
Raw checkpoint or a local diffusers directory). Krea 2 shares the Qwen-Image VAE and a
|
||||
similar single-stream MMDiT, so this test mirrors the structure/fixtures/markers of
|
||||
``test_qwen_image_expansion.py``.
|
||||
|
||||
Coverage mirrors the offline suite: a basic functional smoke plus the layerwise-CPU-offload
|
||||
path that the pipeline declares via ``SupportsComponentDiscovery`` /
|
||||
``_layerwise_offload_blocks_attrs``. Sizing is kept light (512x512, few steps) so the case
|
||||
fits single-GPU H100 CI rather than the full 2048x2048 distilled resolution.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import os
|
||||
from io import BytesIO
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
import requests
|
||||
from PIL import Image
|
||||
|
||||
from tests.helpers.mark import hardware_marks
|
||||
from tests.helpers.runtime import OmniServer, OmniServerParams, OpenAIClientHandler, dummy_messages_from_mix_data
|
||||
|
||||
pytestmark = [pytest.mark.diffusion, pytest.mark.full_model]
|
||||
|
||||
MODEL = os.environ.get("KREA2_MODEL", "krea/Krea-2-Turbo")
|
||||
# vLLM-Omni-compatible PEFT repackaging of krea/Krea-2-LoRA-darkbrush (264 modules, r=alpha=32).
|
||||
LORA = os.environ.get("KREA2_LORA", "NagaSaiAbhinay/Krea-2-vllm-darkbrush-LoRA")
|
||||
PROMPT = "a fox in the snow, photorealistic"
|
||||
# darkbrush is a trigger-word LoRA; include its trigger phrase so the LoRA case has a strong signal.
|
||||
LORA_PROMPT = "a fox in the snow, monochrome ink wash style"
|
||||
|
||||
SINGLE_CARD_FEATURE_MARKS = hardware_marks(res={"cuda": "H100"})
|
||||
|
||||
|
||||
def _get_diffusion_feature_cases(model: str):
|
||||
"""Return single-card L4 online-serving cases for Krea 2.
|
||||
|
||||
Mirrors the offline suite: a basic functional smoke plus layerwise CPU offload.
|
||||
"""
|
||||
return [
|
||||
# Basic functional smoke (single-card).
|
||||
pytest.param(
|
||||
OmniServerParams(
|
||||
model=model,
|
||||
server_args=["--enforce-eager"],
|
||||
),
|
||||
id="single_card_basic",
|
||||
marks=SINGLE_CARD_FEATURE_MARKS,
|
||||
),
|
||||
# Layerwise CPU offload (single-card): SupportsComponentDiscovery +
|
||||
# _layerwise_offload_blocks_attrs.
|
||||
pytest.param(
|
||||
OmniServerParams(
|
||||
model=model,
|
||||
server_args=["--enforce-eager", "--enable-layerwise-offload"],
|
||||
),
|
||||
id="single_card_layerwise_offload",
|
||||
marks=SINGLE_CARD_FEATURE_MARKS,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"omni_server",
|
||||
_get_diffusion_feature_cases(MODEL),
|
||||
indirect=True,
|
||||
)
|
||||
def test_krea2(omni_server: OmniServer, openai_client: OpenAIClientHandler):
|
||||
"""Serve Krea 2 and validate the OpenAI-compatible image-generation route.
|
||||
|
||||
guidance_scale resolution is checkpoint-aware inside the pipeline (distilled -> no-CFG,
|
||||
Raw -> CFG), so guidance_scale=0.0 stays agnostic to which checkpoint KREA2_MODEL points at.
|
||||
Validation is delegated to assert_diffusion_response, which checks output dimensions and
|
||||
basic correctness.
|
||||
"""
|
||||
messages = dummy_messages_from_mix_data(content_text=PROMPT)
|
||||
request_config = {
|
||||
"model": omni_server.model,
|
||||
"messages": messages,
|
||||
"extra_body": {
|
||||
"height": 512,
|
||||
"width": 512,
|
||||
"num_inference_steps": 8,
|
||||
"guidance_scale": 0.0,
|
||||
"seed": 42,
|
||||
},
|
||||
}
|
||||
openai_client.send_diffusion_request(request_config)
|
||||
|
||||
|
||||
def _basic_lora_payload(lora: dict | None = None) -> dict:
|
||||
payload = {
|
||||
"prompt": LORA_PROMPT,
|
||||
"n": 1,
|
||||
"size": "512x512",
|
||||
"num_inference_steps": 8,
|
||||
"guidance_scale": 0.0,
|
||||
"seed": 42,
|
||||
}
|
||||
if lora is not None:
|
||||
payload["lora"] = lora
|
||||
return payload
|
||||
|
||||
|
||||
def _post_image(server: OmniServer, payload: dict) -> np.ndarray:
|
||||
url = f"http://{server.host}:{server.port}/v1/images/generations"
|
||||
resp = requests.post(url, json=payload, headers={"Authorization": "Bearer EMPTY"}, timeout=900)
|
||||
resp.raise_for_status()
|
||||
b64 = resp.json()["data"][0]["b64_json"]
|
||||
img = Image.open(BytesIO(base64.b64decode(b64)))
|
||||
img.load()
|
||||
return np.asarray(img.convert("RGB"), dtype=np.int16)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"omni_server",
|
||||
[
|
||||
pytest.param(
|
||||
OmniServerParams(model=MODEL, server_args=["--enforce-eager"]),
|
||||
id="single_card_lora",
|
||||
marks=SINGLE_CARD_FEATURE_MARKS,
|
||||
)
|
||||
],
|
||||
indirect=True,
|
||||
)
|
||||
def test_krea2_lora(omni_server: OmniServer):
|
||||
"""Serve Krea 2 and validate per-request LoRA over ``/v1/images/generations``.
|
||||
|
||||
Passes ``LORA`` via the top-level ``lora`` object (``{name, path, scale}``) the diffusion
|
||||
image route parses into a ``LoRARequest`` — no server-side LoRA flag needed. Confirms the
|
||||
adapter has a visible effect and that omitting ``lora`` cleanly restores the base output.
|
||||
"""
|
||||
baseline = _post_image(omni_server, _basic_lora_payload())
|
||||
with_lora = _post_image(omni_server, _basic_lora_payload({"name": "darkbrush", "path": LORA, "scale": 1.0}))
|
||||
restored = _post_image(omni_server, _basic_lora_payload())
|
||||
|
||||
diff_lora = np.abs(baseline - with_lora).mean()
|
||||
diff_restored = np.abs(baseline - restored).mean()
|
||||
|
||||
assert diff_lora > 0.5, f"LoRA had no visible effect over the serving path: diff={diff_lora}"
|
||||
assert diff_restored < 5.0, f"LoRA did not deactivate cleanly: diff_restored={diff_restored:.2f}"
|
||||
@@ -56,6 +56,7 @@ EXCLUDED_MODELS = [
|
||||
"NextStep11Pipeline",
|
||||
"FluxPipeline",
|
||||
"FluxDMD2Pipeline",
|
||||
"Krea2Pipeline",
|
||||
"QwenImageDMD2Pipeline",
|
||||
"OmniGen2Pipeline",
|
||||
"HeliosPipeline",
|
||||
|
||||
+33
@@ -975,6 +975,38 @@ def enable_cache_for_cosmos3(pipeline: Any, cache_config: Any) -> RefreshCacheCo
|
||||
return enable_cache_for_dit(pipeline, cache_config, block_adapter)
|
||||
|
||||
|
||||
def enable_cache_for_krea2(pipeline: Any, cache_config: Any) -> RefreshCacheContextFunc:
|
||||
"""Enable cache-dit for Krea 2.
|
||||
|
||||
Krea 2 is a single-stream MMDiT: each ``Krea2TransformerBlock`` takes and returns only
|
||||
``hidden_states`` (text is fused into the token stream), so the blocks follow
|
||||
``ForwardPattern.Pattern_3``.
|
||||
|
||||
``has_separate_cfg`` is checkpoint-dependent, which is why this needs a custom enabler
|
||||
rather than a static ``_cache_dit_adapter_config``: the distilled Turbo checkpoint runs
|
||||
no-CFG (a single transformer forward per denoise step), while the Raw checkpoint runs CFG
|
||||
as two separate forwards. cache-dit tells cond/uncond apart purely by transformer-forward
|
||||
parity, so the flag must match the actual per-step forward count — ``True`` only for the
|
||||
CFG (Raw) path. The pipeline exposes this via ``is_distilled`` (read from ``model_index.json``).
|
||||
|
||||
Args:
|
||||
pipeline: The Krea2Pipeline instance.
|
||||
cache_config: DiffusionCacheConfig instance with cache configuration.
|
||||
|
||||
Returns:
|
||||
A refresh function that can be called to update cache context with new num_inference_steps.
|
||||
"""
|
||||
transformer = default_get_pipeline_transformer(pipeline)
|
||||
block_adapter = BlockAdapter(
|
||||
transformer=transformer,
|
||||
blocks=[transformer.transformer_blocks],
|
||||
forward_pattern=[ForwardPattern.Pattern_3],
|
||||
has_separate_cfg=not pipeline.is_distilled,
|
||||
check_forward_pattern=False,
|
||||
)
|
||||
return enable_cache_for_dit(pipeline, cache_config, block_adapter)
|
||||
|
||||
|
||||
# Register custom cache-dit enablers after function definitions
|
||||
CUSTOM_DIT_ENABLERS.update(
|
||||
{
|
||||
@@ -984,6 +1016,7 @@ CUSTOM_DIT_ENABLERS.update(
|
||||
"Wan22VACEPipeline": enable_cache_for_wan22,
|
||||
"Wan22S2VPipeline": enable_cache_for_wan22_s2v,
|
||||
"Cosmos3OmniDiffusersPipeline": enable_cache_for_cosmos3,
|
||||
"Krea2Pipeline": enable_cache_for_krea2,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
"""Krea 2 diffusion model components."""
|
||||
|
||||
from vllm_omni.diffusion.models.krea2.krea2_transformer import Krea2Transformer2DModel
|
||||
from vllm_omni.diffusion.models.krea2.pipeline_krea2 import (
|
||||
Krea2Pipeline,
|
||||
get_krea2_post_process_func,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"Krea2Pipeline",
|
||||
"Krea2Transformer2DModel",
|
||||
"get_krea2_post_process_func",
|
||||
]
|
||||
@@ -0,0 +1,571 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
|
||||
# Copyright 2026 Krea AI and The HuggingFace Team. All rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import math
|
||||
from collections.abc import Iterable
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
from diffusers.models.embeddings import apply_rotary_emb, get_1d_rotary_pos_embed
|
||||
from vllm.logger import init_logger
|
||||
from vllm.model_executor.layers.linear import ReplicatedLinear
|
||||
from vllm.model_executor.model_loader.weight_utils import default_weight_loader
|
||||
|
||||
from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata
|
||||
from vllm_omni.diffusion.attention.layer import Attention
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from vllm.model_executor.layers.quantization.base_config import QuantizationConfig
|
||||
|
||||
from vllm_omni.diffusion.data import OmniDiffusionConfig
|
||||
|
||||
logger = init_logger(__name__)
|
||||
|
||||
# Devices that do not support float64; RoPE frequencies fall back to float32 there.
|
||||
_FP64_UNSUPPORTED_DEVICES = frozenset({"mps", "npu", "neuron"})
|
||||
|
||||
|
||||
def _rope_freqs_dtype(device: torch.device) -> torch.dtype:
|
||||
return torch.float32 if device.type in _FP64_UNSUPPORTED_DEVICES else torch.float64
|
||||
|
||||
|
||||
def _join_prefix(prefix: str, name: str) -> str:
|
||||
return f"{prefix}.{name}" if prefix else name
|
||||
|
||||
|
||||
def _linear(
|
||||
in_features: int,
|
||||
out_features: int,
|
||||
bias: bool,
|
||||
quant_config: "QuantizationConfig | None",
|
||||
prefix: str,
|
||||
) -> ReplicatedLinear:
|
||||
# ReplicatedLinear (not nn.Linear) so the diffusion LoRA manager can wrap these projections.
|
||||
return ReplicatedLinear(
|
||||
in_features,
|
||||
out_features,
|
||||
bias=bias,
|
||||
quant_config=quant_config,
|
||||
prefix=prefix,
|
||||
return_bias=False,
|
||||
)
|
||||
|
||||
|
||||
class Krea2RMSNorm(nn.Module):
|
||||
"""RMSNorm with a zero-centered scale: the effective multiplier is ``1 + weight``, matching the Krea 2 checkpoint
|
||||
format. Activations are upcast so the normalization runs in float32."""
|
||||
|
||||
def __init__(self, dim: int, eps: float = 1e-5) -> None:
|
||||
super().__init__()
|
||||
self.dim = dim
|
||||
self.eps = eps
|
||||
self.weight = nn.Parameter(torch.zeros(dim))
|
||||
|
||||
def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
|
||||
dtype = hidden_states.dtype
|
||||
hidden_states = F.rms_norm(hidden_states.float(), (self.dim,), weight=self.weight + 1.0, eps=self.eps)
|
||||
return hidden_states.to(dtype)
|
||||
|
||||
|
||||
class Krea2Attention(nn.Module):
|
||||
"""Self-attention with grouped-query projections, q/k RMSNorm, rotary embeddings and a sigmoid output gate.
|
||||
|
||||
Q/K/V layout is ``[B, seq, heads, head_dim]``; ``attention_mask`` is a 2D boolean key-padding mask
|
||||
``(batch, key_seq_len)`` that the backend broadcasts.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
hidden_size: int,
|
||||
num_heads: int,
|
||||
num_kv_heads: int | None = None,
|
||||
eps: float = 1e-5,
|
||||
role: str = "self",
|
||||
quant_config: "QuantizationConfig | None" = None,
|
||||
prefix: str = "",
|
||||
) -> None:
|
||||
super().__init__()
|
||||
if hidden_size % num_heads != 0:
|
||||
raise ValueError(f"hidden_size={hidden_size} must be divisible by num_heads={num_heads}")
|
||||
self.hidden_size = hidden_size
|
||||
self.num_heads = num_heads
|
||||
self.num_kv_heads = num_kv_heads if num_kv_heads is not None else num_heads
|
||||
self.head_dim = hidden_size // num_heads
|
||||
|
||||
q_dim = self.head_dim * self.num_heads
|
||||
kv_dim = self.head_dim * self.num_kv_heads
|
||||
self.to_q = _linear(hidden_size, q_dim, False, quant_config, _join_prefix(prefix, "to_q"))
|
||||
self.to_k = _linear(hidden_size, kv_dim, False, quant_config, _join_prefix(prefix, "to_k"))
|
||||
self.to_v = _linear(hidden_size, kv_dim, False, quant_config, _join_prefix(prefix, "to_v"))
|
||||
self.to_gate = _linear(hidden_size, hidden_size, False, quant_config, _join_prefix(prefix, "to_gate"))
|
||||
self.norm_q = Krea2RMSNorm(self.head_dim, eps=eps)
|
||||
self.norm_k = Krea2RMSNorm(self.head_dim, eps=eps)
|
||||
self.to_out = nn.ModuleList(
|
||||
[_linear(hidden_size, hidden_size, False, quant_config, _join_prefix(prefix, "to_out.0")), nn.Dropout(0.0)]
|
||||
)
|
||||
|
||||
self.attn = Attention(
|
||||
num_heads=self.num_heads,
|
||||
head_size=self.head_dim,
|
||||
softmax_scale=1.0 / (self.head_dim**0.5),
|
||||
causal=False,
|
||||
num_kv_heads=self.num_kv_heads,
|
||||
role=role,
|
||||
)
|
||||
|
||||
def forward(
|
||||
self,
|
||||
hidden_states: torch.Tensor,
|
||||
attention_mask: torch.Tensor | None = None,
|
||||
image_rotary_emb: tuple[torch.Tensor, torch.Tensor] | None = None,
|
||||
) -> torch.Tensor:
|
||||
query = self.to_q(hidden_states).unflatten(-1, (self.num_heads, self.head_dim))
|
||||
key = self.to_k(hidden_states).unflatten(-1, (self.num_kv_heads, self.head_dim))
|
||||
value = self.to_v(hidden_states).unflatten(-1, (self.num_kv_heads, self.head_dim))
|
||||
gate = self.to_gate(hidden_states)
|
||||
|
||||
query = self.norm_q(query)
|
||||
key = self.norm_k(key)
|
||||
|
||||
if image_rotary_emb is not None:
|
||||
query = apply_rotary_emb(query, image_rotary_emb, sequence_dim=1)
|
||||
key = apply_rotary_emb(key, image_rotary_emb, sequence_dim=1)
|
||||
query = query.to(value.dtype)
|
||||
key = key.to(value.dtype)
|
||||
|
||||
attn_metadata = AttentionMetadata(attn_mask=attention_mask) if attention_mask is not None else None
|
||||
hidden_states = self.attn(query, key, value, attn_metadata)
|
||||
hidden_states = hidden_states.flatten(2, 3)
|
||||
hidden_states = hidden_states * torch.sigmoid(gate)
|
||||
return self.to_out[0](hidden_states)
|
||||
|
||||
|
||||
class Krea2SwiGLU(nn.Module):
|
||||
"""SwiGLU feed-forward network."""
|
||||
|
||||
def __init__(
|
||||
self, dim: int, hidden_dim: int, quant_config: "QuantizationConfig | None" = None, prefix: str = ""
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self.gate = _linear(dim, hidden_dim, False, quant_config, _join_prefix(prefix, "gate"))
|
||||
self.up = _linear(dim, hidden_dim, False, quant_config, _join_prefix(prefix, "up"))
|
||||
self.down = _linear(hidden_dim, dim, False, quant_config, _join_prefix(prefix, "down"))
|
||||
|
||||
def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
|
||||
return self.down(F.silu(self.gate(hidden_states)) * self.up(hidden_states))
|
||||
|
||||
|
||||
class Krea2TextFusionBlock(nn.Module):
|
||||
"""Pre-norm transformer block (no rotary embeddings, no time modulation) used by the text fusion stage."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
dim: int,
|
||||
num_heads: int,
|
||||
num_kv_heads: int,
|
||||
intermediate_size: int,
|
||||
eps: float,
|
||||
quant_config: "QuantizationConfig | None" = None,
|
||||
prefix: str = "",
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self.norm1 = Krea2RMSNorm(dim, eps=eps)
|
||||
self.norm2 = Krea2RMSNorm(dim, eps=eps)
|
||||
self.attn = Krea2Attention(
|
||||
dim,
|
||||
num_heads,
|
||||
num_kv_heads,
|
||||
eps=eps,
|
||||
role="self",
|
||||
quant_config=quant_config,
|
||||
prefix=_join_prefix(prefix, "attn"),
|
||||
)
|
||||
self.ff = Krea2SwiGLU(dim, intermediate_size, quant_config=quant_config, prefix=_join_prefix(prefix, "ff"))
|
||||
|
||||
def forward(self, hidden_states: torch.Tensor, attention_mask: torch.Tensor | None = None) -> torch.Tensor:
|
||||
hidden_states = hidden_states + self.attn(self.norm1(hidden_states), attention_mask=attention_mask)
|
||||
hidden_states = hidden_states + self.ff(self.norm2(hidden_states))
|
||||
return hidden_states
|
||||
|
||||
|
||||
class Krea2TextFusion(nn.Module):
|
||||
"""Fuses the stack of tapped text-encoder hidden states into a single sequence of text features.
|
||||
|
||||
Two ``layerwise_blocks`` attend across the ``num_text_layers`` axis independently for every token, a linear
|
||||
``projector`` collapses that axis, and two ``refiner_blocks`` attend across the token sequence.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
num_text_layers: int,
|
||||
dim: int,
|
||||
num_heads: int,
|
||||
num_kv_heads: int,
|
||||
intermediate_size: int,
|
||||
num_layerwise_blocks: int,
|
||||
num_refiner_blocks: int,
|
||||
eps: float,
|
||||
quant_config: "QuantizationConfig | None" = None,
|
||||
prefix: str = "",
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self.layerwise_blocks = nn.ModuleList(
|
||||
[
|
||||
Krea2TextFusionBlock(
|
||||
dim,
|
||||
num_heads,
|
||||
num_kv_heads,
|
||||
intermediate_size,
|
||||
eps,
|
||||
quant_config=quant_config,
|
||||
prefix=_join_prefix(prefix, f"layerwise_blocks.{i}"),
|
||||
)
|
||||
for i in range(num_layerwise_blocks)
|
||||
]
|
||||
)
|
||||
self.projector = _linear(num_text_layers, 1, False, quant_config, _join_prefix(prefix, "projector"))
|
||||
self.refiner_blocks = nn.ModuleList(
|
||||
[
|
||||
Krea2TextFusionBlock(
|
||||
dim,
|
||||
num_heads,
|
||||
num_kv_heads,
|
||||
intermediate_size,
|
||||
eps,
|
||||
quant_config=quant_config,
|
||||
prefix=_join_prefix(prefix, f"refiner_blocks.{i}"),
|
||||
)
|
||||
for i in range(num_refiner_blocks)
|
||||
]
|
||||
)
|
||||
|
||||
def forward(self, encoder_hidden_states: torch.Tensor, attention_mask: torch.Tensor | None = None) -> torch.Tensor:
|
||||
batch_size, seq_len, num_text_layers, dim = encoder_hidden_states.shape
|
||||
|
||||
hidden_states = encoder_hidden_states.reshape(batch_size * seq_len, num_text_layers, dim)
|
||||
for block in self.layerwise_blocks:
|
||||
hidden_states = block(hidden_states.contiguous())
|
||||
|
||||
hidden_states = hidden_states.reshape(batch_size, seq_len, num_text_layers, dim).permute(0, 1, 3, 2)
|
||||
hidden_states = self.projector(hidden_states).squeeze(-1)
|
||||
|
||||
for block in self.refiner_blocks:
|
||||
hidden_states = block(hidden_states, attention_mask=attention_mask)
|
||||
|
||||
return hidden_states
|
||||
|
||||
|
||||
class Krea2TransformerBlock(nn.Module):
|
||||
def __init__(
|
||||
self,
|
||||
hidden_size: int,
|
||||
intermediate_size: int,
|
||||
num_heads: int,
|
||||
num_kv_heads: int,
|
||||
norm_eps: float,
|
||||
quant_config: "QuantizationConfig | None" = None,
|
||||
prefix: str = "",
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self.scale_shift_table = nn.Parameter(torch.zeros(6, hidden_size))
|
||||
self.norm1 = Krea2RMSNorm(hidden_size, eps=norm_eps)
|
||||
self.norm2 = Krea2RMSNorm(hidden_size, eps=norm_eps)
|
||||
self.attn = Krea2Attention(
|
||||
hidden_size,
|
||||
num_heads,
|
||||
num_kv_heads,
|
||||
eps=norm_eps,
|
||||
role="self",
|
||||
quant_config=quant_config,
|
||||
prefix=_join_prefix(prefix, "attn"),
|
||||
)
|
||||
self.ff = Krea2SwiGLU(
|
||||
hidden_size, intermediate_size, quant_config=quant_config, prefix=_join_prefix(prefix, "ff")
|
||||
)
|
||||
|
||||
def forward(
|
||||
self,
|
||||
hidden_states: torch.Tensor,
|
||||
temb: torch.Tensor,
|
||||
image_rotary_emb: tuple[torch.Tensor, torch.Tensor],
|
||||
attention_mask: torch.Tensor | None = None,
|
||||
) -> torch.Tensor:
|
||||
# temb: (B, 1, 6 * hidden_size), shared across all blocks; each block only learns an additive table.
|
||||
modulation = temb.unflatten(-1, (6, -1)) + self.scale_shift_table
|
||||
prescale, preshift, pregate, postscale, postshift, postgate = modulation.unbind(-2)
|
||||
|
||||
attn_out = self.attn(
|
||||
(1.0 + prescale) * self.norm1(hidden_states) + preshift,
|
||||
attention_mask=attention_mask,
|
||||
image_rotary_emb=image_rotary_emb,
|
||||
)
|
||||
hidden_states = hidden_states + pregate * attn_out
|
||||
ff_out = self.ff((1.0 + postscale) * self.norm2(hidden_states) + postshift)
|
||||
hidden_states = hidden_states + postgate * ff_out
|
||||
return hidden_states
|
||||
|
||||
|
||||
class Krea2TimestepEmbedding(nn.Module):
|
||||
"""Sinusoidal flow-time embedding (cos-first, input scaled by 1000) followed by a two-layer MLP.
|
||||
|
||||
Keeps the sequence dimension at size 1 so the per-block modulations broadcast over tokens.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self, embed_dim: int, hidden_size: int, quant_config: "QuantizationConfig | None" = None, prefix: str = ""
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self.embed_dim = embed_dim
|
||||
self.linear_1 = _linear(embed_dim, hidden_size, True, quant_config, _join_prefix(prefix, "linear_1"))
|
||||
self.linear_2 = _linear(hidden_size, hidden_size, True, quant_config, _join_prefix(prefix, "linear_2"))
|
||||
|
||||
def forward(self, timestep: torch.Tensor, dtype: torch.dtype) -> torch.Tensor:
|
||||
half = self.embed_dim // 2
|
||||
freqs = torch.exp(-math.log(1e4) * torch.arange(half, dtype=torch.float32, device=timestep.device) / half)
|
||||
args = (timestep.float() * 1e3)[:, None, None] * freqs
|
||||
emb = torch.cat([torch.cos(args), torch.sin(args)], dim=-1).to(dtype)
|
||||
return self.linear_2(F.gelu(self.linear_1(emb), approximate="tanh"))
|
||||
|
||||
|
||||
class Krea2TextProjection(nn.Module):
|
||||
"""Projects the fused text features into the transformer width."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
text_dim: int,
|
||||
hidden_size: int,
|
||||
eps: float,
|
||||
quant_config: "QuantizationConfig | None" = None,
|
||||
prefix: str = "",
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self.norm = Krea2RMSNorm(text_dim, eps=eps)
|
||||
self.linear_1 = _linear(text_dim, hidden_size, True, quant_config, _join_prefix(prefix, "linear_1"))
|
||||
self.linear_2 = _linear(hidden_size, hidden_size, True, quant_config, _join_prefix(prefix, "linear_2"))
|
||||
|
||||
def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
|
||||
hidden_states = self.linear_1(self.norm(hidden_states))
|
||||
return self.linear_2(F.gelu(hidden_states, approximate="tanh"))
|
||||
|
||||
|
||||
class Krea2FinalLayer(nn.Module):
|
||||
"""Final adaptive RMSNorm and output projection."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
hidden_size: int,
|
||||
out_channels: int,
|
||||
eps: float,
|
||||
quant_config: "QuantizationConfig | None" = None,
|
||||
prefix: str = "",
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self.scale_shift_table = nn.Parameter(torch.zeros(2, hidden_size))
|
||||
self.norm = Krea2RMSNorm(hidden_size, eps=eps)
|
||||
self.linear = _linear(hidden_size, out_channels, True, quant_config, _join_prefix(prefix, "linear"))
|
||||
|
||||
def forward(self, hidden_states: torch.Tensor, temb: torch.Tensor) -> torch.Tensor:
|
||||
modulation = temb + self.scale_shift_table
|
||||
scale, shift = modulation.chunk(2, dim=1)
|
||||
hidden_states = (1.0 + scale) * self.norm(hidden_states) + shift
|
||||
return self.linear(hidden_states)
|
||||
|
||||
|
||||
class Krea2RotaryPosEmbed(nn.Module):
|
||||
"""Multi-axis (t, h, w) rotary position embedding, following the Flux/Krea 2 convention."""
|
||||
|
||||
def __init__(self, theta: int, axes_dim: list[int]):
|
||||
super().__init__()
|
||||
self.theta = theta
|
||||
self.axes_dim = axes_dim
|
||||
|
||||
def forward(self, ids: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
|
||||
n_axes = ids.shape[-1]
|
||||
cos_out = []
|
||||
sin_out = []
|
||||
pos = ids.float()
|
||||
freqs_dtype = _rope_freqs_dtype(ids.device)
|
||||
for i in range(n_axes):
|
||||
cos, sin = get_1d_rotary_pos_embed(
|
||||
self.axes_dim[i],
|
||||
pos[:, i],
|
||||
theta=self.theta,
|
||||
repeat_interleave_real=True,
|
||||
use_real=True,
|
||||
freqs_dtype=freqs_dtype,
|
||||
)
|
||||
cos_out.append(cos)
|
||||
sin_out.append(sin)
|
||||
freqs_cos = torch.cat(cos_out, dim=-1).to(ids.device)
|
||||
freqs_sin = torch.cat(sin_out, dim=-1).to(ids.device)
|
||||
return freqs_cos, freqs_sin
|
||||
|
||||
|
||||
class Krea2Transformer2DModel(nn.Module):
|
||||
r"""The single-stream MMDiT flow-matching backbone used by the Krea 2 pipeline (vLLM-Omni port).
|
||||
|
||||
Text conditioning enters as a stack of hidden states tapped from several layers of a multimodal text encoder. A
|
||||
small text-fusion transformer collapses the layer axis and refines the token sequence; the result is concatenated
|
||||
with the patchified image latents into a single ``[text, image]`` sequence processed by the transformer blocks.
|
||||
The timestep conditions every block through one shared modulation vector plus per-block learned tables.
|
||||
"""
|
||||
|
||||
_repeated_blocks = ["Krea2TransformerBlock"]
|
||||
_layerwise_offload_blocks_attrs = ["transformer_blocks"]
|
||||
|
||||
@staticmethod
|
||||
def _is_transformer_block(name: str, module) -> bool:
|
||||
return "transformer_blocks" in name and name.split(".")[-1].isdigit()
|
||||
|
||||
_hsdp_shard_conditions = [_is_transformer_block]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
in_channels: int = 64,
|
||||
num_layers: int = 28,
|
||||
attention_head_dim: int = 128,
|
||||
num_attention_heads: int = 48,
|
||||
num_key_value_heads: int = 12,
|
||||
intermediate_size: int = 16384,
|
||||
timestep_embed_dim: int = 256,
|
||||
text_hidden_dim: int = 2560,
|
||||
num_text_layers: int = 12,
|
||||
text_num_attention_heads: int = 20,
|
||||
text_num_key_value_heads: int = 20,
|
||||
text_intermediate_size: int = 6912,
|
||||
num_layerwise_text_blocks: int = 2,
|
||||
num_refiner_text_blocks: int = 2,
|
||||
axes_dims_rope: tuple[int, int, int] = (32, 48, 48),
|
||||
rope_theta: float = 1000.0,
|
||||
norm_eps: float = 1e-5,
|
||||
od_config: "OmniDiffusionConfig | None" = None,
|
||||
quant_config: "QuantizationConfig | None" = None,
|
||||
) -> None:
|
||||
super().__init__()
|
||||
# The pipeline reads ``self.dtype`` to cast inputs before the transformer forward.
|
||||
self.dtype = od_config.dtype if od_config is not None else torch.get_default_dtype()
|
||||
self.od_config = od_config
|
||||
|
||||
hidden_size = attention_head_dim * num_attention_heads
|
||||
if sum(axes_dims_rope) != attention_head_dim:
|
||||
raise ValueError(
|
||||
f"sum(axes_dims_rope)={sum(axes_dims_rope)} must equal attention_head_dim={attention_head_dim}"
|
||||
)
|
||||
|
||||
self.in_channels = in_channels
|
||||
self.out_channels = in_channels
|
||||
self.hidden_size = hidden_size
|
||||
self.num_layers = num_layers
|
||||
|
||||
self.img_in = _linear(in_channels, hidden_size, True, quant_config, "img_in")
|
||||
self.time_embed = Krea2TimestepEmbedding(timestep_embed_dim, hidden_size, quant_config, "time_embed")
|
||||
self.time_mod_proj = _linear(hidden_size, 6 * hidden_size, True, quant_config, "time_mod_proj")
|
||||
self.text_fusion = Krea2TextFusion(
|
||||
num_text_layers=num_text_layers,
|
||||
dim=text_hidden_dim,
|
||||
num_heads=text_num_attention_heads,
|
||||
num_kv_heads=text_num_key_value_heads,
|
||||
intermediate_size=text_intermediate_size,
|
||||
num_layerwise_blocks=num_layerwise_text_blocks,
|
||||
num_refiner_blocks=num_refiner_text_blocks,
|
||||
eps=norm_eps,
|
||||
quant_config=quant_config,
|
||||
prefix="text_fusion",
|
||||
)
|
||||
self.txt_in = Krea2TextProjection(text_hidden_dim, hidden_size, norm_eps, quant_config, "txt_in")
|
||||
self.rotary_emb = Krea2RotaryPosEmbed(theta=rope_theta, axes_dim=list(axes_dims_rope))
|
||||
|
||||
self.transformer_blocks = nn.ModuleList(
|
||||
[
|
||||
Krea2TransformerBlock(
|
||||
hidden_size=hidden_size,
|
||||
intermediate_size=intermediate_size,
|
||||
num_heads=num_attention_heads,
|
||||
num_kv_heads=num_key_value_heads,
|
||||
norm_eps=norm_eps,
|
||||
quant_config=quant_config,
|
||||
prefix=f"transformer_blocks.{i}",
|
||||
)
|
||||
for i in range(num_layers)
|
||||
]
|
||||
)
|
||||
|
||||
self.final_layer = Krea2FinalLayer(hidden_size, in_channels, norm_eps, quant_config, "final_layer")
|
||||
|
||||
def forward(
|
||||
self,
|
||||
hidden_states: torch.Tensor,
|
||||
encoder_hidden_states: torch.Tensor,
|
||||
timestep: torch.Tensor,
|
||||
position_ids: torch.Tensor,
|
||||
encoder_attention_mask: torch.Tensor | None = None,
|
||||
) -> torch.Tensor:
|
||||
r"""Predict the flow-matching velocity for the image tokens.
|
||||
|
||||
Args:
|
||||
hidden_states: Packed (patchified) noisy image latents ``(batch, image_seq_len, in_channels)``.
|
||||
encoder_hidden_states: Tapped text-encoder hidden states
|
||||
``(batch, text_seq_len, num_text_layers, text_hidden_dim)``.
|
||||
timestep: Flow-matching time in ``[0, 1]`` of shape ``(batch,)``.
|
||||
position_ids: ``(t, h, w)`` rotary coordinates ``(text_seq_len + image_seq_len, 3)``.
|
||||
encoder_attention_mask: Boolean mask marking valid text tokens ``(batch, text_seq_len)`` or ``None``.
|
||||
"""
|
||||
if position_ids.ndim != 2 or position_ids.shape[-1] != 3:
|
||||
raise ValueError(f"`position_ids` must have shape (sequence_length, 3), got {tuple(position_ids.shape)}.")
|
||||
|
||||
batch_size, image_seq_len, _ = hidden_states.shape
|
||||
|
||||
temb = self.time_embed(timestep, dtype=hidden_states.dtype)
|
||||
temb_mod = self.time_mod_proj(F.gelu(temb, approximate="tanh"))
|
||||
|
||||
# 2D key-padding masks: padded text tokens are excluded as keys; their lanes are dropped at the output slice.
|
||||
text_attention_mask = None
|
||||
attention_mask = None
|
||||
if encoder_attention_mask is not None:
|
||||
text_attention_mask = encoder_attention_mask.bool()
|
||||
image_mask = text_attention_mask.new_ones((batch_size, image_seq_len))
|
||||
attention_mask = torch.cat([text_attention_mask, image_mask], dim=1)
|
||||
|
||||
encoder_hidden_states = self.text_fusion(encoder_hidden_states, attention_mask=text_attention_mask)
|
||||
encoder_hidden_states = self.txt_in(encoder_hidden_states)
|
||||
|
||||
text_seq_len = encoder_hidden_states.shape[1]
|
||||
hidden_states = self.img_in(hidden_states)
|
||||
hidden_states = torch.cat([encoder_hidden_states, hidden_states], dim=1)
|
||||
|
||||
image_rotary_emb = self.rotary_emb(position_ids)
|
||||
|
||||
for block in self.transformer_blocks:
|
||||
hidden_states = block(hidden_states, temb_mod, image_rotary_emb, attention_mask)
|
||||
|
||||
hidden_states = hidden_states[:, text_seq_len:]
|
||||
return self.final_layer(hidden_states, temb)
|
||||
|
||||
def load_weights(self, weights: Iterable[tuple[str, torch.Tensor]]) -> set[str]:
|
||||
# Module/parameter names mirror the diffusers ``Krea2Transformer2DModel`` exactly, so weights map 1:1.
|
||||
params_dict = dict(self.named_parameters())
|
||||
loaded_params: set[str] = set()
|
||||
for name, loaded_weight in weights:
|
||||
if name not in params_dict:
|
||||
continue
|
||||
param = params_dict[name]
|
||||
weight_loader = getattr(param, "weight_loader", default_weight_loader)
|
||||
weight_loader(param, loaded_weight)
|
||||
loaded_params.add(name)
|
||||
return loaded_params
|
||||
@@ -0,0 +1,551 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
|
||||
# Copyright 2026 Krea AI and The HuggingFace Team. All rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import inspect
|
||||
import logging
|
||||
import os
|
||||
from collections.abc import Iterable
|
||||
from typing import ClassVar
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
from diffusers.image_processor import VaeImageProcessor
|
||||
from diffusers.schedulers.scheduling_flow_match_euler_discrete import (
|
||||
FlowMatchEulerDiscreteScheduler,
|
||||
)
|
||||
from diffusers.utils.torch_utils import randn_tensor
|
||||
from torch import nn
|
||||
from transformers import AutoTokenizer, Qwen3VLModel
|
||||
from vllm.model_executor.models.utils import AutoWeightsLoader
|
||||
from vllm.transformers_utils.config import get_hf_file_to_dict
|
||||
|
||||
from vllm_omni.diffusion.data import DiffusionOutput, OmniDiffusionConfig
|
||||
from vllm_omni.diffusion.distributed.autoencoders.autoencoder_kl_qwenimage import DistributedAutoencoderKLQwenImage
|
||||
from vllm_omni.diffusion.distributed.utils import get_local_device
|
||||
from vllm_omni.diffusion.model_loader.diffusers_loader import DiffusersPipelineLoader
|
||||
from vllm_omni.diffusion.model_loader.hub_prefetch import from_pretrained_with_prefetch, prefetch_subfolders
|
||||
from vllm_omni.diffusion.models.interface import SupportsComponentDiscovery
|
||||
from vllm_omni.diffusion.models.krea2.krea2_transformer import Krea2Transformer2DModel
|
||||
from vllm_omni.diffusion.models.progress_bar import ProgressBarMixin
|
||||
from vllm_omni.diffusion.profiler.diffusion_pipeline_profiler import DiffusionPipelineProfilerMixin
|
||||
from vllm_omni.diffusion.request import OmniDiffusionRequest
|
||||
from vllm_omni.diffusion.utils.tf_utils import get_transformer_config_kwargs
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Text-encoder layer taps used when model_index.json does not specify ``text_encoder_select_layers``.
|
||||
DEFAULT_TEXT_ENCODER_SELECT_LAYERS = (2, 5, 8, 11, 14, 17, 20, 23, 26, 29, 32, 35)
|
||||
|
||||
|
||||
def get_krea2_post_process_func(od_config: OmniDiffusionConfig):
|
||||
model_name = od_config.model
|
||||
# Read only the VAE config (``get_hf_file_to_dict`` resolves a local dir via
|
||||
# ``Path(model)/file`` and a hub repo via a single-file download) instead of
|
||||
# pulling all weights just to compute ``vae_scale_factor``.
|
||||
vae_config = get_hf_file_to_dict("vae/config.json", model_name) or {}
|
||||
vae_scale_factor = 2 ** len(vae_config["temperal_downsample"]) if "temperal_downsample" in vae_config else 8
|
||||
|
||||
patch_size = 2
|
||||
try:
|
||||
model_index = get_hf_file_to_dict("model_index.json", model_name) or {}
|
||||
patch_size = int(model_index.get("patch_size", patch_size))
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
image_processor = VaeImageProcessor(vae_scale_factor=vae_scale_factor * patch_size)
|
||||
|
||||
def post_process_func(images: torch.Tensor):
|
||||
return image_processor.postprocess(images)
|
||||
|
||||
return post_process_func
|
||||
|
||||
|
||||
# Copied from diffusers.pipelines.flux.pipeline_flux.calculate_shift
|
||||
def calculate_shift(
|
||||
image_seq_len,
|
||||
base_seq_len: int = 256,
|
||||
max_seq_len: int = 4096,
|
||||
base_shift: float = 0.5,
|
||||
max_shift: float = 1.15,
|
||||
):
|
||||
m = (max_shift - base_shift) / (max_seq_len - base_seq_len)
|
||||
b = base_shift - m * base_seq_len
|
||||
mu = image_seq_len * m + b
|
||||
return mu
|
||||
|
||||
|
||||
# Copied from diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.retrieve_timesteps
|
||||
def retrieve_timesteps(
|
||||
scheduler,
|
||||
num_inference_steps: int | None = None,
|
||||
device: str | torch.device | None = None,
|
||||
timesteps: list[int] | None = None,
|
||||
sigmas: list[float] | None = None,
|
||||
**kwargs,
|
||||
) -> tuple[torch.Tensor, int]:
|
||||
if timesteps is not None and sigmas is not None:
|
||||
raise ValueError("Only one of `timesteps` or `sigmas` can be passed. Please choose one to set custom values")
|
||||
if timesteps is not None:
|
||||
accepts_timesteps = "timesteps" in set(inspect.signature(scheduler.set_timesteps).parameters.keys())
|
||||
if not accepts_timesteps:
|
||||
raise ValueError(
|
||||
f"The current scheduler class {scheduler.__class__}'s `set_timesteps` does not support custom"
|
||||
f" timestep schedules. Please check whether you are using the correct scheduler."
|
||||
)
|
||||
scheduler.set_timesteps(timesteps=timesteps, device=device, **kwargs)
|
||||
timesteps = scheduler.timesteps
|
||||
num_inference_steps = len(timesteps)
|
||||
elif sigmas is not None:
|
||||
accept_sigmas = "sigmas" in set(inspect.signature(scheduler.set_timesteps).parameters.keys())
|
||||
if not accept_sigmas:
|
||||
raise ValueError(
|
||||
f"The current scheduler class {scheduler.__class__}'s `set_timesteps` does not support custom"
|
||||
f" sigmas schedules. Please check whether you are using the correct scheduler."
|
||||
)
|
||||
scheduler.set_timesteps(sigmas=sigmas, device=device, **kwargs)
|
||||
timesteps = scheduler.timesteps
|
||||
num_inference_steps = len(timesteps)
|
||||
else:
|
||||
scheduler.set_timesteps(num_inference_steps, device=device, **kwargs)
|
||||
timesteps = scheduler.timesteps
|
||||
return timesteps, num_inference_steps
|
||||
|
||||
|
||||
class Krea2Pipeline(nn.Module, DiffusionPipelineProfilerMixin, ProgressBarMixin, SupportsComponentDiscovery):
|
||||
"""The Krea 2 text-to-image pipeline (vLLM-Omni port of ``diffusers.Krea2Pipeline``).
|
||||
|
||||
Components:
|
||||
- ``scheduler``: ``FlowMatchEulerDiscreteScheduler`` (resolution-aware exponential time shift).
|
||||
- ``vae``: the Qwen-Image VAE (f8, 16 latent channels).
|
||||
- ``text_encoder``: a Qwen3-VL model; the pipeline consumes a stack of hidden states tapped from several
|
||||
decoder layers rather than the last hidden state.
|
||||
- ``tokenizer``: the tokenizer paired with the text encoder.
|
||||
- ``transformer``: the Krea 2 single-stream MMDiT that predicts the flow-matching velocity.
|
||||
"""
|
||||
|
||||
_dit_modules: ClassVar[list[str]] = ["transformer"]
|
||||
_encoder_modules: ClassVar[list[str]] = ["text_encoder"]
|
||||
_vae_modules: ClassVar[list[str]] = ["vae"]
|
||||
|
||||
_PROFILER_TARGETS: ClassVar[list[str]] = ["text_encoder.forward", "diffuse", "vae.decode"]
|
||||
|
||||
def __init__(self, *, od_config: OmniDiffusionConfig, prefix: str = ""):
|
||||
super().__init__()
|
||||
self.od_config = od_config
|
||||
self.parallel_config = od_config.parallel_config
|
||||
# Only the transformer goes through the standard weights loader; text encoder and VAE load eagerly below.
|
||||
self.weights_sources = [
|
||||
DiffusersPipelineLoader.ComponentSource(
|
||||
model_or_path=od_config.model,
|
||||
subfolder="transformer",
|
||||
revision=od_config.revision,
|
||||
prefix="transformer.",
|
||||
fall_back_to_pt=True,
|
||||
)
|
||||
]
|
||||
|
||||
self.device = get_local_device()
|
||||
model = od_config.model
|
||||
local_files_only = os.path.isdir(model)
|
||||
|
||||
# Pipeline-level config lives in model_index.json, not the transformer config.
|
||||
model_index = get_hf_file_to_dict("model_index.json", model) or {}
|
||||
select_layers = model_index.get("text_encoder_select_layers") or DEFAULT_TEXT_ENCODER_SELECT_LAYERS
|
||||
self.text_encoder_select_layers = tuple(int(i) for i in select_layers)
|
||||
self.is_distilled = bool(model_index.get("is_distilled", False))
|
||||
self.patch_size = int(model_index.get("patch_size", 2))
|
||||
|
||||
subfolders = ["scheduler", "text_encoder", "vae", "tokenizer"]
|
||||
# See ``hub_prefetch.py`` for the transformers v5 subfolder race.
|
||||
prefetch_subfolders(model, subfolders, local_files_only=local_files_only)
|
||||
|
||||
self.scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
|
||||
model, subfolder="scheduler", local_files_only=local_files_only
|
||||
)
|
||||
|
||||
self.text_encoder = from_pretrained_with_prefetch(
|
||||
Qwen3VLModel.from_pretrained,
|
||||
model,
|
||||
subfolder="text_encoder",
|
||||
prefetch_list=subfolders,
|
||||
local_files_only=local_files_only,
|
||||
torch_dtype=od_config.dtype,
|
||||
)
|
||||
# Drop the unused Qwen3-VL vision tower before moving to GPU so it never consumes GPU memory.
|
||||
if hasattr(self.text_encoder, "visual"):
|
||||
del self.text_encoder.visual
|
||||
else:
|
||||
logger.warning("Krea2: vision tower not found on text encoder; skipping drop")
|
||||
self.text_encoder = self.text_encoder.to(self.device)
|
||||
|
||||
self.vae = from_pretrained_with_prefetch(
|
||||
DistributedAutoencoderKLQwenImage.from_pretrained,
|
||||
model,
|
||||
subfolder="vae",
|
||||
prefetch_list=subfolders,
|
||||
local_files_only=local_files_only,
|
||||
torch_dtype=od_config.dtype,
|
||||
).to(self.device)
|
||||
|
||||
transformer_kwargs = get_transformer_config_kwargs(od_config.tf_model_config, Krea2Transformer2DModel)
|
||||
self.transformer = Krea2Transformer2DModel(
|
||||
od_config=od_config, quant_config=od_config.quantization_config, **transformer_kwargs
|
||||
)
|
||||
|
||||
self.tokenizer = AutoTokenizer.from_pretrained(model, subfolder="tokenizer", local_files_only=local_files_only)
|
||||
|
||||
self.vae_scale_factor = 2 ** len(self.vae.temperal_downsample) if getattr(self, "vae", None) else 8
|
||||
self.image_processor = VaeImageProcessor(vae_scale_factor=self.vae_scale_factor * self.patch_size)
|
||||
self.default_sample_size = 1024
|
||||
|
||||
# Chat template tokenized as a fixed-length block: prompt padded to a fixed length first, assistant suffix
|
||||
# appended after the padding; the first ``prompt_template_encode_start_idx`` system tokens are later dropped.
|
||||
self.prompt_template_encode_prefix = (
|
||||
"<|im_start|>system\nDescribe the image by detailing the color, shape, size, texture, quantity, text, "
|
||||
"spatial relationships of the objects and background:<|im_end|>\n<|im_start|>user\n"
|
||||
)
|
||||
self.prompt_template_encode_suffix = "<|im_end|>\n<|im_start|>assistant\n"
|
||||
self.prompt_template_encode_start_idx = 34
|
||||
self.prompt_template_encode_num_suffix_tokens = 5
|
||||
|
||||
self.setup_diffusion_pipeline_profiler(
|
||||
enable_diffusion_pipeline_profiler=self.od_config.enable_diffusion_pipeline_profiler
|
||||
)
|
||||
|
||||
def get_text_hidden_states(
|
||||
self,
|
||||
prompt: str | list[str],
|
||||
max_sequence_length: int = 512,
|
||||
device: torch.device | None = None,
|
||||
) -> tuple[torch.Tensor, torch.Tensor]:
|
||||
"""Tokenize ``prompt`` into the fixed-length Krea 2 layout and tap the selected encoder hidden states.
|
||||
|
||||
Returns ``(hidden_states, attention_mask)`` of shapes ``(batch, text_seq_len, num_text_layers,
|
||||
text_hidden_dim)`` and ``(batch, text_seq_len)`` (bool).
|
||||
"""
|
||||
device = device or self.device
|
||||
prompt = [prompt] if isinstance(prompt, str) else prompt
|
||||
prefix_idx = self.prompt_template_encode_start_idx
|
||||
text = [self.prompt_template_encode_prefix + e for e in prompt]
|
||||
text_tokens = self.tokenizer(
|
||||
text,
|
||||
truncation=True,
|
||||
padding="max_length",
|
||||
max_length=max_sequence_length + prefix_idx - self.prompt_template_encode_num_suffix_tokens,
|
||||
return_tensors="pt",
|
||||
).to(device)
|
||||
suffix_tokens = self.tokenizer([self.prompt_template_encode_suffix] * len(text), return_tensors="pt").to(device)
|
||||
|
||||
input_ids = torch.cat([text_tokens.input_ids, suffix_tokens.input_ids], dim=1)
|
||||
attention_mask = torch.cat([text_tokens.attention_mask, suffix_tokens.attention_mask], dim=1).bool()
|
||||
|
||||
# Padding sits in the middle (``[prefix | prompt | PAD | suffix]``), so positions must count only real tokens
|
||||
# (padding consumes no position). Broadcast across the 3 mRoPE axes (T/H/W equal for text).
|
||||
position_ids = (attention_mask.long().cumsum(dim=-1) - 1).clamp(min=0)
|
||||
position_ids = position_ids.unsqueeze(0).expand(3, -1, -1)
|
||||
|
||||
outputs = self.text_encoder(
|
||||
input_ids=input_ids,
|
||||
attention_mask=attention_mask,
|
||||
position_ids=position_ids,
|
||||
output_hidden_states=True,
|
||||
)
|
||||
hidden_states = torch.stack([outputs.hidden_states[i] for i in self.text_encoder_select_layers], dim=2)
|
||||
|
||||
hidden_states = hidden_states[:, prefix_idx:]
|
||||
attention_mask = attention_mask[:, prefix_idx:]
|
||||
return hidden_states, attention_mask
|
||||
|
||||
def encode_prompt(
|
||||
self,
|
||||
prompt: str | list[str],
|
||||
device: torch.device | None = None,
|
||||
num_images_per_prompt: int = 1,
|
||||
prompt_embeds: torch.Tensor | None = None,
|
||||
prompt_embeds_mask: torch.Tensor | None = None,
|
||||
max_sequence_length: int = 512,
|
||||
) -> tuple[torch.Tensor, torch.Tensor]:
|
||||
device = device or self.device
|
||||
|
||||
if prompt_embeds is None:
|
||||
prompt_embeds, prompt_embeds_mask = self.get_text_hidden_states(prompt, max_sequence_length, device)
|
||||
|
||||
batch_size, seq_len, num_text_layers, dim = prompt_embeds.shape
|
||||
prompt_embeds = prompt_embeds.repeat(1, num_images_per_prompt, 1, 1)
|
||||
prompt_embeds = prompt_embeds.view(batch_size * num_images_per_prompt, seq_len, num_text_layers, dim)
|
||||
prompt_embeds_mask = prompt_embeds_mask.repeat(1, num_images_per_prompt)
|
||||
prompt_embeds_mask = prompt_embeds_mask.view(batch_size * num_images_per_prompt, seq_len)
|
||||
|
||||
return prompt_embeds, prompt_embeds_mask
|
||||
|
||||
def _pack_latents(self, latents, batch_size, num_channels_latents, height, width):
|
||||
p = self.patch_size
|
||||
latents = latents.view(batch_size, num_channels_latents, height // p, p, width // p, p)
|
||||
latents = latents.permute(0, 2, 4, 1, 3, 5)
|
||||
latents = latents.reshape(batch_size, (height // p) * (width // p), num_channels_latents * p * p)
|
||||
return latents
|
||||
|
||||
def _unpack_latents(self, latents, height, width):
|
||||
batch_size, _, channels = latents.shape
|
||||
p = self.patch_size
|
||||
|
||||
height = p * (int(height) // (self.vae_scale_factor * p))
|
||||
width = p * (int(width) // (self.vae_scale_factor * p))
|
||||
|
||||
latents = latents.view(batch_size, height // p, width // p, channels // (p * p), p, p)
|
||||
latents = latents.permute(0, 3, 1, 4, 2, 5)
|
||||
latents = latents.reshape(batch_size, channels // (p * p), 1, height, width)
|
||||
return latents
|
||||
|
||||
@staticmethod
|
||||
def prepare_position_ids(text_seq_len: int, grid_height: int, grid_width: int, device: torch.device):
|
||||
"""Build the ``(text_seq_len + grid_height * grid_width, 3)`` rotary coordinates: text tokens sit at the
|
||||
origin, image tokens carry their ``(0, h, w)`` latent-grid coordinates."""
|
||||
text_ids = torch.zeros(text_seq_len, 3, device=device)
|
||||
image_ids = torch.zeros(grid_height, grid_width, 3, device=device)
|
||||
image_ids[..., 1] = torch.arange(grid_height, device=device)[:, None]
|
||||
image_ids[..., 2] = torch.arange(grid_width, device=device)[None, :]
|
||||
image_ids = image_ids.reshape(grid_height * grid_width, 3)
|
||||
return torch.cat([text_ids, image_ids], dim=0)
|
||||
|
||||
def prepare_latents(self, batch_size, num_channels_latents, height, width, dtype, device, generator, latents=None):
|
||||
if latents is not None:
|
||||
return latents.to(device=device, dtype=dtype)
|
||||
|
||||
latent_height = height // self.vae_scale_factor
|
||||
latent_width = width // self.vae_scale_factor
|
||||
shape = (batch_size, num_channels_latents, latent_height, latent_width)
|
||||
|
||||
if isinstance(generator, list) and len(generator) != batch_size:
|
||||
raise ValueError(
|
||||
f"You have passed a list of generators of length {len(generator)}, but requested an effective batch"
|
||||
f" size of {batch_size}. Make sure the batch size matches the length of the generators."
|
||||
)
|
||||
|
||||
latents = randn_tensor(shape, generator=generator, device=device, dtype=dtype)
|
||||
latents = self._pack_latents(latents, batch_size, num_channels_latents, latent_height, latent_width)
|
||||
return latents
|
||||
|
||||
def _extract_prompts(self, prompts):
|
||||
"""Extract prompt and negative_prompt from the request's OmniPromptType list."""
|
||||
prompt = [p if isinstance(p, str) else (p.get("prompt") or "") for p in prompts] or None
|
||||
if all(isinstance(p, str) or p.get("negative_prompt") is None for p in prompts):
|
||||
negative_prompt = None
|
||||
elif prompts:
|
||||
negative_prompt = ["" if isinstance(p, str) else (p.get("negative_prompt") or "") for p in prompts]
|
||||
else:
|
||||
negative_prompt = None
|
||||
return prompt, negative_prompt
|
||||
|
||||
@property
|
||||
def do_classifier_free_guidance(self):
|
||||
return self._guidance_scale > 0
|
||||
|
||||
def diffuse(
|
||||
self,
|
||||
latents: torch.Tensor,
|
||||
timesteps: torch.Tensor,
|
||||
position_ids: torch.Tensor,
|
||||
prompt_embeds: torch.Tensor,
|
||||
prompt_embeds_mask: torch.Tensor,
|
||||
negative_prompt_embeds: torch.Tensor | None,
|
||||
negative_prompt_embeds_mask: torch.Tensor | None,
|
||||
guidance_scale: float,
|
||||
) -> torch.Tensor:
|
||||
"""Run the flow-matching denoising loop and return the final latents."""
|
||||
self.scheduler.set_begin_index(0)
|
||||
transformer_dtype = self.transformer.dtype
|
||||
with self.progress_bar(total=len(timesteps)) as progress_bar:
|
||||
for _, t in enumerate(timesteps):
|
||||
timestep = (t / self.scheduler.config.num_train_timesteps).expand(latents.shape[0])
|
||||
timestep = timestep.to(transformer_dtype)
|
||||
latent_input = latents.to(transformer_dtype)
|
||||
|
||||
noise_pred = self.transformer(
|
||||
hidden_states=latent_input,
|
||||
encoder_hidden_states=prompt_embeds.to(transformer_dtype),
|
||||
timestep=timestep,
|
||||
position_ids=position_ids,
|
||||
encoder_attention_mask=prompt_embeds_mask,
|
||||
)
|
||||
|
||||
if self.do_classifier_free_guidance:
|
||||
neg_noise_pred = self.transformer(
|
||||
hidden_states=latent_input,
|
||||
encoder_hidden_states=negative_prompt_embeds.to(transformer_dtype),
|
||||
timestep=timestep,
|
||||
position_ids=position_ids,
|
||||
encoder_attention_mask=negative_prompt_embeds_mask,
|
||||
)
|
||||
noise_pred = noise_pred + guidance_scale * (noise_pred - neg_noise_pred)
|
||||
|
||||
latents = self.scheduler.step(noise_pred, t, latents, return_dict=False)[0]
|
||||
progress_bar.update()
|
||||
|
||||
return latents
|
||||
|
||||
def forward(
|
||||
self,
|
||||
req: OmniDiffusionRequest,
|
||||
prompt: str | list[str] | None = None,
|
||||
negative_prompt: str | list[str] | None = None,
|
||||
height: int = 1024,
|
||||
width: int = 1024,
|
||||
num_inference_steps: int = 28,
|
||||
sigmas: list[float] | None = None,
|
||||
guidance_scale: float = 4.5,
|
||||
num_images_per_prompt: int = 1,
|
||||
generator: torch.Generator | list[torch.Generator] | None = None,
|
||||
latents: torch.Tensor | None = None,
|
||||
output_type: str | None = "pil",
|
||||
max_sequence_length: int = 512,
|
||||
) -> DiffusionOutput:
|
||||
extracted_prompt, extracted_negative = self._extract_prompts(req.prompts)
|
||||
prompt = extracted_prompt or prompt
|
||||
if extracted_negative is not None:
|
||||
negative_prompt = extracted_negative
|
||||
|
||||
sp = req.sampling_params
|
||||
height = sp.height or height
|
||||
width = sp.width or width
|
||||
num_inference_steps = sp.num_inference_steps or num_inference_steps
|
||||
sigmas = sp.sigmas or sigmas
|
||||
max_sequence_length = sp.max_sequence_length or max_sequence_length
|
||||
generator = sp.generator or generator
|
||||
if sp.guidance_scale_provided:
|
||||
guidance_scale = sp.guidance_scale
|
||||
else:
|
||||
# Distilled checkpoint is trained guidance-free (and the request layer coerces an explicit
|
||||
# guidance_scale=0 to "not provided"), so default it to no CFG.
|
||||
guidance_scale = 0.0 if self.is_distilled else guidance_scale
|
||||
num_images_per_prompt = sp.num_outputs_per_prompt if sp.num_outputs_per_prompt > 0 else num_images_per_prompt
|
||||
|
||||
multiple = self.vae_scale_factor * self.patch_size
|
||||
if height % multiple != 0 or width % multiple != 0:
|
||||
rounded_height = ((height + multiple - 1) // multiple) * multiple
|
||||
rounded_width = ((width + multiple - 1) // multiple) * multiple
|
||||
logger.warning(
|
||||
"`height` and `width` must be multiples of %d; rounding up from %dx%d to %dx%d.",
|
||||
multiple,
|
||||
height,
|
||||
width,
|
||||
rounded_height,
|
||||
rounded_width,
|
||||
)
|
||||
height, width = rounded_height, rounded_width
|
||||
|
||||
device = self.device
|
||||
self._guidance_scale = guidance_scale
|
||||
|
||||
if prompt is not None and isinstance(prompt, str):
|
||||
batch_size = 1
|
||||
elif prompt is not None and isinstance(prompt, list):
|
||||
batch_size = len(prompt)
|
||||
else:
|
||||
batch_size = 1
|
||||
|
||||
prompt_embeds, prompt_embeds_mask = self.encode_prompt(
|
||||
prompt=prompt,
|
||||
device=device,
|
||||
num_images_per_prompt=num_images_per_prompt,
|
||||
max_sequence_length=max_sequence_length,
|
||||
)
|
||||
negative_prompt_embeds = None
|
||||
negative_prompt_embeds_mask = None
|
||||
if self.do_classifier_free_guidance:
|
||||
if negative_prompt is None:
|
||||
negative_prompt = ""
|
||||
if isinstance(negative_prompt, str):
|
||||
negative_prompt = [negative_prompt] * batch_size
|
||||
negative_prompt_embeds, negative_prompt_embeds_mask = self.encode_prompt(
|
||||
prompt=negative_prompt,
|
||||
device=device,
|
||||
num_images_per_prompt=num_images_per_prompt,
|
||||
max_sequence_length=max_sequence_length,
|
||||
)
|
||||
|
||||
num_channels_latents = self.transformer.in_channels // (self.patch_size**2)
|
||||
latents = self.prepare_latents(
|
||||
batch_size * num_images_per_prompt,
|
||||
num_channels_latents,
|
||||
height,
|
||||
width,
|
||||
prompt_embeds.dtype,
|
||||
device,
|
||||
generator,
|
||||
latents,
|
||||
)
|
||||
grid_height = height // (self.vae_scale_factor * self.patch_size)
|
||||
grid_width = width // (self.vae_scale_factor * self.patch_size)
|
||||
position_ids = self.prepare_position_ids(prompt_embeds.shape[1], grid_height, grid_width, device)
|
||||
|
||||
sigmas = np.linspace(1.0, 1 / num_inference_steps, num_inference_steps) if sigmas is None else sigmas
|
||||
image_seq_len = latents.shape[1]
|
||||
if self.is_distilled:
|
||||
mu = 1.15
|
||||
else:
|
||||
mu = calculate_shift(
|
||||
image_seq_len,
|
||||
self.scheduler.config.get("base_image_seq_len", 256),
|
||||
self.scheduler.config.get("max_image_seq_len", 6400),
|
||||
self.scheduler.config.get("base_shift", 0.5),
|
||||
self.scheduler.config.get("max_shift", 1.15),
|
||||
)
|
||||
timesteps, num_inference_steps = retrieve_timesteps(
|
||||
self.scheduler, num_inference_steps, device, sigmas=sigmas, mu=mu
|
||||
)
|
||||
|
||||
latents = self.diffuse(
|
||||
latents=latents,
|
||||
timesteps=timesteps,
|
||||
position_ids=position_ids,
|
||||
prompt_embeds=prompt_embeds,
|
||||
prompt_embeds_mask=prompt_embeds_mask,
|
||||
negative_prompt_embeds=negative_prompt_embeds,
|
||||
negative_prompt_embeds_mask=negative_prompt_embeds_mask,
|
||||
guidance_scale=guidance_scale,
|
||||
)
|
||||
|
||||
if output_type == "latent":
|
||||
image = latents
|
||||
else:
|
||||
latents = self._unpack_latents(latents, height, width)
|
||||
latents = latents.to(self.vae.dtype)
|
||||
latents_mean = (
|
||||
torch.tensor(self.vae.config.latents_mean)
|
||||
.view(1, self.vae.config.z_dim, 1, 1, 1)
|
||||
.to(latents.device, latents.dtype)
|
||||
)
|
||||
latents_std = 1.0 / torch.tensor(self.vae.config.latents_std).view(1, self.vae.config.z_dim, 1, 1, 1).to(
|
||||
latents.device, latents.dtype
|
||||
)
|
||||
latents = latents / latents_std + latents_mean
|
||||
image = self.vae.decode(latents, return_dict=False)[0][:, :, 0]
|
||||
|
||||
return DiffusionOutput(
|
||||
output=image,
|
||||
stage_durations=self.stage_durations if hasattr(self, "stage_durations") else None,
|
||||
)
|
||||
|
||||
def load_weights(self, weights: Iterable[tuple[str, torch.Tensor]]) -> set[str]:
|
||||
loader = AutoWeightsLoader(self)
|
||||
loaded_weights = loader.load_weights(weights)
|
||||
# VAE and text encoder load eagerly in __init__; record their params so strict load checks pass.
|
||||
loaded_weights |= {f"vae.{name}" for name, _ in self.vae.named_parameters()}
|
||||
if self.text_encoder is not None:
|
||||
loaded_weights |= {f"text_encoder.{name}" for name, _ in self.text_encoder.named_parameters()}
|
||||
return loaded_weights
|
||||
@@ -306,6 +306,11 @@ _DIFFUSION_MODELS = {
|
||||
"pipeline_sdxl",
|
||||
"StableDiffusionXLPipeline",
|
||||
),
|
||||
"Krea2Pipeline": (
|
||||
"krea2",
|
||||
"pipeline_krea2",
|
||||
"Krea2Pipeline",
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
@@ -533,6 +538,7 @@ _DIFFUSION_POST_PROCESS_FUNCS = {
|
||||
"Cosmos3OmniDiffusersPipeline": "get_cosmos3_post_process_func",
|
||||
"HiDreamImagePipeline": "get_hidream_image_post_process_func",
|
||||
"StableDiffusionXLPipeline": "get_sdxl_image_post_process_func",
|
||||
"Krea2Pipeline": "get_krea2_post_process_func",
|
||||
}
|
||||
|
||||
_DIFFUSION_ACTION_POST_PROCESS_FUNCS = {
|
||||
|
||||
Reference in new issue
Block a user