anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Refactor]: Remove sensenova examples (#5201)

Browse Source 
Signed-off-by: princepride <wangzhipeng628@gmail.com>
This commit is contained in:
汪志鹏 authored and GitHub committed 2026-07-22 15:36:00 +00:00
1 parent 8219c01b57
commit f44649ffd0
10 files changed
+182 -604
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

examples/offline_inference/image_to_image/image_edit.py
+4
View File
@@ -97,6 +97,7 @@ import torch
from PIL import Image
from vllm_omni.diffusion.data import DiffusionParallelConfig
from vllm_omni.diffusion.utils.image_output import extract_images_from_outputs
from vllm_omni.diffusion.utils.param_utils import apply_declared_extra_args
from vllm_omni.entrypoints.omni import Omni
from vllm_omni.entrypoints.openai.stage_params import clone_sampling_params
@@ -625,6 +626,9 @@ def main():
if images:
break
if not images:
images = extract_images_from_outputs(outputs)
if not images:
raise ValueError("No images found in request_output")
examples/offline_inference/sensenova_u1/README.md
-214
View File
@@ -1,214 +0,0 @@
# SenseNova-U1: Offline inference
## Architecture
SenseNova-U1 is a unified Qwen3-based LLM with Mixture-of-Tokenizers (MoT) attention. Unlike two-stage pipelines (e.g., BAGEL), it handles text encoding, optional reasoning ("think mode"), and flow-matching-based image denoising entirely within a single diffusion stage.
| Feature | Description |
| :------ | :---------- |
| **Model type** | Unified LLM (single-stage diffusion pipeline) |
| **Base LLM** | Qwen3 with MoT attention and 3D RoPE |
| **Image generation** | Flow-matching Euler sampler, no separate VAE |
| **Think mode** | Optional chain-of-thought reasoning before image generation |
| **Parallelism** | Tensor Parallelism (TP) with fused QKV and fused gate/up projections |
| **Modalities** | text2img, img2img, img2text (understanding), text2text |
## Quick Start
```bash
cd examples/offline_inference/sensenova_u1
# Text-to-image
python end2end.py --prompt "A cute cat" --think
# Image-to-image editing
python end2end.py --prompt "Turn this into an oil painting" \
--image input.png --think
# Image understanding (img2text)
python end2end.py --modality img2text \
--prompt "Describe this image in detail" \
--image photo.jpg
# Text-to-text (pure chat)
python end2end.py --modality text2text \
--prompt "What is the capital of France?"
# Custom resolution (image generation)
python end2end.py --prompt "A futuristic cityscape at sunset" \
--width 2048 --height 1024 --think
# Text-to-image with cache acceleration (cache_dit or tea_cache)
python end2end.py --prompt "A cute cat" \
--cache-backend tea_cache
```
> **Note**: Default configuration works on a single **NVIDIA A100 (80GB)** or **H100** GPU.
## Text-to-Image (t2i)
Standard text-to-image with optional think mode:
```bash
python end2end.py \
--prompt "Close portrait of an elderly woman by a farmhouse window, textured skin, gentle smile, warm natural light" \
--width 1536 --height 2720 \
--think --print-think
```
## Image-to-Image Editing (img2img)
Pass one or more `--image` paths to trigger img2img mode. The model uses the
input image(s) as visual context for editing:
```bash
# Single input image
python end2end.py \
--prompt "Add a sunset sky in the background" \
--image photo.jpg \
--width 2048 --height 2048 \
--think
# Multiple reference images
python end2end.py \
--prompt "Combine the style of Image-1 with the content of Image-2" \
--image style_ref.png content_ref.png \
--width 2048 --height 2048 \
--think
```
### Dual CFG for img2img
img2img supports dual classifier-free guidance (text CFG + image CFG):
```bash
python end2end.py \
--prompt "Make the person smile" \
--image portrait.jpg \
--cfg-scale 4.0 \
--img-cfg-scale 2.0 \
--think
```
| `img_cfg_scale` | Behavior |
| :-------------- | :------- |
| `1.0` (default) | Text-only CFG: guidance = image-condition → full-condition |
| `== cfg_scale` | Standard CFG: guidance = unconditional → full-condition |
| Other value | Dual CFG: separate text and image guidance strengths |
## Full Parameter Reference
### General Parameters
| Parameter | Default | Description |
| :-------- | :------ | :---------- |
| `--modality` | `auto` | Task modality: `auto`, `text2img`, `img2img`, `img2text`, `text2text` |
| `--prompt` | "A cute cat..." | Text prompt / editing instruction / question |
| `--image` | None | Input image path(s) for img2img or img2text |
### Image Generation Parameters (text2img / img2img)
| Parameter | Default | Description |
| :-------- | :------ | :---------- |
| `--height` | 2048 | Height of generated image (pixels) |
| `--width` | 2048 | Width of generated image (pixels) |
| `--seed` | 42 | Random seed for reproducibility |
| `--num-steps` | 50 | Number of denoising steps |
| `--cfg-scale` | 4.0 | Text classifier-free guidance scale |
| `--img-cfg-scale` | 1.0 | Image CFG scale (img2img only, 1.0 = disabled) |
| `--cfg-norm` | "none" | CFG normalization: none, global, channel, cfg_zero_star (t2i only) |
| `--timestep-shift` | 3.0 | Timestep shift for flow-matching schedule |
| `--t-eps` | 0.02 | Epsilon for timestep schedule |
| `--think` | False | Enable think mode |
| `--print-think` | False | Print think text to stdout |
### Text Generation Parameters (text2text / img2text)
| Parameter | Default | Description |
| :-------- | :------ | :---------- |
| `--max-tokens` | 512 | Maximum number of tokens to generate |
| `--do-sample` | False | Use sampling instead of greedy decoding |
| `--temperature` | 0.7 | Sampling temperature (higher = more diverse) |
### Infrastructure Parameters
| Parameter | Default | Description |
| :-------- | :------ | :---------- |
| `--model` | `SenseNova/SenseNova-U1-8B-MoT` | HuggingFace model ID or local path |
| `--output` | `.` | Output directory for saved images |
| `--tensor-parallel-size` | 1 | Number of GPUs for tensor parallelism |
| `--enforce-eager` | False | Disable torch.compile |
| `--enable-cpu-offload` | False | Enable module-wise (sequential) CPU offload to reduce peak VRAM |
| `--cache-backend` | None | Set to `cache_dit` or `tea_cache` for image generation acceleration |
| `--enable-cache-dit-summary` | False | Print Cache-DiT cache statistics after generation |
## Reducing GPU Memory Usage
For hardware with limited VRAM, enable **module-wise CPU offload** with
`--enable-cpu-offload`. The pipeline implements `SupportsComponentDiscovery`, so the
vision encoder (`vision_model`) and the Qwen3 LLM (`language_model`) are
swapped between CPU and GPU on demand:
- During text/vision encoding, the LLM is on CPU.
- During the diffusion loop, the vision encoder is on CPU.
- Lightweight FM modules stay resident on GPU.
This lowers peak VRAM at the cost of extra CPU<->GPU transfers (use pinned
memory) — useful for running SenseNova-U1 on consumer-grade GPUs.
```bash
# Text-to-image with CPU offload
python end2end.py \
--prompt "A cute cat sitting on a windowsill" \
--width 2048 --height 2048 \
--enable-cpu-offload --think
# Image-to-image editing with CPU offload
python end2end.py \
--prompt "Turn this into an oil painting" \
--image input.png \
--width 2048 --height 2048 \
--enable-cpu-offload --think
# Image understanding (img2text) with CPU offload
python end2end.py \
--modality img2text \
--prompt "Describe this image in detail" \
--image photo.jpg \
--enable-cpu-offload
```
> **Notes**
> - CPU offload is **single-GPU only** (incompatible with `--tensor-parallel-size > 1`).
> - First-step latency is higher because of the cold-start CPU<->GPU transfers.
> - For more details and other offloading strategies, see
> [CPU Offloading for Diffusion Models](../../../docs/user_guide/diffusion/cpu_offload_diffusion.md).
## Reproducing the E2E Test
The following command reproduces the pixel-validated CI test case:
```bash
python end2end.py \
--prompt "Close portrait of an elderly woman by a farmhouse window, textured skin, gentle smile, warm natural light, emotional documentary look. The portrait should feel polished and natural, with sharp eyes, realistic skin texture, accurate facial anatomy, and premium lighting that keeps the face as the main focus." \
--width 1536 --height 2720 \
--seed 42 --num-steps 50 \
--cfg-scale 4.0 --timestep-shift 3.0 --cfg-norm none \
--think --print-think \
--output outputs
```
The corresponding pytest:
```bash
pytest -s -v tests/e2e/offline_inference/test_sensenova_u1_text2img.py \
-m "advanced_model" --run-level "advanced_model"
```
The img2img counterpart lives at
`tests/e2e/offline_inference/test_sensenova_u1_img2img.py`.
## Online Serving
For OpenAI-compatible API serving, see
[`examples/online_serving/sensenova_u1/`](../../online_serving/sensenova_u1/README.md).
examples/offline_inference/sensenova_u1/end2end.py
-307
View File
@@ -1,307 +0,0 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
"""
End-to-end offline inference for SenseNova-U1-8B-MoT.
Supports four modalities: text2img, img2img, img2text, text2text.
Text-to-image:
python end2end.py --prompt "A cute cat" --think
Image-to-image (editing):
python end2end.py --prompt "Turn this into an oil painting" \
--image input.png --think
Image understanding:
python end2end.py --modality img2text \
--prompt "Describe this image" --image photo.jpg
Text chat:
python end2end.py --modality text2text \
--prompt "What is the capital of France?"
See README.md for more examples.
"""
import argparse
import os
from PIL import Image
from vllm_omni.entrypoints.omni import Omni
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
def parse_args():
parser = argparse.ArgumentParser(
description="SenseNova-U1 text-to-image / image-to-image / text / understanding via vLLM-Omni.",
)
parser.add_argument(
"--model",
default="SenseNova/SenseNova-U1-8B-MoT",
help="HuggingFace model ID or local path.",
)
parser.add_argument(
"--modality",
default="auto",
choices=["auto", "text2img", "img2img", "img2text", "text2text"],
help="Task modality. 'auto' infers from --image (img2img/img2text if images, else t2i/t2t).",
)
parser.add_argument(
"--prompt",
default="A cute cat sitting on a windowsill, soft natural light",
help="Text prompt for generation or editing instruction.",
)
parser.add_argument(
"--image",
nargs="+",
metavar="PATH",
default=None,
help="Input image path(s) for img2img or img2text.",
)
parser.add_argument(
"--output",
type=str,
default=".",
help="Output directory to save generated images.",
)
# Image dimensions
parser.add_argument("--height", type=int, default=2048, help="Height of generated image.")
parser.add_argument("--width", type=int, default=2048, help="Width of generated image.")
# Generation parameters
parser.add_argument("--seed", type=int, default=42, help="Random seed for deterministic results.")
parser.add_argument("--num-steps", type=int, default=50, help="Number of denoising steps.")
parser.add_argument("--cfg-scale", type=float, default=4.0, help="Text classifier-free guidance scale.")
parser.add_argument(
"--img-cfg-scale",
type=float,
default=1.0,
help="Image CFG scale for img2img (1.0 = disabled). "
"When both --cfg-scale and --img-cfg-scale differ, dual CFG is used.",
)
parser.add_argument(
"--cfg-norm",
type=str,
default="none",
choices=["none", "global", "channel", "cfg_zero_star"],
help="CFG normalization mode. cfg_zero_star is only for t2i.",
)
parser.add_argument(
"--timestep-shift",
type=float,
default=3.0,
help="Timestep shift for flow-matching schedule.",
)
parser.add_argument(
"--t-eps",
type=float,
default=0.02,
help="Epsilon for flow-matching timestep schedule.",
)
# Text generation parameters (for text2text / img2text)
parser.add_argument(
"--max-tokens",
type=int,
default=2048,
help="Max tokens for text generation (text2text / img2text).",
)
parser.add_argument(
"--do-sample",
action="store_true",
help="Use sampling instead of greedy decoding for text generation.",
)
parser.add_argument(
"--temperature",
type=float,
default=0.7,
help="Sampling temperature for text generation.",
)
# Think mode
parser.add_argument(
"--think",
action="store_true",
help="Enable think mode: the model reasons about the prompt before generating the image.",
)
parser.add_argument(
"--print-think",
action="store_true",
help="Print the generated think text to stdout.",
)
# Advanced
parser.add_argument(
"--tensor-parallel-size",
type=int,
default=1,
help="Number of GPUs for tensor parallelism.",
)
parser.add_argument(
"--cfg-parallel-size",
type=int,
default=1,
help="Number of GPUs for CFG parallelism.",
)
parser.add_argument(
"--enforce-eager",
action="store_true",
help="Disable torch.compile and force eager execution.",
)
parser.add_argument(
"--enable-cpu-offload",
action="store_true",
help=(
"Enable model-level (sequential) CPU offloading. "
"Mutually exclusive between vision_model and language_model: "
"only one is on GPU at a time, reducing peak VRAM at the cost "
"of extra CPU<->GPU transfers."
),
)
parser.add_argument(
"--cache-backend",
type=str,
default=None,
choices=["cache_dit", "tea_cache"],
help="Cache backend to use for image generation acceleration.",
)
parser.add_argument(
"--enable-cache-dit-summary",
action="store_true",
help="Enable cache-dit summary logging after diffusion forward passes.",
)
return parser.parse_args()
def _resolve_modality(args):
"""Determine the effective modality from CLI args."""
if args.modality != "auto":
return args.modality
has_images = args.image is not None
# Default: image-producing tasks for backward compatibility
return "img2img" if has_images else "text2img"
def main():
args = parse_args()
os.makedirs(args.output, exist_ok=True)
modality = _resolve_modality(args)
is_text_output = modality in ("text2text", "img2text")
has_images = modality in ("img2img", "img2text")
omni = Omni(
model=args.model,
tensor_parallel_size=args.tensor_parallel_size,
enforce_eager=args.enforce_eager,
enable_cpu_offload=args.enable_cpu_offload,
cache_backend=args.cache_backend,
enable_cache_dit_summary=args.enable_cache_dit_summary,
cfg_parallel_size=args.cfg_parallel_size,
)
extra_args = {
"cfg_scale": args.cfg_scale,
"cfg_norm": args.cfg_norm,
"timestep_shift": args.timestep_shift,
"cfg_interval": (0.0, 1.0),
"batch_size": 1,
"think": args.think,
"t_eps": args.t_eps,
}
if modality == "img2img":
extra_args["img_cfg_scale"] = args.img_cfg_scale
if is_text_output:
extra_args["max_tokens"] = args.max_tokens
extra_args["do_sample"] = args.do_sample
extra_args["temperature"] = args.temperature
sampling_params = OmniDiffusionSamplingParams(
height=args.height,
width=args.width,
seed=args.seed,
num_inference_steps=args.num_steps,
extra_args=extra_args,
)
print(f"\n{'=' * 60}")
print(f"SenseNova-U1 Configuration ({modality}):")
print(f" Model : {args.model}")
if not is_text_output:
print(f" Image size : {args.width}x{args.height}")
print(f" Steps : {args.num_steps}")
print(f" CFG scale : {args.cfg_scale}")
if modality == "img2img":
print(f" Img CFG scale : {args.img_cfg_scale}")
if has_images:
print(f" Input images : {args.image}")
if is_text_output:
print(f" Max tokens : {args.max_tokens}")
print(f" Temperature : {args.temperature}")
print(f" Seed : {args.seed}")
print(f" Think mode : {args.think}")
print(f" TP size : {args.tensor_parallel_size}")
print(f" CFG size : {args.cfg_parallel_size}")
print(f" Cache backend : {args.cache_backend or 'none'}")
print(f"{'=' * 60}\n")
# Build prompt dict
if has_images:
if not args.image:
raise ValueError(f"{modality} requires --image.")
input_images = [Image.open(p).convert("RGB") for p in args.image]
if is_text_output:
prompt_dict = {
"prompt": args.prompt,
"multi_modal_data": {"image": input_images},
"modalities": ["text"],
}
else:
prompt_dict = {
"prompt": args.prompt,
"multi_modal_data": {"image": input_images},
"modalities": ["img2img"],
}
elif is_text_output:
prompt_dict = {"prompt": args.prompt, "modalities": ["text"]}
else:
prompt_dict = {"prompt": args.prompt, "modalities": ["image"]}
outputs = list(
omni.generate(
prompts=prompt_dict,
sampling_params_list=sampling_params,
)
)
for req_output in outputs:
multimodal_output = getattr(req_output, "multimodal_output", {}) or {}
metadata = multimodal_output.get("metadata", {}) if isinstance(multimodal_output, dict) else {}
text_metadata = metadata.get("text", {}) if isinstance(metadata, dict) else {}
think_text = text_metadata.get("think_text") if isinstance(text_metadata, dict) else None
if args.print_think and think_text:
print(f"[Think]\n{think_text}\n")
if is_text_output:
text = multimodal_output.get("text", "") if isinstance(multimodal_output, dict) else ""
if not text:
text = text_metadata.get("text_output", "") if isinstance(text_metadata, dict) else ""
if not text:
text = getattr(req_output, "text", "") or ""
print(f"[Response]\n{text}")
else:
images = getattr(req_output, "images", None) or []
if not images:
print("[Warning] No images generated.")
continue
for j, img in enumerate(images):
save_path = os.path.join(args.output, f"sensenova_u1_output_{j}.png")
img.save(save_path)
print(f"[Output] Saved {img.size[0]}x{img.size[1]} image to {save_path}")
if __name__ == "__main__":
main()
examples/offline_inference/text_to_image/text_to_image.py
+2 -35
View File
@@ -11,6 +11,7 @@ from typing import Any
import torch
from vllm_omni.diffusion.data import logger
from vllm_omni.diffusion.utils.image_output import extract_images_from_outputs
from vllm_omni.diffusion.utils.param_utils import apply_declared_extra_args
from vllm_omni.entrypoints.omni import Omni
from vllm_omni.entrypoints.openai.stage_params import clone_sampling_params
@@ -601,11 +602,8 @@ def main():
if images:
break
# Fallback: generation-stage pipelines (e.g. MammothModa2's AR->DiT) return the
# generated image as a tensor under multimodal_output instead of populating the
# `images` field that diffusion-stage pipelines fill.
if not images:
images = _images_from_multimodal_output(outputs)
images = extract_images_from_outputs(outputs)
if not images:
raise ValueError("No images found in request_output")
@@ -624,36 +622,5 @@ def main():
print(f"Saved generated image to {save_path}")
def _images_from_multimodal_output(outputs: list[Any]) -> list[Any]:
"""Extract PIL images from multimodal_output tensors.
Generation-stage pipelines (e.g. MammothModa2's AR->DiT) return the generated
image as a tensor (normalized to [-1, 1], CHW) under ``multimodal_output``
rather than populating the ``images`` field. Convert any such tensors to PIL.
"""
from PIL import Image
pil_images: list[Any] = []
for output in outputs:
req_out = getattr(output, "request_output", output)
for completion in getattr(req_out, "outputs", None) or []:
# multimodal_output is a MultimodalPayload (a Mapping) keyed by modality,
# matching how omni examples (ming_flash_omni / magi_human / dynin) read it.
mm = getattr(completion, "multimodal_output", None) or {}
if "image" not in mm:
continue
payload = mm["image"]
for tensor in payload if isinstance(payload, list) else [payload]:
if not isinstance(tensor, torch.Tensor):
continue
img = tensor.detach().to("cpu", dtype=torch.float32)
if img.ndim == 4:
img = img[0]
img = (img / 2 + 0.5).clamp(0, 1).mul(255).to(torch.uint8)
img = img.permute(1, 2, 0).contiguous().numpy()
pil_images.append(Image.fromarray(img))
return pil_images
if __name__ == "__main__":
main()
recipes/SenseNova/SenseNova-U1.md
+40 -36
View File
@@ -22,8 +22,10 @@ text-to-text chat.
## References
- Offline example:
[`examples/offline_inference/sensenova_u1/end2end.py`](../../examples/offline_inference/sensenova_u1/end2end.py)
- Offline text-to-image:
[`examples/offline_inference/text_to_image/text_to_image.py`](../../examples/offline_inference/text_to_image/text_to_image.py)
- Offline image-to-image:
[`examples/offline_inference/image_to_image/image_edit.py`](../../examples/offline_inference/image_to_image/image_edit.py)
- Online serving:
[`examples/online_serving/sensenova_u1/`](../../examples/online_serving/sensenova_u1/)
- E2E tests:
@@ -48,13 +50,14 @@ text-to-text chat.
#### Command
```bash
python examples/offline_inference/sensenova_u1/end2end.py \
python examples/offline_inference/text_to_image/text_to_image.py \
--model SenseNova/SenseNova-U1-8B-MoT \
--prompt "Close portrait of an elderly woman by a farmhouse window, textured skin, gentle smile, warm natural light, emotional documentary look. The portrait should feel polished and natural, with sharp eyes, realistic skin texture, accurate facial anatomy, and premium lighting that keeps the face as the main focus." \
--width 1536 --height 2720 \
--seed 42 --num-steps 50 \
--cfg-scale 4.0 --timestep-shift 3.0 --cfg-norm none \
--think --print-think \
--output outputs
--seed 42 --num-inference-steps 50 \
--cfg-scale 4.0 \
--extra-body '{"think": true, "cfg_norm": "none", "timestep_shift": 3.0, "t_eps": 0.02}' \
--output outputs/sensenova_u1_output.png
```
#### Verification
@@ -74,8 +77,7 @@ pytest -s -v tests/e2e/offline_inference/test_sensenova_u1_text2img.py \
#### Text-to-Image via the shared example
SenseNova-U1 is also registered with the shared text-to-image example, so you
can run it through `text_to_image.py` instead of `end2end.py`. Forward the
SenseNova-U1 is registered with the shared text-to-image example. Forward
SenseNova-specific generation parameters as a JSON object through `--extra-body`:
```bash
@@ -94,48 +96,49 @@ The `--extra-body` keys are filtered against the model's declared
[`vllm_omni/model_extras/sensenova_u1.py`](../../vllm_omni/model_extras/sensenova_u1.py)):
`think`, `cfg_scale`, `cfg_norm`, `timestep_shift`, `t_eps`, `img_cfg_scale`,
and `max_tokens`. No deploy YAML is needed — the engine auto-generates a
single-stage diffusion config. For richer task modalities (img2text, text2text)
and CPU offload, use the dedicated `end2end.py` script described above.
single-stage diffusion config.
#### Image-to-Image Editing (img2img)
```bash
python examples/offline_inference/sensenova_u1/end2end.py \
python examples/offline_inference/image_to_image/image_edit.py \
--model SenseNova/SenseNova-U1-8B-MoT \
--prompt "Turn this into an oil painting" \
--image input.png \
--width 2048 --height 2048 \
--seed 42 --num-steps 50 \
--cfg-scale 4.0 --img-cfg-scale 1.0 --cfg-norm none \
--think --print-think \
--output outputs
--resolution 2048 \
--seed 42 --num-inference-steps 50 \
--cfg-scale 4.0 \
--extra-args '{"think": true, "img_cfg_scale": 1.0, "cfg_norm": "none", "timestep_shift": 3.0}' \
--output outputs/sensenova_u1_edit.png
```
- img2img uses dual CFG: `--cfg-scale` controls text guidance, `--img-cfg-scale`
controls image guidance (1.0 = image CFG disabled).
- Pass multiple `--image` paths for multi-reference editing.
- img2img uses dual CFG: `--cfg-scale` controls text guidance, while
`img_cfg_scale` in `--extra-args` controls image guidance (1.0 = image CFG
disabled).
- Pass multiple `--image` paths for multi-reference editing when the underlying
pipeline supports them.
#### Image Understanding (img2text)
```bash
python examples/offline_inference/sensenova_u1/end2end.py \
--modality img2text \
vllm serve SenseNova/SenseNova-U1-8B-MoT --omni --port 8091
python examples/online_serving/sensenova_u1/openai_chat_client.py \
--prompt "Describe this image in detail" \
--image photo.jpg \
--max-tokens 512
--modality img2text \
--image-url photo.jpg
```
#### Text-to-Text Chat (text2text)
```bash
python examples/offline_inference/sensenova_u1/end2end.py \
--modality text2text \
vllm serve SenseNova/SenseNova-U1-8B-MoT --omni --port 8091
python examples/online_serving/sensenova_u1/openai_chat_client.py \
--prompt "Explain the theory of relativity in simple terms" \
--max-tokens 256
--modality text2text
```
- For img2text and text2text, image generation parameters (height, width,
num-steps, cfg-scale) are ignored.
- Use `--do-sample --temperature 0.7` for more diverse text responses.
- For img2text and text2text, use the online chat-compatible example. The
offline generic image examples intentionally focus on image-producing tasks.
### 2x H200 (144GB) — TP=2
@@ -149,19 +152,20 @@ python examples/offline_inference/sensenova_u1/end2end.py \
#### Command
```bash
python examples/offline_inference/sensenova_u1/end2end.py \
python examples/offline_inference/text_to_image/text_to_image.py \
--model SenseNova/SenseNova-U1-8B-MoT \
--prompt "Close portrait of an elderly woman by a farmhouse window, textured skin, gentle smile, warm natural light, emotional documentary look. The portrait should feel polished and natural, with sharp eyes, realistic skin texture, accurate facial anatomy, and premium lighting that keeps the face as the main focus." \
--width 1536 --height 2720 \
--seed 42 --num-steps 50 \
--cfg-scale 4.0 --timestep-shift 3.0 --cfg-norm none \
--think --print-think \
--seed 42 --num-inference-steps 50 \
--cfg-scale 4.0 \
--extra-body '{"think": true, "cfg_norm": "none", "timestep_shift": 3.0, "t_eps": 0.02}' \
--tensor-parallel-size 2 \
--output outputs
--output outputs/sensenova_u1_output.png
```
#### Verification
Verify the output image is generated at `outputs/sensenova_u1_output_0.png`
Verify the output image is generated at `outputs/sensenova_u1_output.png`
with the expected 1536×2720 resolution.
#### Notes
tests/diffusion/test_diffusion_output_formatter.py
+2
View File
@@ -107,6 +107,8 @@ def test_formatter_preserves_single_video_audio_actions_and_metadata(
"diffusion_engine_total_time_ms": 60.0,
"image_num": 1,
"resolution": 512,
"width": None,
"height": None,
"postprocess_time_ms": 30.0,
}
tests/e2e/offline_inference/test_sensenova_u1_img2img_expansion.py
+12 -8
View File
@@ -9,13 +9,14 @@ an input image and text prompt that match expected reference pixel values
within a ±10 tolerance.
Equivalent to running:
python examples/offline_inference/sensenova_u1/end2end.py \
python examples/offline_inference/image_to_image/image_edit.py \
--model SenseNova/SenseNova-U1-8B-MoT \
--prompt "Turn this into an oil painting" \
--image 2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg \
--width 2560 --height 1669 \
--seed 42 --num-steps 50 \
--cfg-scale 4.0 --img-cfg-scale 1.0 --cfg-norm none \
--think --print-think
--resolution 2048 \
--seed 42 --num-inference-steps 50 \
--cfg-scale 4.0 \
--extra-args '{"think": true, "img_cfg_scale": 1.0, "cfg_norm": "none", "timestep_shift": 3.0}'
"""
import os
@@ -33,11 +34,14 @@ from vllm_omni.entrypoints.omni import Omni
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
# Reference pixel data extracted from the known-good output image generated by:
# python examples/offline_inference/sensenova_u1/end2end.py \
# python examples/offline_inference/image_to_image/image_edit.py \
# --model SenseNova/SenseNova-U1-8B-MoT \
# --prompt "Turn this into an oil painting" \
# --image 2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg \
# --width 2560 --height 1669 --seed 42 --num-steps 50 \
# --cfg-scale 4.0 --img-cfg-scale 1.0 --cfg-norm none --think
# --width 2560 --height 1669 \
# --seed 42 --num-inference-steps 50 \
# --cfg-scale 4.0 \
# --extra-args '{"think": true, "img_cfg_scale": 1.0, "cfg_norm": "none", "timestep_shift": 3.0}'
REFERENCE_PIXELS = [
{"position": (100, 100), "rgb": (95, 131, 174)},
{"position": (400, 50), "rgb": (142, 153, 174)},
tests/e2e/offline_inference/test_sensenova_u1_text2img_expansion.py
+5 -3
View File
@@ -30,10 +30,12 @@ from vllm_omni.entrypoints.omni import Omni
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
# Reference pixel data extracted from the known-good output image generated by:
# python examples/offline_inference/sensenova_u1/end2end.py \
# python examples/offline_inference/text_to_image/text_to_image.py \
# --model SenseNova/SenseNova-U1-8B-MoT \
# --prompt "Close portrait of an elderly woman ..." \
# --width 1536 --height 2720 --seed 42 --num-steps 50 \
# --cfg-scale 4.0 --timestep-shift 3.0 --cfg-norm none --think
# --width 1536 --height 2720 --seed 42 --num-inference-steps 50 \
# --cfg-scale 4.0 \
# --extra-body '{"think": true, "cfg_norm": "none", "timestep_shift": 3.0, "t_eps": 0.02}'
REFERENCE_PIXELS = [
{"position": (100, 100), "rgb": (247, 249, 250)},
{"position": (768, 200), "rgb": (176, 135, 97)},
vllm_omni/diffusion/output_formatter.py
+5 -1
View File
@@ -145,7 +145,11 @@ def format_diffusion_outputs(
"diffusion_engine_exec_time_ms": timings.exec_time_s * 1000,
"diffusion_engine_total_time_ms": timings.total_time_ms,
"image_num": int(request.sampling_params.num_outputs_per_prompt),
"resolution": int(request.sampling_params.resolution),
"resolution": (
int(request.sampling_params.resolution) if request.sampling_params.resolution is not None else None
),
"width": request.sampling_params.width,
"height": request.sampling_params.height,
"postprocess_time_ms": timings.postprocess_time_s * 1000,
}
vllm_omni/diffusion/utils/image_output.py
+112
View File
@@ -0,0 +1,112 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
from collections.abc import Iterable, Mapping, Sequence
from typing import Any
import torch
from PIL import Image
from vllm.outputs import RequestOutput
from vllm_omni.outputs import OmniRequestOutput
ImageOutput = OmniRequestOutput | RequestOutput
ImageOutputs = ImageOutput | Sequence[ImageOutput] | None
def extract_images_from_outputs(outputs: ImageOutputs) -> list[Image.Image]:
"""Extract PIL images from known Omni output types.
Supported inputs:
- OmniRequestOutput / list[OmniRequestOutput]
- RequestOutput / list[RequestOutput] (compatibility fallback)
"""
for output in _iter_known_outputs(outputs):
if isinstance(output, OmniRequestOutput):
images = _coerce_images(output.images)
if images:
return images
for payload in _iter_multimodal_image_payloads(output):
images = _coerce_images(payload)
if images:
return images
return []
def _iter_known_outputs(value: ImageOutputs) -> Iterable[ImageOutput]:
if value is None:
return
if isinstance(value, OmniRequestOutput | RequestOutput):
yield value
return
if isinstance(value, Sequence):
for item in value:
if isinstance(item, OmniRequestOutput | RequestOutput):
yield item
def _iter_multimodal_image_payloads(output: OmniRequestOutput | RequestOutput) -> Iterable[Any]:
if isinstance(output, OmniRequestOutput):
if output.multimodal_output is not None:
yield from _image_values_from_mapping_like(output.multimodal_output)
if output.request_output is not None:
yield from _iter_request_output_payloads(output.request_output)
else:
yield from _iter_request_output_payloads(output)
def _iter_request_output_payloads(request_output: RequestOutput) -> Iterable[Any]:
mm = getattr(request_output, "multimodal_output", None)
if mm is not None:
yield from _image_values_from_mapping_like(mm)
for completion in getattr(request_output, "outputs", []):
mm = getattr(completion, "multimodal_output", None)
if mm is not None:
yield from _image_values_from_mapping_like(mm)
def _image_values_from_mapping_like(value: Any) -> Iterable[Any]:
if hasattr(value, "to_dict") and callable(value.to_dict):
value = value.to_dict()
if not isinstance(value, Mapping):
return
for key in ("image", "images", "model_outputs"):
if key in value:
yield value[key]
def _coerce_images(payload: Any) -> list[Image.Image]:
if payload is None:
return []
if isinstance(payload, Image.Image):
return [payload]
if isinstance(payload, torch.Tensor):
return _tensor_to_images(payload)
if isinstance(payload, list | tuple):
images: list[Image.Image] = []
for item in payload:
images.extend(_coerce_images(item))
return images
return []
def _tensor_to_images(tensor: torch.Tensor) -> list[Image.Image]:
img = tensor.detach().to("cpu", dtype=torch.float32)
if img.ndim == 4:
images: list[Image.Image] = []
for single in img:
images.extend(_tensor_to_images(single))
return images
if img.ndim != 3:
return []
if img.shape[0] in (1, 3, 4):
img = img.permute(1, 2, 0)
# Some diffusion models emit image tensors normalized in [-1, 1].
if img.min().item() < 0.0:
img = img / 2 + 0.5
img = img.clamp(0, 1).mul(255).to(torch.uint8).contiguous().numpy()
return [Image.fromarray(img)]