anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Doc] Improve diffusion generation parameter docs for online serving (#2051)

Browse Source 
Signed-off-by: samithuang <285365963@qq.com>
Signed-off-by: Samit <285365963@qq.com>
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
Co-authored-by: Zeyu Huang | 黃澤宇 <11222265+fhfuih@users.noreply.github.com>
Co-authored-by: gcanlin <canlinguosdu@gmail.com>
This commit is contained in:
SamitZeyu Huang | 黃澤宇gcanlin
 authored and GitHub committed 2026-03-26 15:12:16 +00:00
1 parent 574ec99ef4
commit d96d63cb7f
8 files changed
+568 -167
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/user_guide/examples/online_serving/glm_image.md
+16 -64
View File
@@ -73,90 +73,42 @@ The default yaml configuration deploys AR on GPU 0 and DiT on GPU 1. You can use
### Text-to-Image
Generate images from text prompts:
**Using Python client**
```bash
python openai_chat_client.py \
--prompt "A photorealistic mountain landscape at sunset" \
--height 1024 \
--width 1024 \
--output landscape.png
```
**Using curl**
```bash
curl -s http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "user", "content": "A beautiful sunset over the ocean with sailing boats"}
],
"extra_body": {
"height": 1024,
"width": 1024,
"num_inference_steps": 50,
"guidance_scale": 1.5,
"seed": 42
}
}' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
```
Or use the script:
```bash
# Or use the curl script:
bash run_curl_text_to_image.sh "A futuristic city skyline at night"
```
### Image-to-Image (Image Editing)
Edit images with text instructions:
**Using Python client**
```bash
python openai_chat_client.py \
--prompt "Convert this image to watercolor style" \
--image input.png \
--output watercolor.png
```
**Using curl**
```bash
IMG_B64=$(base64 < input.png | tr -d '\n')
curl -s http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d @- <<EOF | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
{
"messages": [{
"role": "user",
"content": [
{"type": "text", "text": "Convert this image to watercolor style"},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,'$IMG_B64'"}}
]
}],
"extra_body": {
"height": 1024,
"width": 1024,
"num_inference_steps": 50,
"guidance_scale": 1.5,
"seed": 42
}
}
EOF
```
Or use the script:
```bash
# Or use the curl script:
bash run_curl_image_edit.sh input.png "Convert to watercolor style"
```
## Generation Parameters (extra_body)
For general-purpose request methods (curl, OpenAI SDK, Python `requests`), see
the [Text-to-Image](text_to_image.md) and [Image-to-Image](image_to_image.md)
guides.
## Generation Parameters
When using `/v1/chat/completions`, pass these inside `extra_body` in the curl
JSON, or via the `extra_body` keyword argument in the OpenAI Python SDK (see the
[Diffusion Chat API guide](../../../../serving/diffusion_chat_api.md)).
When using the dedicated [`/v1/images/generations`](../../../../serving/image_generation_api.md)
or [`/v1/images/edits`](../../../../serving/image_edit_api.md) endpoints, pass
the supported generation controls as top-level fields directly. For image
dimensions and count, use `size` and `n` rather than `height` or `width`.
| Parameter | Type | Default | Description |
| --------------------- | ----- | ------- | ----------------------------------- |
@@ -164,7 +116,7 @@ bash run_curl_image_edit.sh input.png "Convert to watercolor style"
| `width` | int | 1024 | Image width in pixels |
| `num_inference_steps` | int | 50 | Number of diffusion denoising steps |
| `guidance_scale` | float | 1.5 | Classifier-free guidance scale |
| `seed` | int | 42 | Random seed for reproducibility |
| `seed` | int | None | Optional random seed; `/v1/images/*` generates one server-side if omitted |
| `negative_prompt` | str | None | Negative prompt |
## Response Format
docs/user_guide/examples/online_serving/image_to_image.md
+198 -6
View File
@@ -69,10 +69,49 @@ cat <<EOF > request.json
}
EOF
curl -s http://localhost:8092/v1/chat/completions -H "Content-Type: application/json" -d @request.json | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2 | base64 -d > output.png
curl -s http://localhost:8092/v1/chat/completions \
-H "Content-Type: application/json" \
-d @request.json \
| jq -r '.choices[0].message.content[0].image_url.url' \
| cut -d',' -f2 | base64 -d > output.png
```
### Method 2: Using Python Client
### Method 2: Using OpenAI Python SDK
```python
import base64
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8092/v1", api_key="none")
with open("input.png", "rb") as f:
img_b64 = base64.b64encode(f.read()).decode()
response = client.chat.completions.create(
model="Qwen/Qwen-Image-Edit",
messages=[{
"role": "user",
"content": [
{"type": "text", "text": "Convert to watercolor style"},
{"type": "image_url", "image_url": {
"url": f"data:image/png;base64,{img_b64}"
}},
],
}],
extra_body={
"num_inference_steps": 50,
"guidance_scale": 1,
"seed": 42,
},
)
img_url = response.choices[0].message.content[0].image_url.url
_, b64_data = img_url.split(",", 1)
with open("output.png", "wb") as f:
f.write(base64.b64decode(b64_data))
```
### Method 3: Using Python Client Script
```bash
python openai_chat_client.py --input input.png --prompt "Convert to oil painting style" --output output.png
@@ -81,7 +120,7 @@ python openai_chat_client.py --input input.png --prompt "Convert to oil painting
python openai_chat_client.py --input input1.png input2.png --prompt "Combine these images into a single scene" --output output.png
```
### Method 3: Using Gradio Demo
### Method 4: Using Gradio Demo
```bash
python gradio_demo.py
@@ -124,7 +163,7 @@ python gradio_demo.py
### Image Editing with Parameters
Use `extra_body` to pass generation parameters:
Wrap generation parameters inside `extra_body` in the request JSON:
```json
{
@@ -147,6 +186,149 @@ Use `extra_body` to pass generation parameters:
}
```
!!! tip "Using the OpenAI SDK"
When using the OpenAI Python SDK, pass these parameters via the `extra_body`
keyword argument. The SDK merges them into the top-level request body automatically:
```python
client.chat.completions.create(
model="Qwen/Qwen-Image-Edit",
messages=[...],
extra_body={"num_inference_steps": 50, "guidance_scale": 7.5, "seed": 42},
)
```
For details on how generation parameters are handled across different clients, see the
[Diffusion Chat API guide](../../../../serving/diffusion_chat_api.md).
### Layered Image Generation (Qwen-Image-Layered)
Qwen-Image-Layered generates multiple decomposed layers from a reference image and a text prompt.
Start the server with:
```bash
vllm serve Qwen/Qwen-Image-Layered --omni --port 8093
```
=== "curl"
```bash
IMG_B64=$(base64 -w0 input.png)
curl -sS http://localhost:8093/v1/chat/completions \
-H "Content-Type: application/json" \
-d "$(jq -n --arg img "$IMG_B64" '{
messages: [{
role: "user",
content: [
{type: "image_url", image_url: {url: ("data:image/png;base64," + $img)}},
{type: "text", text: "a rabbit"}
]
}],
extra_body: {
num_inference_steps: 50,
cfg_scale: 4.0,
seed: 0,
layers: 4,
resolution: 640
}
}')" \
| jq -r '.choices[0].message.content[] | .image_url.url | split(",")[1]' \
| while IFS= read -r b64; do
((i++)); echo "$b64" | base64 -d > "layer_${i}.png"
done
```
=== "OpenAI SDK"
```python
import base64
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8093/v1", api_key="none")
with open("input.png", "rb") as f:
img_b64 = base64.b64encode(f.read()).decode()
response = client.chat.completions.create(
model="Qwen/Qwen-Image-Layered",
messages=[{
"role": "user",
"content": [
{"type": "image_url", "image_url": {
"url": f"data:image/png;base64,{img_b64}"
}},
{"type": "text", "text": "a rabbit"},
],
}],
extra_body={
"num_inference_steps": 50,
"cfg_scale": 4.0,
"seed": 0,
"layers": 4,
"resolution": 640,
},
)
for i, item in enumerate(response.choices[0].message.content):
_, b64_data = item.image_url.url.split(",", 1)
with open(f"layer_{i}.png", "wb") as f:
f.write(base64.b64decode(b64_data))
```
=== "Python requests"
```python
import base64
import requests
with open("input.png", "rb") as f:
img_b64 = base64.b64encode(f.read()).decode()
payload = {
"messages": [{
"role": "user",
"content": [
{"type": "image_url", "image_url": {
"url": f"data:image/png;base64,{img_b64}"
}},
{"type": "text", "text": "a rabbit"},
],
}],
"extra_body": {
"num_inference_steps": 50,
"cfg_scale": 4.0,
"seed": 0,
"layers": 4,
"resolution": 640,
},
}
resp = requests.post(
"http://localhost:8093/v1/chat/completions",
json=payload,
timeout=600,
)
data = resp.json()
for i, item in enumerate(data["choices"][0]["message"]["content"]):
_, b64_data = item["image_url"]["url"].split(",", 1)
with open(f"layer_{i}.png", "wb") as f:
f.write(base64.b64decode(b64_data))
```
The response contains multiple images in `choices[0].message.content` — one per generated layer.
#### Qwen-Image-Layered Parameters
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `layers` | int | 4 | Number of layers to decompose |
| `resolution` | int | 640 | Resolution for dimension calculation (640 or 1024) |
| `cfg_scale` | float | 4.0 | Classifier-free guidance scale (alias for `true_cfg_scale`) |
| `num_inference_steps` | int | 50 | Number of denoising steps |
| `seed` | int | None | Random seed for reproducibility |
### Multi-Image Editing (Qwen-Image-Edit-2509)
Provide multiple images in `content` (order matters):
@@ -166,7 +348,15 @@ Provide multiple images in `content` (order matters):
}
```
## Generation Parameters (extra_body)
## Generation Parameters
When using `/v1/chat/completions`, pass these inside `extra_body` in the curl
JSON, or via the `extra_body` keyword argument in the OpenAI Python SDK (see the
[Diffusion Chat API guide](../../../../serving/diffusion_chat_api.md)).
When using the dedicated [`/v1/images/edits`](../../../../serving/image_edit_api.md)
endpoint, pass the supported generation controls as top-level form fields
directly. For image dimensions and count, use `size` and `n` rather than
`height`, `width`, or `num_outputs_per_prompt`.
| Parameter | Type | Default | Description |
| ------------------------ | ----- | ------- | ------------------------------------- |
@@ -174,10 +364,12 @@ Provide multiple images in `content` (order matters):
| `width` | int | None | Output image width in pixels |
| `size` | str | None | Output image size (e.g., "1024x1024") |
| `num_inference_steps` | int | 50 | Number of denoising steps |
| `guidance_scale` | float | 7.5 | CFG guidance scale |
| `guidance_scale` | float | 1.0 | CFG guidance scale |
| `seed` | int | None | Random seed (reproducible) |
| `negative_prompt` | str | None | Negative prompt |
| `num_outputs_per_prompt` | int | 1 | Number of images to generate |
| `layers` | int | 4 | Number of layers (Qwen-Image-Layered) |
| `resolution` | int | 640 | Resolution, 640 or 1024 (Qwen-Image-Layered) |
## Response Format
docs/user_guide/examples/online_serving/text_to_image.md
+63 -15
View File
@@ -71,13 +71,39 @@ curl -s http://localhost:8091/v1/chat/completions \
}' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
```
### Method 2: Using Python Client
### Method 2: Using OpenAI Python SDK
```python
from openai import OpenAI
import base64
client = OpenAI(base_url="http://localhost:8091/v1", api_key="none")
response = client.chat.completions.create(
model="Qwen/Qwen-Image",
messages=[{"role": "user", "content": "A beautiful landscape painting"}],
extra_body={
"height": 1024,
"width": 1024,
"num_inference_steps": 50,
"true_cfg_scale": 4.0,
"seed": 42,
},
)
img_url = response.choices[0].message.content[0].image_url.url
_, b64_data = img_url.split(",", 1)
with open("output.png", "wb") as f:
f.write(base64.b64decode(b64_data))
```
### Method 3: Using Python Client Script
```bash
python openai_chat_client.py --prompt "A beautiful landscape painting" --output output.png
```
### Method 3: Using Gradio Demo
### Method 4: Using Gradio Demo
```bash
python gradio_demo.py
@@ -151,7 +177,7 @@ lora_adapter/
### Generation with Parameters
Use `extra_body` to pass generation parameters:
Wrap generation parameters inside `extra_body` in the request JSON:
```json
{
@@ -168,6 +194,21 @@ Use `extra_body` to pass generation parameters:
}
```
!!! tip "Using the OpenAI SDK"
When using the OpenAI Python SDK, pass these parameters via the `extra_body`
keyword argument. The SDK merges them into the top-level request body automatically:
```python
client.chat.completions.create(
model="Qwen/Qwen-Image",
messages=[...],
extra_body={"height": 1024, "width": 1024, "num_inference_steps": 50},
)
```
For details on how generation parameters are handled across different clients, see the
[Diffusion Chat API guide](../../../../serving/diffusion_chat_api.md).
### Multimodal Input (Text + Structured Content)
```json
@@ -183,19 +224,26 @@ Use `extra_body` to pass generation parameters:
}
```
## Generation Parameters (extra_body)
## Generation Parameters
| Parameter | Type | Default | Description |
| ------------------------ | ----- | ------- | ---------------------------------- |
| `height` | int | None | Image height in pixels |
| `width` | int | None | Image width in pixels |
| `size` | str | None | Image size (e.g., "1024x1024") |
| `num_inference_steps` | int | 50 | Number of denoising steps |
| `true_cfg_scale` | float | 4.0 | Qwen-Image CFG scale |
| `seed` | int | None | Random seed (reproducible) |
| `negative_prompt` | str | None | Negative prompt |
| `num_outputs_per_prompt` | int | 1 | Number of images to generate |
| `--cfg-parallel-size`. | int | 1 | Number of GPUs for CFG parallelism |
When using `/v1/chat/completions`, pass these inside `extra_body` in the curl
JSON, or via the `extra_body` keyword argument in the OpenAI Python SDK (see the
[Diffusion Chat API guide](../../../../serving/diffusion_chat_api.md)).
When using the dedicated [`/v1/images/generations`](../../../../serving/image_generation_api.md)
endpoint, pass the supported generation controls as top-level JSON fields
directly. For image dimensions and count, use `size` and `n` rather than
`height`, `width`, or `num_outputs_per_prompt`.
| Parameter | Type | Default | Description |
| ------------------------ | ----- | ------- | ------------------------------ |
| `height` | int | None | Image height in pixels |
| `width` | int | None | Image width in pixels |
| `size` | str | None | Image size (e.g., "1024x1024") |
| `num_inference_steps` | int | 50 | Number of denoising steps |
| `true_cfg_scale` | float | 4.0 | Qwen-Image CFG scale |
| `seed` | int | None | Random seed (reproducible) |
| `negative_prompt` | str | None | Negative prompt |
| `num_outputs_per_prompt` | int | 1 | Number of images to generate |
## Response Format