[Perf] GLM Image (#920)

Signed-off-by: JaredforReal <w13431838023@gmail.com>
Signed-off-by: Jared Wen <w13431838023@gmail.com>
Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com>
Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
authored and GitHub committed 2026-02-27 10:29:14 +08:00
1 parent f6ed1bc4e7
commit 4146095623
25 files changed
+5182 -221

No files matched your search

+247
View File
@@ -0,0 +1,247 @@
# GLM-Image Online Serving
This example demonstrates how to deploy GLM-Image for online image generation using vLLM-Omni.
## 🛠️ Installation
Please refer to [README.md](../../../README.md)
## Run examples (GLM-Image)
**Note**: These examples work with the default configuration on **2× NVIDIA A100 (80GB)** or equivalent. Stage 0 (AR) and Stage 1 (Diffusion) each use one GPU by default. For single-GPU setups, modify the stage configuration to share the same device.
### Launch the Server
```bash
# Use default configuration
vllm serve zai-org/GLM-Image --omni --port 8091
```
Or use the convenience script:
```bash
cd examples/online_serving/glm_image
bash run_server.sh
```
If you have a custom stage configs file:
```bash
vllm serve zai-org/GLM-Image --omni --port 8091 --stage-configs-path /path/to/glm_image.yaml
```
### Send Requests
Get into the glm_image folder:
```bash
cd examples/online_serving/glm_image
```
Send request via Python:
```bash
python openai_chat_client.py --prompt "A cute cat sitting on a window sill"
```
The Python client supports the following command-line arguments:
- `--prompt` (or `-p`): Text prompt for generation (default: `A beautiful sunset over the ocean with sailing boats`)
- `--output` (or `-o`): Output file path (default: `glm_image_output.png`)
- `--server` (or `-s`): Server URL (default: `http://localhost:8091`)
- `--image` (or `-i`): Input image path (for image-to-image editing)
- `--height`: Image height in pixels (default: 1024)
- `--width`: Image width in pixels (default: 1024)
- `--steps`: Number of inference steps (default: 50)
- `--guidance-scale`: Classifier-free guidance scale (default: 1.5)
- `--seed`: Random seed (default: 42)
- `--negative`: Negative prompt
## Modality Control
GLM-Image supports **text-to-image** and **image-to-image** modes.
The default yaml configuration deploys AR on GPU 0 and DiT on GPU 1. You can use the default configuration file: [`glm_image.yaml`](../../../vllm_omni/model_executor/stage_configs/glm_image.yaml)
| Mode | Input | Output | Description |
| -------------- | ------------ | ------ | ---------------------------------- |
| Text-to-Image | Text | Image | Generate images from text prompts |
| Image-to-Image | Image + Text | Image | Edit images with text instructions |
### Text-to-Image
Generate images from text prompts:
**Using Python client**
```bash
python openai_chat_client.py \
--prompt "A photorealistic mountain landscape at sunset" \
--height 1024 \
--width 1024 \
--output landscape.png
```
**Using curl**
```bash
curl -s http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "user", "content": "A beautiful sunset over the ocean with sailing boats"}
],
"extra_body": {
"height": 1024,
"width": 1024,
"num_inference_steps": 50,
"guidance_scale": 1.5,
"seed": 42
}
}' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
```
Or use the script:
```bash
bash run_curl_text_to_image.sh "A futuristic city skyline at night"
```
### Image-to-Image (Image Editing)
Edit images with text instructions:
**Using Python client**
```bash
python openai_chat_client.py \
--prompt "Convert this image to watercolor style" \
--image input.png \
--output watercolor.png
```
**Using curl**
```bash
IMG_B64=$(base64 < input.png | tr -d '\n')
curl -s http://localhost:8091/v1/chat/completions \
-H "Content-Type: application/json" \
-d @- <<EOF | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
{
"messages": [{
"role": "user",
"content": [
{"type": "text", "text": "Convert this image to watercolor style"},
{"type": "image_url", "image_url": {"url": "data:image/png;base64,'$IMG_B64'"}}
]
}],
"extra_body": {
"height": 1024,
"width": 1024,
"num_inference_steps": 50,
"guidance_scale": 1.5,
"seed": 42
}
}
EOF
```
Or use the script:
```bash
bash run_curl_image_edit.sh input.png "Convert to watercolor style"
```
## Generation Parameters (extra_body)
| Parameter | Type | Default | Description |
| --------------------- | ----- | ------- | ----------------------------------- |
| `height` | int | 1024 | Image height in pixels |
| `width` | int | 1024 | Image width in pixels |
| `num_inference_steps` | int | 50 | Number of diffusion denoising steps |
| `guidance_scale` | float | 1.5 | Classifier-free guidance scale |
| `seed` | int | 42 | Random seed for reproducibility |
| `negative_prompt` | str | None | Negative prompt |
## Response Format
```json
{
"id": "chatcmpl-xxx",
"created": 1234567890,
"model": "zai-org/GLM-Image",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": [
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,..."
}
}
]
},
"finish_reason": "stop"
}
],
"usage": {}
}
```
## Extract Image
```bash
# From a saved JSON response
cat response.json | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
```
## Architecture
GLM-Image uses a 2-stage multistage pipeline:
```
Stage 0 (AR Model) Stage 1 (Diffusion)
┌───────────────────┐ ┌─────────────────────┐
│ vLLM-optimized │ prior │ GlmImagePipeline │
│ GlmImageFor │──tokens──►│ ┌───────────────┐ │
│ Conditional │ │ │ DiT Denoiser │ │
│ Generation │ │ └───────┬───────┘ │
│ (9B AR model) │ │ ▼ │
└───────────────────┘ │ ┌───────────────┐ │
▲ │ │ VAE Decode │──┼──► Image
│ │ └───────────────┘ │
Text / Image └─────────────────────┘
Input
```
## VRAM Requirements
| Stage | VRAM |
| :---------------- | :--------------------- |
| Stage-0 (AR) | **~18 GiB + KV Cache** |
| Stage-1 (DiT+VAE) | **~20 GiB** |
| Total | **~38 GiB + KV Cache** |
## File Description
| File | Description |
| --------------------------- | ------------------------------------- |
| `run_server.sh` | Server startup script |
| `run_curl_text_to_image.sh` | Text-to-image curl example |
| `run_curl_image_edit.sh` | Image-to-image (editing) curl example |
| `openai_chat_client.py` | Python client (t2i + i2i) |
## FAQ
- If you encounter OOM errors, adjust `gpu_memory_utilization` in the stage config:
```yaml
# In glm_image.yaml, reduce from default 0.6:
gpu_memory_utilization: 0.5
```
- The first request may be slow due to model warmup. Subsequent requests will be faster.
@@ -0,0 +1,172 @@
#!/usr/bin/env python3
"""
GLM-Image OpenAI-compatible chat client for text-to-image and image-to-image.
Usage:
# Text-to-image
python openai_chat_client.py --prompt "A cute cat" --output output.png
# Image-to-image (image editing)
python openai_chat_client.py --prompt "Convert to watercolor style" --image input.png --output output.png
"""
import argparse
import base64
from pathlib import Path
import requests
def generate_image(
prompt: str,
server_url: str = "http://localhost:8091",
image_path: str | None = None,
height: int = 1024,
width: int = 1024,
steps: int = 50,
guidance_scale: float = 1.5,
seed: int | None = None,
negative_prompt: str | None = None,
) -> bytes | None:
"""Generate or edit an image using the chat completions API.
Args:
prompt: Text description or editing instruction
server_url: Server URL
image_path: Path to input image (for image-to-image editing)
height: Image height in pixels
width: Image width in pixels
steps: Number of inference steps
guidance_scale: Classifier-free guidance scale
seed: Random seed for reproducibility
negative_prompt: Negative prompt
Returns:
Image bytes or None if failed
"""
# Build message content
content: list[dict] = [{"type": "text", "text": prompt}]
if image_path:
img_path = Path(image_path)
if not img_path.exists():
print(f"Error: Image file not found: {image_path}")
return None
b64_data = base64.b64encode(img_path.read_bytes()).decode("utf-8")
suffix = img_path.suffix.lstrip(".").lower()
mime = {"jpg": "jpeg", "jpeg": "jpeg", "png": "png", "webp": "webp"}.get(suffix, "png")
content.append(
{
"type": "image_url",
"image_url": {"url": f"data:image/{mime};base64,{b64_data}"},
}
)
messages = [{"role": "user", "content": content}]
# Build request payload
extra_body: dict = {
"height": height,
"width": width,
"num_inference_steps": steps,
"guidance_scale": guidance_scale,
}
if seed is not None:
extra_body["seed"] = seed
if negative_prompt:
extra_body["negative_prompt"] = negative_prompt
payload = {"messages": messages, "extra_body": extra_body}
# Send request
try:
mode = "image-to-image" if image_path else "text-to-image"
print(f"Sending {mode} request to {server_url}...")
response = requests.post(
f"{server_url}/v1/chat/completions",
headers={"Content-Type": "application/json"},
json=payload,
timeout=600,
)
response.raise_for_status()
data = response.json()
# Extract image from response
choices = data.get("choices", [])
for choice in choices:
choice_content = choice.get("message", {}).get("content")
if isinstance(choice_content, list):
for item in choice_content:
if isinstance(item, dict) and "image_url" in item:
img_url = item["image_url"].get("url", "")
if img_url.startswith("data:image"):
_, b64 = img_url.split(",", 1)
return base64.b64decode(b64)
print(f"Unexpected response format: {data}")
return None
except Exception as e:
print(f"Error: {e}")
return None
def main():
parser = argparse.ArgumentParser(description="GLM-Image chat client")
parser.add_argument(
"--prompt",
"-p",
default="A beautiful sunset over the ocean with sailing boats",
help="Text prompt",
)
parser.add_argument("--output", "-o", default="glm_image_output.png", help="Output file")
parser.add_argument("--server", "-s", default="http://localhost:8091", help="Server URL")
# Image-to-image
parser.add_argument(
"--image",
"-i",
type=str,
help="Input image path (for image-to-image editing)",
)
# Generation parameters
parser.add_argument("--height", type=int, default=1024, help="Image height")
parser.add_argument("--width", type=int, default=1024, help="Image width")
parser.add_argument("--steps", type=int, default=50, help="Inference steps")
parser.add_argument("--guidance-scale", type=float, default=1.5, help="CFG guidance scale")
parser.add_argument("--seed", type=int, default=42, help="Random seed")
parser.add_argument("--negative", help="Negative prompt")
args = parser.parse_args()
mode = "image-to-image" if args.image else "text-to-image"
print(f"Mode: {mode}")
print(f"Prompt: {args.prompt}")
if args.image:
print(f"Input image: {args.image}")
image_bytes = generate_image(
prompt=args.prompt,
server_url=args.server,
image_path=args.image,
height=args.height,
width=args.width,
steps=args.steps,
guidance_scale=args.guidance_scale,
seed=args.seed,
negative_prompt=args.negative,
)
if image_bytes:
output_path = Path(args.output)
output_path.write_bytes(image_bytes)
print(f"Image saved to: {output_path}")
print(f"Size: {len(image_bytes) / 1024:.1f} KB")
else:
print("Failed to generate image")
exit(1)
if __name__ == "__main__":
main()
+61
View File
@@ -0,0 +1,61 @@
#!/bin/bash
# GLM-Image image-edit (image-to-image) curl example
set -euo pipefail
if [[ $# -lt 2 ]]; then
echo "Usage: $0 <input_image> \"<edit_prompt>\" [output_file]" >&2
exit 1
fi
INPUT_IMG=$1
PROMPT=$2
SERVER="${SERVER:-http://localhost:8091}"
CURRENT_TIME=$(date +%Y%m%d%H%M%S)
OUTPUT="${3:-glm_image_i2i_${CURRENT_TIME}.png}"
if [[ ! -f "$INPUT_IMG" ]]; then
echo "Input image not found: $INPUT_IMG" >&2
exit 1
fi
# base64 encode (macOS uses -i, Linux uses -w0)
if [[ "$(uname)" == "Darwin" ]]; then
IMG_B64=$(base64 < "$INPUT_IMG" | tr -d '\n')
else
IMG_B64=$(base64 -w0 "$INPUT_IMG")
fi
REQUEST_JSON=$(
jq -n --arg prompt "$PROMPT" --arg img "$IMG_B64" '{
messages: [{
role: "user",
content: [
{"type": "text", "text": $prompt},
{"type": "image_url", "image_url": {"url": ("data:image/png;base64," + $img)}}
]
}],
extra_body: {
height: 1024,
width: 1024,
num_inference_steps: 50,
guidance_scale: 1.5,
seed: 42
}
}'
)
echo "Generating edited image..."
echo "Server: $SERVER"
echo "Prompt: $PROMPT"
echo "Input : $INPUT_IMG"
echo "Output: $OUTPUT"
curl -s "$SERVER/v1/chat/completions" \
-H "Content-Type: application/json" \
-d "$REQUEST_JSON" \
| jq -r '.choices[0].message.content[0].image_url.url' \
| cut -d',' -f2- \
| base64 -d > "$OUTPUT"
echo "Image saved to: $OUTPUT"
@@ -0,0 +1,30 @@
#!/bin/bash
# GLM-Image text-to-image curl example
set -euo pipefail
PROMPT="${1:-A beautiful sunset over the ocean with sailing boats}"
SERVER="${SERVER:-http://localhost:8091}"
OUTPUT="${OUTPUT:-glm_image_t2i_output.png}"
echo "Generating image..."
echo "Server: $SERVER"
echo "Prompt: $PROMPT"
echo "Output: $OUTPUT"
curl -s "$SERVER/v1/chat/completions" \
-H "Content-Type: application/json" \
-d "{
\"messages\": [
{\"role\": \"user\", \"content\": \"$PROMPT\"}
],
\"extra_body\": {
\"height\": 1024,
\"width\": 1024,
\"num_inference_steps\": 50,
\"guidance_scale\": 1.5,
\"seed\": 42
}
}" | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > "$OUTPUT"
echo "Image saved to: $OUTPUT"
+12
View File
@@ -0,0 +1,12 @@
#!/bin/bash
# GLM-Image online serving startup script
MODEL="${MODEL:-zai-org/GLM-Image}"
PORT="${PORT:-8091}"
echo "Starting GLM-Image server..."
echo "Model: $MODEL"
echo "Port: $PORT"
vllm serve "$MODEL" --omni \
--port "$PORT"