2026-02-27 17:28:35 +08:00
# GLM-Image Online Serving
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/glm_image>.
This example demonstrates how to deploy GLM-Image for online image generation using vLLM-Omni.
## 🛠️ Installation
Please refer to [README.md ](https://github.com/vllm-project/vllm-omni/tree/main/README.md )
## Run examples (GLM-Image)
**Note** : These examples work with the default configuration on **2× or equivalent. Stage 0 (AR) and Stage 1 (Diffusion) each use one GPU by default. For single-GPU setups, modify the stage configuration to share the same device.
### Launch the Server
```bash
# Use default configuration
vllm serve zai-org/GLM-Image --omni --port 8091
```
Or use the convenience script:
```bash
cd examples/online_serving/glm_image
bash run_server.sh
```
If you have a custom stage configs file:
```bash
vllm serve zai-org/GLM-Image --omni --port 8091 --stage-configs-path /path/to/glm_image.yaml
```
### Send Requests
Get into the glm_image folder:
```bash
cd examples/online_serving/glm_image
```
Send request via Python:
```bash
python openai_chat_client.py --prompt "A cute cat sitting on a window sill"
```
The Python client supports the following command-line arguments:
- `--prompt` (or `-p` ): Text prompt for generation (default: `A beautiful sunset over the ocean with sailing boats` )
- `--output` (or `-o` ): Output file path (default: `glm_image_output.png` )
- `--server` (or `-s` ): Server URL (default: `http://localhost:8091` )
- `--image` (or `-i` ): Input image path (for image-to-image editing)
- `--height` : Image height in pixels (default: 1024)
- `--width` : Image width in pixels (default: 1024)
- `--steps` : Number of inference steps (default: 50)
- `--guidance-scale` : Classifier-free guidance scale (default: 1.5)
- `--seed` : Random seed (default: 42)
- `--negative` : Negative prompt
## Modality Control
GLM-Image supports **text-to-image** and **image-to-image** modes.
The default yaml configuration deploys AR on GPU 0 and DiT on GPU 1. You can use the default configuration file: [`glm_image.yaml` ](https://github.com/vllm-project/vllm-omni/tree/main/vllm_omni/model_executor/stage_configs/glm_image.yaml )
| Mode | Input | Output | Description |
| -------------- | ------------ | ------ | ---------------------------------- |
| Text-to-Image | Text | Image | Generate images from text prompts |
| Image-to-Image | Image + Text | Image | Edit images with text instructions |
### Text-to-Image
```bash
python openai_chat_client.py \
--prompt "A photorealistic mountain landscape at sunset" \
--height 1024 \
--width 1024 \
--output landscape.png
2026-03-26 23:12:16 +08:00
# Or use the curl script:
2026-02-27 17:28:35 +08:00
bash run_curl_text_to_image.sh "A futuristic city skyline at night"
```
### Image-to-Image (Image Editing)
```bash
python openai_chat_client.py \
--prompt "Convert this image to watercolor style" \
--image input.png \
--output watercolor.png
2026-03-26 23:12:16 +08:00
# Or use the curl script:
2026-02-27 17:28:35 +08:00
bash run_curl_image_edit.sh input.png "Convert to watercolor style"
```
2026-03-26 23:12:16 +08:00
For general-purpose request methods (curl, OpenAI SDK, Python `requests` ), see
the [Text-to-Image ](text_to_image.md ) and [Image-to-Image ](image_to_image.md )
guides.
## Generation Parameters
When using `/v1/chat/completions` , pass these inside `extra_body` in the curl
JSON, or via the `extra_body` keyword argument in the OpenAI Python SDK (see the
[Diffusion Chat API guide ](../../../../serving/diffusion_chat_api.md )).
When using the dedicated [`/v1/images/generations` ](../../../../serving/image_generation_api.md )
or [`/v1/images/edits` ](../../../../serving/image_edit_api.md ) endpoints, pass
the supported generation controls as top-level fields directly. For image
dimensions and count, use `size` and `n` rather than `height` or `width` .
2026-02-27 17:28:35 +08:00
| Parameter | Type | Default | Description |
| --------------------- | ----- | ------- | ----------------------------------- |
| `height` | int | 1024 | Image height in pixels |
| `width` | int | 1024 | Image width in pixels |
| `num_inference_steps` | int | 50 | Number of diffusion denoising steps |
| `guidance_scale` | float | 1.5 | Classifier-free guidance scale |
2026-03-26 23:12:16 +08:00
| `seed` | int | None | Optional random seed; `/v1/images/*` generates one server-side if omitted |
2026-02-27 17:28:35 +08:00
| `negative_prompt` | str | None | Negative prompt |
## Response Format
```json
{
"id" : "chatcmpl-xxx" ,
"created" : 1234567890 ,
"model" : "zai-org/GLM-Image" ,
"choices" : [
{
"index" : 0 ,
"message" : {
"role" : "assistant" ,
"content" : [
{
"type" : "image_url" ,
"image_url" : {
"url" : "data:image/png;base64,..."
}
}
]
},
"finish_reason" : "stop"
}
],
"usage" : {}
}
```
## Extract Image
```bash
# From a saved JSON response
cat response.json | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2- | base64 -d > output.png
```
## Architecture
GLM-Image uses a 2-stage multistage pipeline:
```
Stage 0 (AR Model) Stage 1 (Diffusion)
┌───────────────────┐ ┌─────────────────────┐
│ vLLM-optimized │ prior │ GlmImagePipeline │
│ GlmImageFor │──tokens──►│ ┌───────────────┐ │
│ Conditional │ │ │ DiT Denoiser │ │
│ Generation │ │ └───────┬───────┘ │
│ (9B AR model) │ │ ▼ │
└───────────────────┘ │ ┌───────────────┐ │
▲ │ │ VAE Decode │──┼──► Image
│ │ └───────────────┘ │
Text / Image └─────────────────────┘
Input
```
## VRAM Requirements
| Stage | VRAM |
| :---------------- | :--------------------- |
| Stage-0 (AR) | ** ~18 GiB + KV Cache** |
| Stage-1 (DiT+VAE) | ** ~20 GiB** |
| Total | ** ~38 GiB + KV Cache** |
## File Description
| File | Description |
| --------------------------- | ------------------------------------- |
| `run_server.sh` | Server startup script |
| `run_curl_text_to_image.sh` | Text-to-image curl example |
| `run_curl_image_edit.sh` | Image-to-image (editing) curl example |
| `openai_chat_client.py` | Python client (t2i + i2i) |
## FAQ
- If you encounter OOM errors, adjust `gpu_memory_utilization` in the stage config:
```yaml
# In glm_image.yaml, reduce from default 0.6:
gpu_memory_utilization : 0.5
```
- The first request may be slow due to model warmup. Subsequent requests will be faster.
## Example materials
??? abstract "openai_chat_client.py"
``````py
--8<-- "examples/online_serving/glm_image/openai_chat_client.py"
` `````
??? abstract "run_curl_image_edit.sh"
` `````sh
--8<-- "examples/online_serving/glm_image/run_curl_image_edit.sh"
` `````
??? abstract "run_curl_text_to_image.sh"
` `````sh
--8<-- "examples/online_serving/glm_image/run_curl_text_to_image.sh"
` `````
??? abstract "run_server.sh"
` `````sh
--8<-- "examples/online_serving/glm_image/run_server.sh"
` `````
