[Doc] [skip ci]Sync. (#1363)
Signed-off-by: Alicia <115451386+congw729@users.noreply.github.com> Co-authored-by: Yueqian Lin <70319226+linyueqian@users.noreply.github.com>
This commit is contained in:
1 parent
bc13f7cf38
commit
64f5ef615c
15 files changed
+1122
-141
No files matched your search
@@ -0,0 +1,156 @@
|
||||
# GLM-Image Multistage End-to-End Inference
|
||||
|
||||
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/offline_inference/glm_image>.
|
||||
|
||||
|
||||
This example demonstrates how to run GLM-Image with the vLLM-Omni multistage architecture.
|
||||
|
||||
## Architecture
|
||||
|
||||
GLM-Image uses a 2-stage pipeline:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ GLM-Image Pipeline │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Stage 0 (AR Model) Stage 1 (Diffusion) │
|
||||
│ ┌─────────────────┐ ┌─────────────────────┐ │
|
||||
│ │ vLLM-optimized │ │ GlmImagePipeline │ │
|
||||
│ │ GlmImageFor │ prior │ ┌───────────────┐ │ │
|
||||
│ │ Conditional │──tokens───►│ │ DiT Denoiser │ │ │
|
||||
│ │ Generation │ │ └───────────────┘ │ │
|
||||
│ │ (9B AR model) │ │ │ │ │
|
||||
│ └─────────────────┘ │ ▼ │ │
|
||||
│ ▲ │ ┌───────────────┐ │ │
|
||||
│ │ │ │ VAE Decode │──┼──► Image
|
||||
│ Text/Image │ └───────────────┘ │ │
|
||||
│ Input └─────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
- **vLLM-optimized AR**: Uses PagedAttention and tensor parallelism for faster prior token generation
|
||||
- **Flexible deployment**: AR and Diffusion stages can run on different GPUs
|
||||
- **Text-to-Image**: Generate images from text descriptions
|
||||
- **Image-to-Image**: Edit existing images with text prompts
|
||||
|
||||
## Usage
|
||||
|
||||
### Text-to-Image
|
||||
|
||||
```bash
|
||||
python end2end.py \
|
||||
--model-path /path/to/glm-image \
|
||||
--config-path ../../vllm_omni/model_executor/stage_configs/glm_image.yaml \
|
||||
--prompt "A beautiful sunset over the ocean with sailing boats" \
|
||||
--height 1024 \
|
||||
--width 1024 \
|
||||
--output output_t2i.png
|
||||
```
|
||||
|
||||
### Image-to-Image (Image Editing)
|
||||
|
||||
```bash
|
||||
python end2end.py \
|
||||
--model-path /path/to/glm-image \
|
||||
--config-path ../../vllm_omni/model_executor/stage_configs/glm_image.yaml \
|
||||
--prompt "Transform this scene into a winter wonderland" \
|
||||
--image input.png \
|
||||
--output output_i2i.png
|
||||
```
|
||||
|
||||
### With Custom Parameters
|
||||
|
||||
```bash
|
||||
python end2end.py \
|
||||
--model-path /path/to/glm-image \
|
||||
--config-path ../../vllm_omni/model_executor/stage_configs/glm_image.yaml \
|
||||
--prompt "A photorealistic cat sitting on a window sill" \
|
||||
--height 1024 \
|
||||
--width 1024 \
|
||||
--num-inference-steps 50 \
|
||||
--guidance-scale 1.5 \
|
||||
--seed 42 \
|
||||
--output output.png
|
||||
```
|
||||
|
||||
## Shell Scripts
|
||||
|
||||
### Run Text-to-Image
|
||||
|
||||
```bash
|
||||
./run_t2i.sh
|
||||
```
|
||||
|
||||
### Run Image-to-Image
|
||||
|
||||
```bash
|
||||
./run_i2i.sh --image /path/to/input.png
|
||||
```
|
||||
|
||||
## Stage Configuration
|
||||
|
||||
The stage config (`glm_image.yaml`) defines:
|
||||
|
||||
- **Stage 0 (AR)**: Uses `GPUARWorker` with vLLM engine
|
||||
|
||||
- Model: `GlmImageForConditionalGeneration`
|
||||
- Output: `token_ids` (prior tokens)
|
||||
|
||||
- **Stage 1 (Diffusion)**: Uses diffusion engine
|
||||
- Model: `GlmImagePipeline`
|
||||
- Output: Generated image
|
||||
|
||||
See `vllm_omni/model_executor/stage_configs/glm_image.yaml` for full configuration.
|
||||
|
||||
## Comparison with Single-Stage
|
||||
|
||||
| Aspect | Single-Stage (transformers) | Multistage (vLLM) |
|
||||
| ----------- | --------------------------- | ------------------- |
|
||||
| AR Model | transformers native | vLLM PagedAttention |
|
||||
| Memory | Higher (no KV cache opt) | Lower (optimized) |
|
||||
| Throughput | Lower | Higher |
|
||||
| Flexibility | Single GPU | Multi-GPU support |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### OOM Error
|
||||
|
||||
Try reducing memory usage:
|
||||
|
||||
```bash
|
||||
# In glm_image.yaml, adjust:
|
||||
gpu_memory_utilization: 0.5 # Reduce from 0.6
|
||||
```
|
||||
|
||||
### Slow Initialization
|
||||
|
||||
The first run loads model weights. Subsequent runs are faster:
|
||||
|
||||
```bash
|
||||
--stage-init-timeout 900 # Increase timeout for slow storage
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
- vLLM-Omni with GLM-Image support
|
||||
- CUDA-capable GPU (recommended: H100/A100 with 80GB)
|
||||
- GLM-Image model weights
|
||||
|
||||
## Example materials
|
||||
|
||||
??? abstract "end2end.py"
|
||||
``````py
|
||||
--8<-- "examples/offline_inference/glm_image/end2end.py"
|
||||
``````
|
||||
??? abstract "run_i2i.sh"
|
||||
``````sh
|
||||
--8<-- "examples/offline_inference/glm_image/run_i2i.sh"
|
||||
``````
|
||||
??? abstract "run_t2i.sh"
|
||||
``````sh
|
||||
--8<-- "examples/offline_inference/glm_image/run_t2i.sh"
|
||||
``````
|
||||
Reference in new issue
Block a user