[Perf] GLM Image (#920)
Signed-off-by: JaredforReal <w13431838023@gmail.com> Signed-off-by: Jared Wen <w13431838023@gmail.com> Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com> Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
25 files changed
+5182
-221
No files matched your search
@@ -0,0 +1,138 @@
|
||||
# GLM-Image Multistage End-to-End Inference
|
||||
|
||||
This example demonstrates how to run GLM-Image with the vLLM-Omni multistage architecture.
|
||||
|
||||
## Architecture
|
||||
|
||||
GLM-Image uses a 2-stage pipeline:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ GLM-Image Pipeline │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ Stage 0 (AR Model) Stage 1 (Diffusion) │
|
||||
│ ┌─────────────────┐ ┌─────────────────────┐ │
|
||||
│ │ vLLM-optimized │ │ GlmImagePipeline │ │
|
||||
│ │ GlmImageFor │ prior │ ┌───────────────┐ │ │
|
||||
│ │ Conditional │──tokens───►│ │ DiT Denoiser │ │ │
|
||||
│ │ Generation │ │ └───────────────┘ │ │
|
||||
│ │ (9B AR model) │ │ │ │ │
|
||||
│ └─────────────────┘ │ ▼ │ │
|
||||
│ ▲ │ ┌───────────────┐ │ │
|
||||
│ │ │ │ VAE Decode │──┼──► Image
|
||||
│ Text/Image │ └───────────────┘ │ │
|
||||
│ Input └─────────────────────┘ │
|
||||
│ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
- **vLLM-optimized AR**: Uses PagedAttention and tensor parallelism for faster prior token generation
|
||||
- **Flexible deployment**: AR and Diffusion stages can run on different GPUs
|
||||
- **Text-to-Image**: Generate images from text descriptions
|
||||
- **Image-to-Image**: Edit existing images with text prompts
|
||||
|
||||
## Usage
|
||||
|
||||
### Text-to-Image
|
||||
|
||||
```bash
|
||||
python end2end.py \
|
||||
--model-path /path/to/glm-image \
|
||||
--config-path ../../vllm_omni/model_executor/stage_configs/glm_image.yaml \
|
||||
--prompt "A beautiful sunset over the ocean with sailing boats" \
|
||||
--height 1024 \
|
||||
--width 1024 \
|
||||
--output output_t2i.png
|
||||
```
|
||||
|
||||
### Image-to-Image (Image Editing)
|
||||
|
||||
```bash
|
||||
python end2end.py \
|
||||
--model-path /path/to/glm-image \
|
||||
--config-path ../../vllm_omni/model_executor/stage_configs/glm_image.yaml \
|
||||
--prompt "Transform this scene into a winter wonderland" \
|
||||
--image input.png \
|
||||
--output output_i2i.png
|
||||
```
|
||||
|
||||
### With Custom Parameters
|
||||
|
||||
```bash
|
||||
python end2end.py \
|
||||
--model-path /path/to/glm-image \
|
||||
--config-path ../../vllm_omni/model_executor/stage_configs/glm_image.yaml \
|
||||
--prompt "A photorealistic cat sitting on a window sill" \
|
||||
--height 1024 \
|
||||
--width 1024 \
|
||||
--num-inference-steps 50 \
|
||||
--guidance-scale 1.5 \
|
||||
--seed 42 \
|
||||
--output output.png
|
||||
```
|
||||
|
||||
## Shell Scripts
|
||||
|
||||
### Run Text-to-Image
|
||||
|
||||
```bash
|
||||
./run_t2i.sh
|
||||
```
|
||||
|
||||
### Run Image-to-Image
|
||||
|
||||
```bash
|
||||
./run_i2i.sh --image /path/to/input.png
|
||||
```
|
||||
|
||||
## Stage Configuration
|
||||
|
||||
The stage config (`glm_image.yaml`) defines:
|
||||
|
||||
- **Stage 0 (AR)**: Uses `GPUARWorker` with vLLM engine
|
||||
|
||||
- Model: `GlmImageForConditionalGeneration`
|
||||
- Output: `token_ids` (prior tokens)
|
||||
|
||||
- **Stage 1 (Diffusion)**: Uses diffusion engine
|
||||
- Model: `GlmImagePipeline`
|
||||
- Output: Generated image
|
||||
|
||||
See `vllm_omni/model_executor/stage_configs/glm_image.yaml` for full configuration.
|
||||
|
||||
## Comparison with Single-Stage
|
||||
|
||||
| Aspect | Single-Stage (transformers) | Multistage (vLLM) |
|
||||
| ----------- | --------------------------- | ------------------- |
|
||||
| AR Model | transformers native | vLLM PagedAttention |
|
||||
| Memory | Higher (no KV cache opt) | Lower (optimized) |
|
||||
| Throughput | Lower | Higher |
|
||||
| Flexibility | Single GPU | Multi-GPU support |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### OOM Error
|
||||
|
||||
Try reducing memory usage:
|
||||
|
||||
```bash
|
||||
# In glm_image.yaml, adjust:
|
||||
gpu_memory_utilization: 0.5 # Reduce from 0.6
|
||||
```
|
||||
|
||||
### Slow Initialization
|
||||
|
||||
The first run loads model weights. Subsequent runs are faster:
|
||||
|
||||
```bash
|
||||
--stage-init-timeout 900 # Increase timeout for slow storage
|
||||
```
|
||||
|
||||
## Requirements
|
||||
|
||||
- vLLM-Omni with GLM-Image support
|
||||
- CUDA-capable GPU (recommended: H100/A100 with 80GB)
|
||||
- GLM-Image model weights
|
||||
@@ -0,0 +1,476 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
|
||||
"""
|
||||
End-to-end offline inference example for GLM-Image with multistage architecture.
|
||||
|
||||
This script tests the multistage pipeline where:
|
||||
- Stage 0 (AR): vLLM-optimized GlmImageForConditionalGeneration generates prior_token_ids
|
||||
- Stage 1 (Diffusion): GlmImagePipeline performs DiT denoising + VAE decode
|
||||
|
||||
Usage (text-to-image):
|
||||
python end2end.py \
|
||||
--model-path /path/to/glm-image \
|
||||
--config-path /path/to/glm_image.yaml \
|
||||
--prompt "A beautiful sunset over the ocean" \
|
||||
--output output_t2i.png
|
||||
|
||||
Usage (image-to-image / image edit):
|
||||
python end2end.py \
|
||||
--model-path /path/to/glm-image \
|
||||
--config-path /path/to/glm_image.yaml \
|
||||
--prompt "Make it look like winter" \
|
||||
--image input.png \
|
||||
--output output_i2i.png
|
||||
|
||||
Usage (with custom parameters):
|
||||
python end2end.py \
|
||||
--model-path /path/to/glm-image \
|
||||
--config-path /path/to/glm_image.yaml \
|
||||
--prompt "A cat sitting on a window sill" \
|
||||
--height 1024 \
|
||||
--width 1024 \
|
||||
--num-inference-steps 50 \
|
||||
--guidance-scale 1.5 \
|
||||
--seed 42
|
||||
|
||||
For more options, run:
|
||||
python end2end.py --help
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from vllm_omni.entrypoints.omni import Omni
|
||||
from vllm_omni.inputs.data import OmniDiffusionSamplingParams
|
||||
|
||||
# Default stage config path (relative to vllm_omni package)
|
||||
DEFAULT_CONFIG_PATH = "vllm_omni/model_executor/stage_configs/glm_image.yaml"
|
||||
|
||||
SEED = 42
|
||||
|
||||
# GLM-Image special tokens
|
||||
GLM_IMAGE_EOS_TOKEN_ID = 16385 # eos_token_id from generation_config.json
|
||||
GLM_IMAGE_VISION_VOCAB_SIZE = 16512 # top_k should be vision_vocab_size
|
||||
|
||||
|
||||
def compute_max_tokens(height: int, width: int, factor: int = 32) -> int:
|
||||
"""
|
||||
Compute max_new_tokens for GLM-Image AR generation.
|
||||
|
||||
GLM-Image generates tokens in this order for text-to-image:
|
||||
1. Small preview image (half resolution in each dimension)
|
||||
2. Large target image (full resolution)
|
||||
3. EOS token
|
||||
|
||||
Args:
|
||||
height: Target image height in pixels
|
||||
width: Target image width in pixels
|
||||
factor: Downsampling factor (32 for GLM-Image AR output)
|
||||
|
||||
Returns:
|
||||
Total number of tokens to generate (small + large + EOS)
|
||||
"""
|
||||
# Large image tokens (target resolution)
|
||||
token_h = height // factor
|
||||
token_w = width // factor
|
||||
large_tokens = token_h * token_w
|
||||
|
||||
# Small preview tokens (half resolution in each dimension)
|
||||
small_h = token_h // 2
|
||||
small_w = token_w // 2
|
||||
small_tokens = small_h * small_w
|
||||
|
||||
# Total: small + large + EOS
|
||||
return small_tokens + large_tokens + 1
|
||||
|
||||
|
||||
def load_image(image_path: str) -> Image.Image:
|
||||
"""Load an image from file path."""
|
||||
if not os.path.exists(image_path):
|
||||
raise FileNotFoundError(f"Image file not found: {image_path}")
|
||||
return Image.open(image_path).convert("RGB")
|
||||
|
||||
|
||||
def save_image(image: Image.Image, output_path: str) -> None:
|
||||
"""Save an image to file path."""
|
||||
output_dir = os.path.dirname(output_path)
|
||||
if output_dir:
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
image.save(output_path)
|
||||
print(f"Image saved to: {output_path}")
|
||||
|
||||
|
||||
def build_prompt_for_t2i(
|
||||
prompt: str,
|
||||
height: int = 1024,
|
||||
width: int = 1024,
|
||||
) -> dict:
|
||||
"""
|
||||
Build prompt dict for text-to-image generation.
|
||||
|
||||
Args:
|
||||
prompt: Text description for image generation
|
||||
height: Target image height
|
||||
width: Target image width
|
||||
|
||||
Returns:
|
||||
Dict containing prompt and generation parameters
|
||||
"""
|
||||
return {
|
||||
"prompt": prompt,
|
||||
"height": height,
|
||||
"width": width,
|
||||
# Pass target dimensions to AR processor for proper grid token generation
|
||||
"mm_processor_kwargs": {
|
||||
"target_h": height,
|
||||
"target_w": width,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def build_prompt_for_i2i(
|
||||
prompt: str,
|
||||
image: Image.Image,
|
||||
height: int | None = None,
|
||||
width: int | None = None,
|
||||
) -> dict:
|
||||
"""
|
||||
Build prompt dict for image-to-image generation.
|
||||
|
||||
Args:
|
||||
prompt: Text description for image editing
|
||||
image: Source image for editing
|
||||
height: Target image height (default: use source image size)
|
||||
width: Target image width (default: use source image size)
|
||||
|
||||
Returns:
|
||||
Dict containing prompt, image, and generation parameters
|
||||
"""
|
||||
# Use source image dimensions if not specified
|
||||
if height is None:
|
||||
height = image.height
|
||||
if width is None:
|
||||
width = image.width
|
||||
|
||||
return {
|
||||
"prompt": prompt,
|
||||
"multi_modal_data": {
|
||||
"image": image,
|
||||
},
|
||||
"height": height,
|
||||
"width": width,
|
||||
# Pass target dimensions to AR processor for proper grid token generation
|
||||
"mm_processor_kwargs": {
|
||||
"target_h": height,
|
||||
"target_w": width,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def main(args: argparse.Namespace) -> None:
|
||||
"""Main entry point for GLM-Image end-to-end inference."""
|
||||
print("=" * 60)
|
||||
print("GLM-Image Multistage End-to-End Inference")
|
||||
print("=" * 60)
|
||||
|
||||
# Validate arguments
|
||||
if not args.model_path:
|
||||
raise ValueError("--model-path is required")
|
||||
|
||||
if not args.prompt:
|
||||
raise ValueError("--prompt is required")
|
||||
|
||||
# Determine config path
|
||||
config_path = args.config_path
|
||||
if config_path is None:
|
||||
# Try to find default config
|
||||
if os.path.exists(DEFAULT_CONFIG_PATH):
|
||||
config_path = DEFAULT_CONFIG_PATH
|
||||
else:
|
||||
# Try relative to script location
|
||||
script_dir = Path(__file__).parent.parent.parent.parent
|
||||
config_path = script_dir / "vllm_omni/model_executor/stage_configs/glm_image.yaml"
|
||||
if not config_path.exists():
|
||||
raise FileNotFoundError(
|
||||
f"Stage config not found. Please specify --config-path. Tried: {DEFAULT_CONFIG_PATH}"
|
||||
)
|
||||
config_path = str(config_path)
|
||||
|
||||
print(f"Model path: {args.model_path}")
|
||||
print(f"Config path: {config_path}")
|
||||
print(f"Prompt: {args.prompt}")
|
||||
|
||||
# Load source image for image-to-image mode
|
||||
source_image = None
|
||||
if args.image:
|
||||
print(f"Source image: {args.image}")
|
||||
source_image = load_image(args.image)
|
||||
print(f" Image size: {source_image.width}x{source_image.height}")
|
||||
|
||||
# Build prompt based on mode
|
||||
if source_image is not None:
|
||||
# Image-to-image mode
|
||||
prompt_dict = build_prompt_for_i2i(
|
||||
prompt=args.prompt,
|
||||
image=source_image,
|
||||
height=args.height,
|
||||
width=args.width,
|
||||
)
|
||||
mode = "image-to-image"
|
||||
else:
|
||||
# Text-to-image mode
|
||||
prompt_dict = build_prompt_for_t2i(
|
||||
prompt=args.prompt,
|
||||
height=args.height or 1024,
|
||||
width=args.width or 1024,
|
||||
)
|
||||
mode = "text-to-image"
|
||||
|
||||
print(f"Mode: {mode}")
|
||||
print(f"Target size: {prompt_dict.get('height', 1024)}x{prompt_dict.get('width', 1024)}")
|
||||
|
||||
# Add generation parameters to prompt
|
||||
prompt_dict["seed"] = args.seed
|
||||
prompt_dict["num_inference_steps"] = args.num_inference_steps
|
||||
prompt_dict["guidance_scale"] = args.guidance_scale
|
||||
|
||||
if args.negative_prompt:
|
||||
prompt_dict["negative_prompt"] = args.negative_prompt
|
||||
|
||||
# Initialize Omni with multistage config
|
||||
print("\nInitializing Omni with multistage pipeline...")
|
||||
start_time = time.time()
|
||||
|
||||
omni = Omni(
|
||||
model=args.model_path,
|
||||
stage_configs_path=config_path,
|
||||
log_stats=args.enable_stats,
|
||||
stage_init_timeout=args.stage_init_timeout,
|
||||
)
|
||||
|
||||
init_time = time.time() - start_time
|
||||
print(f"Initialization completed in {init_time:.2f}s")
|
||||
|
||||
# Prepare prompts (support batch generation)
|
||||
prompts = [prompt_dict for _ in range(args.num_prompts)]
|
||||
|
||||
# No explicit sampling_params for diffusion - parameters are in prompt_dict
|
||||
# For multistage, the AR stage may need sampling params
|
||||
from vllm import SamplingParams
|
||||
|
||||
# Compute max_tokens dynamically based on target image size
|
||||
target_height = prompt_dict.get("height", 1024)
|
||||
target_width = prompt_dict.get("width", 1024)
|
||||
calculated_max_tokens = compute_max_tokens(target_height, target_width)
|
||||
|
||||
# Use calculated value unless user explicitly specified a different value
|
||||
# Default args.max_tokens is 16384 (very large), so prefer calculated value
|
||||
effective_max_tokens = calculated_max_tokens if args.max_tokens == 16384 else args.max_tokens
|
||||
|
||||
if args.verbose:
|
||||
print(f"AR max_tokens: {effective_max_tokens} (calculated: {calculated_max_tokens}, arg: {args.max_tokens})")
|
||||
|
||||
# IMPORTANT: GLM-Image AR model requires these exact sampling parameters
|
||||
# from generation_config.json for proper image token generation.
|
||||
# - temperature=0.9, top_p=0.75, top_k=16512 (vision_vocab_size)
|
||||
# - stop_token_ids=[16385] (eos_token_id) is CRITICAL to stop generation
|
||||
ar_sampling_params = SamplingParams(
|
||||
temperature=0.9, # From generation_config.json
|
||||
top_p=0.75, # From generation_config.json
|
||||
top_k=GLM_IMAGE_VISION_VOCAB_SIZE, # 16512, vision vocabulary size
|
||||
max_tokens=effective_max_tokens,
|
||||
stop_token_ids=[GLM_IMAGE_EOS_TOKEN_ID], # 16385, CRITICAL for stopping
|
||||
seed=args.seed,
|
||||
detokenize=False,
|
||||
)
|
||||
|
||||
# For diffusion stage, sampling_params contains diffusion-specific parameters
|
||||
# These are passed as kwargs to the diffusion engine
|
||||
diffusion_sampling_params = OmniDiffusionSamplingParams(
|
||||
num_inference_steps=args.num_inference_steps,
|
||||
guidance_scale=args.guidance_scale,
|
||||
height=prompt_dict.get("height", 1024),
|
||||
width=prompt_dict.get("width", 1024),
|
||||
seed=args.seed,
|
||||
)
|
||||
|
||||
# For multistage, we need sampling_params for each stage
|
||||
# Stage 0 (AR): SamplingParams for vLLM
|
||||
# Stage 1 (Diffusion): dict with diffusion kwargs
|
||||
sampling_params_list = [ar_sampling_params, diffusion_sampling_params]
|
||||
|
||||
# Run generation
|
||||
print(f"\nGenerating {args.num_prompts} image(s)...")
|
||||
gen_start_time = time.time()
|
||||
|
||||
output_dir = os.path.dirname(args.output) if args.output else "outputs"
|
||||
if output_dir:
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
output_count = 0
|
||||
for stage_outputs in omni.generate(prompts, sampling_params_list, py_generator=True):
|
||||
if stage_outputs.final_output_type == "image":
|
||||
for output in stage_outputs.request_output:
|
||||
request_id = output.request_id
|
||||
|
||||
# Get generated images
|
||||
images = output.images if hasattr(output, "images") else []
|
||||
if not images and hasattr(output, "multimodal_output"):
|
||||
images = output.multimodal_output.get("images", [])
|
||||
|
||||
# Save each generated image
|
||||
for idx, img in enumerate(images):
|
||||
if args.num_prompts == 1 and len(images) == 1:
|
||||
output_path = args.output
|
||||
else:
|
||||
base, ext = os.path.splitext(args.output)
|
||||
output_path = f"{base}_{request_id}_{idx}{ext}"
|
||||
|
||||
if isinstance(img, Image.Image):
|
||||
save_image(img, output_path)
|
||||
else:
|
||||
print(f"Warning: Unexpected image type for request {request_id}: {type(img)}")
|
||||
|
||||
output_count += 1
|
||||
|
||||
elif stage_outputs.final_output_type == "text":
|
||||
# AR stage output (intermediate, for debugging)
|
||||
if args.verbose:
|
||||
for output in stage_outputs.request_output:
|
||||
print(f"AR output for request {output.request_id}:")
|
||||
print(f" Token count: {len(output.outputs[0].token_ids)}")
|
||||
|
||||
gen_time = time.time() - gen_start_time
|
||||
print(f"\nGeneration completed in {gen_time:.2f}s")
|
||||
print(f"Generated {output_count} image(s)")
|
||||
|
||||
# Cleanup
|
||||
omni.close()
|
||||
print("\nDone!")
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
"""Parse command line arguments."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="GLM-Image Multistage End-to-End Inference",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
epilog=__doc__,
|
||||
)
|
||||
|
||||
# Required arguments
|
||||
parser.add_argument(
|
||||
"--model-path",
|
||||
type=str,
|
||||
required=True,
|
||||
help="Path to GLM-Image model directory or HuggingFace model ID",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--prompt",
|
||||
type=str,
|
||||
required=True,
|
||||
help="Text prompt for image generation",
|
||||
)
|
||||
|
||||
# Optional arguments
|
||||
parser.add_argument(
|
||||
"--config-path",
|
||||
type=str,
|
||||
default=None,
|
||||
help="Path to stage config YAML file (default: auto-detect)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--image",
|
||||
type=str,
|
||||
default=None,
|
||||
help="Path to source image for image-to-image mode",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output",
|
||||
type=str,
|
||||
default="output_glm_image.png",
|
||||
help="Output image path (default: output_glm_image.png)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--negative-prompt",
|
||||
type=str,
|
||||
default=None,
|
||||
help="Negative prompt for classifier-free guidance",
|
||||
)
|
||||
|
||||
# Generation parameters
|
||||
parser.add_argument(
|
||||
"--height",
|
||||
type=int,
|
||||
default=None,
|
||||
help="Output image height (default: 1024 for t2i, source size for i2i)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--width",
|
||||
type=int,
|
||||
default=None,
|
||||
help="Output image width (default: 1024 for t2i, source size for i2i)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--num-inference-steps",
|
||||
type=int,
|
||||
default=50,
|
||||
help="Number of diffusion denoising steps (default: 50)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--guidance-scale",
|
||||
type=float,
|
||||
default=1.5,
|
||||
help="Classifier-free guidance scale (default: 1.5)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--seed",
|
||||
type=int,
|
||||
default=SEED,
|
||||
help=f"Random seed for reproducibility (default: {SEED})",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-tokens",
|
||||
type=int,
|
||||
default=16384,
|
||||
help="Maximum tokens for AR generation (default: 16384)",
|
||||
)
|
||||
|
||||
# Batch processing
|
||||
parser.add_argument(
|
||||
"--num-prompts",
|
||||
type=int,
|
||||
default=1,
|
||||
help="Number of images to generate (default: 1)",
|
||||
)
|
||||
|
||||
# Runtime options
|
||||
parser.add_argument(
|
||||
"--enable-stats",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Enable statistics logging",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--stage-init-timeout",
|
||||
type=int,
|
||||
default=600,
|
||||
help="Timeout for stage initialization in seconds (default: 600)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose",
|
||||
"-v",
|
||||
action="store_true",
|
||||
default=False,
|
||||
help="Enable verbose output",
|
||||
)
|
||||
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
args = parse_args()
|
||||
main(args)
|
||||
+93
@@ -0,0 +1,93 @@
|
||||
#!/bin/bash
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# Run GLM-Image image-to-image (editing) with multistage pipeline
|
||||
|
||||
set -e
|
||||
|
||||
# Default values
|
||||
MODEL_PATH="${MODEL_PATH:-/path/to/glm-image}"
|
||||
CONFIG_PATH="${CONFIG_PATH:-vllm_omni/model_executor/stage_configs/glm_image.yaml}"
|
||||
PROMPT="${PROMPT:-Transform this image into an oil painting style}"
|
||||
INPUT_IMAGE=""
|
||||
OUTPUT="${OUTPUT:-output_i2i.png}"
|
||||
NUM_STEPS="${NUM_STEPS:-50}"
|
||||
GUIDANCE="${GUIDANCE:-1.5}"
|
||||
SEED="${SEED:-42}"
|
||||
|
||||
# Parse command line arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
--model-path)
|
||||
MODEL_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--config-path)
|
||||
CONFIG_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--prompt)
|
||||
PROMPT="$2"
|
||||
shift 2
|
||||
;;
|
||||
--image)
|
||||
INPUT_IMAGE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--output)
|
||||
OUTPUT="$2"
|
||||
shift 2
|
||||
;;
|
||||
--num-steps)
|
||||
NUM_STEPS="$2"
|
||||
shift 2
|
||||
;;
|
||||
--guidance)
|
||||
GUIDANCE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--seed)
|
||||
SEED="$2"
|
||||
shift 2
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $1"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Check if input image is provided
|
||||
if [ -z "${INPUT_IMAGE}" ]; then
|
||||
echo "Error: --image is required for image-to-image mode"
|
||||
echo "Usage: ./run_i2i.sh --image /path/to/input.png [--prompt \"edit instruction\"]"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ ! -f "${INPUT_IMAGE}" ]; then
|
||||
echo "Error: Input image not found: ${INPUT_IMAGE}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "=============================================="
|
||||
echo "GLM-Image Image-to-Image Generation"
|
||||
echo "=============================================="
|
||||
echo "Model: ${MODEL_PATH}"
|
||||
echo "Config: ${CONFIG_PATH}"
|
||||
echo "Input: ${INPUT_IMAGE}"
|
||||
echo "Prompt: ${PROMPT}"
|
||||
echo "Output: ${OUTPUT}"
|
||||
echo "Steps: ${NUM_STEPS}"
|
||||
echo "Guidance: ${GUIDANCE}"
|
||||
echo "Seed: ${SEED}"
|
||||
echo "=============================================="
|
||||
|
||||
python end2end.py \
|
||||
--model-path "${MODEL_PATH}" \
|
||||
--config-path "${CONFIG_PATH}" \
|
||||
--prompt "${PROMPT}" \
|
||||
--image "${INPUT_IMAGE}" \
|
||||
--output "${OUTPUT}" \
|
||||
--num-inference-steps "${NUM_STEPS}" \
|
||||
--guidance-scale "${GUIDANCE}" \
|
||||
--seed "${SEED}" \
|
||||
--verbose
|
||||
+87
@@ -0,0 +1,87 @@
|
||||
#!/bin/bash
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
# Run GLM-Image text-to-image generation with multistage pipeline
|
||||
|
||||
set -e
|
||||
|
||||
# Default values
|
||||
MODEL_PATH="${MODEL_PATH:-/path/to/glm-image}"
|
||||
CONFIG_PATH="${CONFIG_PATH:-vllm_omni/model_executor/stage_configs/glm_image.yaml}"
|
||||
PROMPT="${PROMPT:-A beautiful mountain landscape with snow-capped peaks and a clear blue lake}"
|
||||
OUTPUT="${OUTPUT:-output_t2i.png}"
|
||||
HEIGHT="${HEIGHT:-1024}"
|
||||
WIDTH="${WIDTH:-1024}"
|
||||
NUM_STEPS="${NUM_STEPS:-50}"
|
||||
GUIDANCE="${GUIDANCE:-1.5}"
|
||||
SEED="${SEED:-42}"
|
||||
|
||||
# Parse command line arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
--model-path)
|
||||
MODEL_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--config-path)
|
||||
CONFIG_PATH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--prompt)
|
||||
PROMPT="$2"
|
||||
shift 2
|
||||
;;
|
||||
--output)
|
||||
OUTPUT="$2"
|
||||
shift 2
|
||||
;;
|
||||
--height)
|
||||
HEIGHT="$2"
|
||||
shift 2
|
||||
;;
|
||||
--width)
|
||||
WIDTH="$2"
|
||||
shift 2
|
||||
;;
|
||||
--num-steps)
|
||||
NUM_STEPS="$2"
|
||||
shift 2
|
||||
;;
|
||||
--guidance)
|
||||
GUIDANCE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--seed)
|
||||
SEED="$2"
|
||||
shift 2
|
||||
;;
|
||||
*)
|
||||
echo "Unknown option: $1"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo "=============================================="
|
||||
echo "GLM-Image Text-to-Image Generation"
|
||||
echo "=============================================="
|
||||
echo "Model: ${MODEL_PATH}"
|
||||
echo "Config: ${CONFIG_PATH}"
|
||||
echo "Prompt: ${PROMPT}"
|
||||
echo "Output: ${OUTPUT}"
|
||||
echo "Size: ${WIDTH}x${HEIGHT}"
|
||||
echo "Steps: ${NUM_STEPS}"
|
||||
echo "Guidance: ${GUIDANCE}"
|
||||
echo "Seed: ${SEED}"
|
||||
echo "=============================================="
|
||||
|
||||
python end2end.py \
|
||||
--model-path "${MODEL_PATH}" \
|
||||
--config-path "${CONFIG_PATH}" \
|
||||
--prompt "${PROMPT}" \
|
||||
--output "${OUTPUT}" \
|
||||
--height "${HEIGHT}" \
|
||||
--width "${WIDTH}" \
|
||||
--num-inference-steps "${NUM_STEPS}" \
|
||||
--guidance-scale "${GUIDANCE}" \
|
||||
--seed "${SEED}" \
|
||||
--verbose
|
||||
Reference in new issue
Block a user