[Entrypoints] Support Online Serving for Diffusion-only Models (#259)
Signed-off-by: Chenguang ZHENG <645327136@qq.com> Signed-off-by: yinpe <11810305@mail.sustech.edu.cn> Co-authored-by: yinpe <11810305@mail.sustech.edu.cn>
This commit is contained in:
Chenguang Zhengandyinpe authored and GitHub committed
1 parent
8722c6374c
commit
acf66ee2cc
19 files changed
+2506
-20
No files matched your search
@@ -0,0 +1,197 @@
|
||||
# Qwen-Image-Edit Online Serving
|
||||
|
||||
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/qwen_image_edit>.
|
||||
|
||||
|
||||
This example demonstrates how to deploy Qwen-Image-Edit model for online image editing service using vLLM-Omni.
|
||||
|
||||
## Start Server
|
||||
|
||||
### Basic Start
|
||||
|
||||
```bash
|
||||
vllm serve Qwen/Qwen-Image-Edit --omni --port 8092
|
||||
```
|
||||
|
||||
### Start with Parameters
|
||||
|
||||
Or use the startup script:
|
||||
|
||||
```bash
|
||||
bash run_server.sh
|
||||
```
|
||||
|
||||
## API Calls
|
||||
|
||||
### Method 1: Using curl (Image Editing)
|
||||
|
||||
```bash
|
||||
# Image editing
|
||||
bash run_curl_image_edit.sh input.png "Convert this image to watercolor style"
|
||||
|
||||
# Or execute directly
|
||||
IMG_B64=$(base64 -w0 input.png)
|
||||
curl -s http://localhost:8092/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "{
|
||||
\"messages\": [{
|
||||
\"role\": \"user\",
|
||||
\"content\": [
|
||||
{\"type\": \"text\", \"text\": \"Convert this image to watercolor style\"},
|
||||
{\"type\": \"image_url\", \"image_url\": {\"url\": \"data:image/png;base64,\$IMG_B64\"}}
|
||||
]
|
||||
}],
|
||||
\"extra_body\": {
|
||||
\"height\": 1024,
|
||||
\"width\": 1024,
|
||||
\"num_inference_steps\": 50,
|
||||
\"guidance_scale\": 7.5,
|
||||
\"seed\": 42
|
||||
}
|
||||
}" | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2 | base64 -d > output.png
|
||||
```
|
||||
|
||||
### Method 2: Using Python Client
|
||||
|
||||
```bash
|
||||
python openai_chat_client.py --input input.png --prompt "Convert to oil painting style" --output output.png
|
||||
```
|
||||
|
||||
### Method 3: Using Gradio Demo
|
||||
|
||||
```bash
|
||||
python gradio_demo.py
|
||||
# Visit http://localhost:7861
|
||||
```
|
||||
|
||||
## Request Format
|
||||
|
||||
### Image Editing (Using image_url Format)
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "Convert this image to watercolor style"},
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Image Editing (Using Simplified image Format)
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"text": "Convert this image to watercolor style"},
|
||||
{"image": "BASE64_IMAGE_DATA"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Image Editing with Parameters
|
||||
|
||||
Use `extra_body` to pass generation parameters:
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "Convert to ink wash painting style"},
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
|
||||
]
|
||||
}
|
||||
],
|
||||
"extra_body": {
|
||||
"height": 1024,
|
||||
"width": 1024,
|
||||
"num_inference_steps": 50,
|
||||
"guidance_scale": 7.5,
|
||||
"seed": 42
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Generation Parameters (extra_body)
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `num_inference_steps` | int | 50 | Number of denoising steps |
|
||||
| `guidance_scale` | float | 7.5 | CFG guidance scale |
|
||||
| `seed` | int | None | Random seed (reproducible) |
|
||||
| `negative_prompt` | str | None | Negative prompt |
|
||||
| `num_outputs_per_prompt` | int | 1 | Number of images to generate |
|
||||
|
||||
## Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "chatcmpl-xxx",
|
||||
"created": 1234567890,
|
||||
"model": "Qwen/Qwen-Image-Edit",
|
||||
"choices": [{
|
||||
"index": 0,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": [{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": "data:image/png;base64,..."
|
||||
}
|
||||
}]
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
}],
|
||||
"usage": {...}
|
||||
}
|
||||
```
|
||||
|
||||
## Common Editing Instructions Examples
|
||||
|
||||
| Instruction | Description |
|
||||
|-------------|-------------|
|
||||
| `Convert this image to watercolor style` | Style transfer |
|
||||
| `Convert the image to black and white` | Desaturation |
|
||||
| `Enhance the color saturation` | Color adjustment |
|
||||
| `Convert to cartoon style` | Cartoonization |
|
||||
| `Add vintage filter effect` | Filter effect |
|
||||
| `Convert daytime scene to nighttime` | Scene conversion |
|
||||
|
||||
## File Description
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `run_server.sh` | Server startup script |
|
||||
| `run_curl_image_edit.sh` | curl image editing example |
|
||||
| `openai_chat_client.py` | Python client |
|
||||
| `gradio_demo.py` | Gradio interactive interface |
|
||||
|
||||
## Example materials
|
||||
|
||||
??? abstract "gradio_demo.py"
|
||||
``````py
|
||||
--8<-- "examples/online_serving/qwen_image_edit/gradio_demo.py"
|
||||
``````
|
||||
??? abstract "openai_chat_client.py"
|
||||
``````py
|
||||
--8<-- "examples/online_serving/qwen_image_edit/openai_chat_client.py"
|
||||
``````
|
||||
??? abstract "run_curl_image_edit.sh"
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/qwen_image_edit/run_curl_image_edit.sh"
|
||||
``````
|
||||
??? abstract "run_server.sh"
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/qwen_image_edit/run_server.sh"
|
||||
``````
|
||||
@@ -0,0 +1,178 @@
|
||||
# Qwen-Image/Z-Image Online Serving
|
||||
|
||||
Source <https://github.com/vllm-project/vllm-omni/tree/main/examples/online_serving/qwen_image>.
|
||||
|
||||
|
||||
This example demonstrates how to deploy Qwen-Image model for online image generation service using vLLM-Omni.
|
||||
|
||||
## Start Server
|
||||
|
||||
### Basic Start
|
||||
|
||||
```bash
|
||||
vllm serve Qwen/Qwen-Image --omni --port 8091
|
||||
```
|
||||
|
||||
### Start with Parameters
|
||||
|
||||
|
||||
Or use the startup script:
|
||||
|
||||
```bash
|
||||
bash run_server.sh
|
||||
```
|
||||
|
||||
## API Calls
|
||||
|
||||
### Method 1: Using curl
|
||||
|
||||
```bash
|
||||
# Basic text-to-image generation
|
||||
bash run_curl_text_to_image.sh
|
||||
|
||||
# Or execute directly
|
||||
curl -s http://localhost:8091/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"messages": [
|
||||
{"role": "user", "content": "A beautiful landscape painting"}
|
||||
],
|
||||
"extra_body": {
|
||||
"height": 1024,
|
||||
"width": 1024,
|
||||
"num_inference_steps": 50,
|
||||
"true_cfg_scale": 4.0,
|
||||
"seed": 42
|
||||
}
|
||||
}' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2 | base64 -d > output.png
|
||||
```
|
||||
|
||||
### Method 2: Using Python Client
|
||||
|
||||
```bash
|
||||
python openai_chat_client.py --prompt "A beautiful landscape painting" --output output.png
|
||||
```
|
||||
|
||||
### Method 3: Using Gradio Demo
|
||||
|
||||
```bash
|
||||
python gradio_demo.py
|
||||
# Visit http://localhost:7860
|
||||
```
|
||||
|
||||
## Request Format
|
||||
|
||||
### Simple Text-to-Image Generation
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{"role": "user", "content": "A beautiful landscape painting"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
### Generation with Parameters
|
||||
|
||||
Use `extra_body` to pass generation parameters:
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{"role": "user", "content": "A beautiful landscape painting"}
|
||||
],
|
||||
"extra_body": {
|
||||
"height": 1024,
|
||||
"width": 1024,
|
||||
"num_inference_steps": 50,
|
||||
"true_cfg_scale": 4.0,
|
||||
"seed": 42
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Multimodal Input (Text + Structured Content)
|
||||
|
||||
```json
|
||||
{
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": "A beautiful landscape painting"}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Generation Parameters (extra_body)
|
||||
|
||||
| Parameter | Type | Default | Description |
|
||||
|-----------|------|---------|-------------|
|
||||
| `height` | int | None | Image height in pixels |
|
||||
| `width` | int | None | Image width in pixels |
|
||||
| `num_inference_steps` | int | 50 | Number of denoising steps |
|
||||
| `true_cfg_scale` | float | 4.0 | Qwen-Image CFG scale |
|
||||
| `seed` | int | None | Random seed (reproducible) |
|
||||
| `negative_prompt` | str | None | Negative prompt |
|
||||
| `num_outputs_per_prompt` | int | 1 | Number of images to generate |
|
||||
|
||||
## Response Format
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "chatcmpl-xxx",
|
||||
"created": 1234567890,
|
||||
"model": "Qwen/Qwen-Image",
|
||||
"choices": [{
|
||||
"index": 0,
|
||||
"message": {
|
||||
"role": "assistant",
|
||||
"content": [{
|
||||
"type": "image_url",
|
||||
"image_url": {
|
||||
"url": "data:image/png;base64,..."
|
||||
}
|
||||
}]
|
||||
},
|
||||
"finish_reason": "stop"
|
||||
}],
|
||||
"usage": {...}
|
||||
}
|
||||
```
|
||||
|
||||
## Extract Image
|
||||
|
||||
```bash
|
||||
# Extract base64 from response and decode to image
|
||||
cat response.json | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2 | base64 -d > output.png
|
||||
```
|
||||
|
||||
## File Description
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `run_server.sh` | Server startup script |
|
||||
| `run_curl_text_to_image.sh` | curl example |
|
||||
| `openai_chat_client.py` | Python client |
|
||||
| `gradio_demo.py` | Gradio interactive interface |
|
||||
|
||||
## Example materials
|
||||
|
||||
??? abstract "gradio_demo.py"
|
||||
``````py
|
||||
--8<-- "examples/online_serving/qwen_image/gradio_demo.py"
|
||||
``````
|
||||
??? abstract "openai_chat_client.py"
|
||||
``````py
|
||||
--8<-- "examples/online_serving/qwen_image/openai_chat_client.py"
|
||||
``````
|
||||
??? abstract "run_curl_text_to_image.sh"
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/qwen_image/run_curl_text_to_image.sh"
|
||||
``````
|
||||
??? abstract "run_server.sh"
|
||||
``````sh
|
||||
--8<-- "examples/online_serving/qwen_image/run_server.sh"
|
||||
``````
|
||||
Reference in new issue
Block a user