[Model] Add JoyAI-Image-Edit pipeline support (#4112)
Signed-off-by: nagisa-kun <1434936049@qq.com> Signed-off-by: nagisa <1434936049@qq.com> Signed-off-by: hurukawa <61525444+nagisa-kunhah@users.noreply.github.com>
This commit is contained in:
1 parent
6a70bda0f6
commit
3e705fcdb1
21 files changed
+4141
-10
No files matched your search
@@ -39,6 +39,20 @@ Usage (multiple images):
|
||||
--cfg-scale 4.0 \
|
||||
--guidance-scale 1.0
|
||||
|
||||
Usage (JoyAI-Image-Edit, single image):
|
||||
python image_edit.py \
|
||||
--model jdopensource/JoyAI-Image-Edit-Diffusers \
|
||||
--image input.png \
|
||||
--prompt "Change the background to a clean studio while preserving the subject." \
|
||||
--height 1024 \
|
||||
--width 1024 \
|
||||
--num-inference-steps 50 \
|
||||
--guidance-scale 4.0 \
|
||||
--output output_joyai_edit.png
|
||||
|
||||
Note: JoyAI-Image-Edit snaps requested dimensions to the nearest supported
|
||||
Joy bucket; for square outputs, use 1024x1024.
|
||||
|
||||
Usage (with cache-dit acceleration):
|
||||
python image_edit.py \
|
||||
--image input.png \
|
||||
@@ -270,9 +284,10 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument(
|
||||
"--cfg-scale",
|
||||
type=float,
|
||||
default=4.0,
|
||||
default=None,
|
||||
help=(
|
||||
"True classifier-free guidance scale (default: 4.0). Guidance scale as defined in Classifier-Free "
|
||||
"True classifier-free guidance scale. When omitted, the pipeline uses its model-specific default. "
|
||||
"Guidance scale as defined in Classifier-Free "
|
||||
"Diffusion Guidance. Classifier-free guidance is enabled by setting cfg_scale > 1 and providing "
|
||||
"a negative_prompt. Higher guidance scale encourages images closely linked to the text prompt, "
|
||||
"usually at the expense of lower image quality."
|
||||
@@ -281,9 +296,9 @@ def parse_args() -> argparse.Namespace:
|
||||
parser.add_argument(
|
||||
"--guidance-scale",
|
||||
type=float,
|
||||
default=1.0,
|
||||
default=None,
|
||||
help=(
|
||||
"Guidance scale for guidance-distilled models (default: 1.0, disabled). "
|
||||
"Guidance scale for guidance-distilled models. When omitted, the pipeline uses its model-specific default. "
|
||||
"Unlike classifier-free guidance (--cfg-scale), guidance-distilled models take the guidance scale "
|
||||
"directly as an input parameter. Enabled when guidance_scale > 1. Ignored when not using guidance-distilled models."
|
||||
),
|
||||
@@ -523,6 +538,12 @@ def parse_args() -> argparse.Namespace:
|
||||
"silently masking with a possibly-wrong prompt."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--init-timeout",
|
||||
type=int,
|
||||
default=600,
|
||||
help="Overall pipeline initialization timeout in seconds.",
|
||||
)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
@@ -593,6 +614,7 @@ def main():
|
||||
enable_cpu_offload=args.enable_cpu_offload,
|
||||
enable_diffusion_pipeline_profiler=args.enable_diffusion_pipeline_profiler,
|
||||
profiler_config=args.profiler_config,
|
||||
init_timeout=args.init_timeout,
|
||||
)
|
||||
if args.enforce_eager is not None:
|
||||
omni_kwargs["enforce_eager"] = args.enforce_eager
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Image-To-Image
|
||||
|
||||
This example edits an input image with `Qwen/Qwen-Image-Edit` using the `image_edit.py` CLI.
|
||||
This example edits an input image with `Qwen/Qwen-Image-Edit` and `JoyAI-Image-Edit` using the `image_edit.py` CLI.
|
||||
|
||||
## Local CLI Usage
|
||||
|
||||
@@ -23,6 +23,27 @@ python image_edit.py \
|
||||
--cfg-scale 4.0
|
||||
```
|
||||
|
||||
### JoyAI-Image-Edit
|
||||
|
||||
JoyAI-Image-Edit uses the same `image_edit.py` CLI and supports one input
|
||||
image per request. Requested dimensions are snapped to the nearest supported Joy
|
||||
bucket; for square outputs, use 1024x1024.
|
||||
|
||||
```bash
|
||||
python image_edit.py \
|
||||
--model jdopensource/JoyAI-Image-Edit-Diffusers \
|
||||
--image input.png \
|
||||
--prompt "Change the background to a clean studio while preserving the subject." \
|
||||
--height 1024 \
|
||||
--width 1024 \
|
||||
--num-inference-steps 50 \
|
||||
--guidance-scale 4.0 \
|
||||
--output output_joyai_edit.png
|
||||
```
|
||||
|
||||
JoyAI defaults to guidance scale `4.0`; pass `--guidance-scale` explicitly when
|
||||
you need a different value.
|
||||
|
||||
### Multiple Image Editing (Qwen-Image-Edit-2509)
|
||||
|
||||
For multiple image inputs, use `Qwen/Qwen-Image-Edit-2509` or `Qwen/Qwen-Image-Edit-2511`:
|
||||
@@ -40,7 +61,7 @@ python image_edit.py \
|
||||
|
||||
Key arguments:
|
||||
|
||||
- `--model`: model name or path. Use `Qwen/Qwen-Image-Edit-2509` or later for multiple image support.
|
||||
- `--model`: model name or path. Use `Qwen/Qwen-Image-Edit-2509` or later for multiple image support. `jdopensource/JoyAI-Image-Edit-Diffusers` supports one input image.
|
||||
- `--image`: path(s) to the source image(s) (PNG/JPG, converted to RGB). Can specify multiple images.
|
||||
- `--prompt` / `--negative-prompt`: text description (string).
|
||||
- `--cfg-scale`: true classifier-free guidance scale (default: 4.0). Classifier-free guidance is enabled by setting cfg_scale > 1 and providing a negative_prompt. Higher guidance scale encourages images closely linked to the text prompt, usually at the expense of lower image quality.
|
||||
|
||||
@@ -2,10 +2,12 @@
|
||||
|
||||
This example demonstrates how to deploy image-to-image models for online image editing service using vLLM-Omni.
|
||||
|
||||
Supported models include Qwen-Image-Edit, BAGEL, and other image-to-image pipelines.
|
||||
Supported models include Qwen-Image-Edit, BAGEL, JoyAI-Image-Edit, and other image-to-image pipelines.
|
||||
|
||||
For **multi-image** input editing, use **Qwen-Image-Edit-2509** (QwenImageEditPlusPipeline) and send multiple images in the user message content.
|
||||
|
||||
For **JoyAI-Image-Edit**, use `jdopensource/JoyAI-Image-Edit-Diffusers` and send exactly one input image.
|
||||
|
||||
## Start Server
|
||||
|
||||
### Basic Start
|
||||
@@ -29,6 +31,12 @@ vllm serve Qwen/Qwen-Image-Edit-2509 --omni --port 8092
|
||||
vllm serve ByteDance-Seed/BAGEL-7B-MoT --omni --port 8091
|
||||
```
|
||||
|
||||
### JoyAI-Image-Edit
|
||||
|
||||
```bash
|
||||
vllm serve jdopensource/JoyAI-Image-Edit-Diffusers --omni --port 8092
|
||||
```
|
||||
|
||||
### Start with Parameters
|
||||
|
||||
|
||||
@@ -77,6 +85,11 @@ EOF
|
||||
curl -s http://localhost:8092/v1/chat/completions -H "Content-Type: application/json" -d @request.json | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2 | base64 -d > output.png
|
||||
```
|
||||
|
||||
For JoyAI-Image-Edit, set classifier-free guidance with `guidance_scale`
|
||||
in `extra_body` (default: `4.0`). `true_cfg_scale` remains supported as an
|
||||
alias; if both are provided, their values must match. Values at or below `1.0`
|
||||
disable CFG.
|
||||
|
||||
### Method 2: Using OpenAI Python SDK
|
||||
|
||||
```python
|
||||
|
||||
Reference in new issue
Block a user