[Doc] update diffusion_acceleration.md & FAQ memory troubleshooting (#406)
Signed-off-by: David Chen <530634352@qq.com>
This commit is contained in:
1 parent
74c142460d
commit
d9faeb29c0
6 files changed
+67
-11
No files matched your search
@@ -40,4 +40,27 @@ For more usages, please refer to [offline inference](../user_guide/examples/offl
|
||||
|
||||
## Online Serving with OpenAI-Completions API
|
||||
|
||||
Please refer to [online serving](../user_guide/examples/online_serving/qwen2_5_omni.md)
|
||||
Text-to-image generation quickstart with vLLM-Omni:
|
||||
|
||||
```bash
|
||||
vllm serve Tongyi-MAI/Z-Image-Turbo --omni --port 8091
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8091/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"messages": [
|
||||
{"role": "user", "content": "a cup of coffee on the table"}
|
||||
],
|
||||
"extra_body": {
|
||||
"height": 1024,
|
||||
"width": 1024,
|
||||
"num_inference_steps": 50,
|
||||
"guidance_scale": 4.0,
|
||||
"seed": 42
|
||||
}
|
||||
}' | jq -r '.choices[0].message.content[0].image_url.url' | cut -d',' -f2 | base64 -d > coffee.png
|
||||
```
|
||||
|
||||
For more details, please refer to [online serving](../user_guide/examples/online_serving/text_to_image.md).
|
||||
@@ -27,7 +27,7 @@ th {
|
||||
| `WanPipeline` | Wan2.2 | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` |
|
||||
| `OvisImagePipeline` | Ovis-Image | `OvisAI/Ovis-Image` |
|
||||
|`LongcatImagePipeline` | LongCat-Image | `meituan-longcat/LongCat-Image` |
|
||||
|`LongCatImageEditPipeline` | LongCat-Image | `meituan-longcat/LongCat-Image-Edit` |
|
||||
|`LongCatImageEditPipeline` | LongCat-Image-Edit | `meituan-longcat/LongCat-Image-Edit` |
|
||||
|
||||
|
||||
## List of Supported Models for NPU
|
||||
|
||||
@@ -12,6 +12,10 @@ sudo apt update
|
||||
sudo apt install ffmpeg
|
||||
```
|
||||
|
||||
> Q: I see GPU OOM or "free memory is less than desired GPU memory utilization" errors. How can I fix it?
|
||||
|
||||
A: Refer to [GPU memory calculation and configuration](../configuration/gpu_memory_utilization.md) for guidance on tuning `gpu_memory_utilization` and related settings.
|
||||
|
||||
> Q: I encounter some bugs or CI problems, which is urgent. How can I solve it?
|
||||
|
||||
A: At first, you can check current [issues](https://github.com/vllm-project/vllm-omni/issues) to find possible solutions. If none of these satisfy your demand and it is urgent, please find these [volunteers](https://docs.vllm.ai/projects/vllm-omni/en/latest/community/volunteers/) for help.
|
||||
|
||||
@@ -11,12 +11,24 @@ The following parallelism methods are currently supported in vLLM-Omni:
|
||||
|
||||
The following table shows which models are currently supported by parallelism method:
|
||||
|
||||
### ImageGen
|
||||
|
||||
| Model | Model Identifier | Ulysses-SP |
|
||||
|-------|-----------------|-----------|
|
||||
| **Qwen-Image** | `Qwen/Qwen-Image` | ✅ |
|
||||
| **Z-Image** | `Tongyi-MAI/Z-Image-Turbo` | ❌ |
|
||||
| Model | Model Identifier | Ulysses-SP |
|
||||
|-------|------------------|-----------|
|
||||
| **LongCat-Image** | `meituan-longcat/LongCat-Image` | ❌ |
|
||||
| **LongCat-Image-Edit** | `meituan-longcat/LongCat-Image-Edit` | ❌ |
|
||||
| **Ovis-Image** | `OvisAI/Ovis-Image` | ❌ |
|
||||
| **Qwen-Image** | `Qwen/Qwen-Image` | ✅ |
|
||||
| **Qwen-Image-Edit** | `Qwen/Qwen-Image-Edit` | ✅ |
|
||||
| **Qwen-Image-Edit-2509** | `Qwen/Qwen-Image-Edit-2509` | ✅ |
|
||||
| **Qwen-Image-Layered** | `Qwen/Qwen-Image-Layered` | ✅ |
|
||||
| **Z-Image** | `Tongyi-MAI/Z-Image-Turbo` | ❌ |
|
||||
|
||||
### VideoGen
|
||||
|
||||
| Model | Model Identifier | Ulysses-SP |
|
||||
|-------|------------------|-----------|
|
||||
| **Wan2.2** | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | ❌ |
|
||||
|
||||
### Sequence Parallelism
|
||||
|
||||
|
||||
@@ -97,6 +97,8 @@ cache_config={"rel_l1_thresh": 0.1} # More conservative caching
|
||||
|
||||
## Supported Models
|
||||
|
||||
### ImageGen
|
||||
|
||||
<style>
|
||||
th {
|
||||
white-space: nowrap;
|
||||
@@ -108,7 +110,11 @@ th {
|
||||
|--------------|--------|-------------------|
|
||||
| `QwenImagePipeline` | Qwen-Image | `Qwen/Qwen-Image` |
|
||||
| `QwenImageEditPipeline` | Qwen-Image-Edit | `Qwen/Qwen-Image-Edit` |
|
||||
| `QwenImageEditPlusPipeline` | Qwen-Image-Edit | `Qwen/Qwen-Image-Edit-2509` |
|
||||
| `QwenImageEditPlusPipeline` | Qwen-Image-Edit-2509 | `Qwen/Qwen-Image-Edit-2509` |
|
||||
|
||||
### VideoGen
|
||||
|
||||
No VideoGen models are supported by TeaCache yet.
|
||||
|
||||
### Coming Soon
|
||||
|
||||
|
||||
@@ -31,13 +31,24 @@ vLLM-Omni also supports the sequence parallelism (SP) for the diffusion model, t
|
||||
|
||||
The following table shows which models are currently supported by each acceleration method:
|
||||
|
||||
### ImageGen
|
||||
|
||||
| Model | Model Identifier | TeaCache | Cache-DiT | Ulysses-SP |
|
||||
|-------|-----------------|----------|-----------|-----------|
|
||||
|-------|------------------|----------|-----------|-----------|
|
||||
| **LongCat-Image** | `meituan-longcat/LongCat-Image` | ❌ | ✅ | ❌ |
|
||||
| **LongCat-Image-Edit** | `meituan-longcat/LongCat-Image-Edit` | ❌ | ✅ | ❌ |
|
||||
| **Ovis-Image** | `OvisAI/Ovis-Image` | ❌ | ✅ | ❌ |
|
||||
| **Qwen-Image** | `Qwen/Qwen-Image` | ✅ | ✅ | ✅ |
|
||||
| **Z-Image** | `Tongyi-MAI/Z-Image-Turbo` | ❌ | ✅ |❌ |
|
||||
| **Qwen-Image-Edit** | `Qwen/Qwen-Image-Edit` | ✅ | ✅ |✅ |
|
||||
| **Qwen-Image-Edit-2509** | `Qwen/Qwen-Image-Edit-2509` | ❌ | ✅ |✅ |
|
||||
| **Qwen-Image-Edit** | `Qwen/Qwen-Image-Edit` | ✅ | ✅ | ✅ |
|
||||
| **Qwen-Image-Edit-2509** | `Qwen/Qwen-Image-Edit-2509` | ✅ | ✅ | ✅ |
|
||||
| **Qwen-Image-Layered** | `Qwen/Qwen-Image-Layered` | ❌ | ✅ | ✅ |
|
||||
| **Z-Image** | `Tongyi-MAI/Z-Image-Turbo` | ❌ | ✅ | ❌ |
|
||||
|
||||
### VideoGen
|
||||
|
||||
| Model | Model Identifier | TeaCache | Cache-DiT | Ulysses-SP |
|
||||
|-------|------------------|----------|-----------|-----------|
|
||||
| **Wan2.2** | `Wan-AI/Wan2.2-T2V-A14B-Diffusers` | ❌ | ✅ | ❌ |
|
||||
|
||||
|
||||
## Performance Benchmarks
|
||||
|
||||
Reference in new issue
Block a user