anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity

[Entrypoints] online serving support usp args (#366)

Browse Source 
Signed-off-by: David Chen <530634352@qq.com>
This commit is contained in:
WeiQing Chen authored and GitHub committed 2025-12-19 11:59:07 +08:00
1 parent 5216cf4ae6
commit 9464e1479d
3 files changed
+32 -2
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/user_guide/acceleration/parallelism_acceleration.md
+9
View File
@@ -40,6 +40,15 @@ outputs = omni.generate(prompt="A cat sitting on a windowsill", num_inference_st
See `examples/offline_inference/text_to_image/text_to_image.py` for a complete working example.
##### Online Serving
You can enable Ulysses-SP in online serving for diffusion models via `--usp`:
```bash
# Text-to-image (requires >= 2 GPUs)
vllm serve Qwen/Qwen-Image --omni --port 8091 --usp 2
```
##### Benchmarks
!!! note "Benchmark Disclaimer"
These benchmarks are provided for **general reference only**. The configurations shown use default or common parameter settings and have not been exhaustively optimized for maximum performance. Actual performance may vary based on: