vllm-omni/docs/getting_started/quickstart.md

Quickstart

This guide will help you quickly get started with vLLM-Omni to perform:

• Offline batched inference
• Online serving using OpenAI-compatible server

Prerequisites

• OS: Linux
• Python: 3.12

Installation

For installation on GPU using pre-built-wheel:

uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm==0.11.0 --torch-backend=auto
uv pip install vllm-omni

For additional details—including alternative installation methods, installation on NPU and other platforms — please see the installation guide in installation

Offline Inference

Text-to-image generation quickstart with vLLM-Omni:

from vllm_omni.entrypoints.omni import Omni

if __name__ == "__main__":
    omni = Omni(model="Tongyi-MAI/Z-Image-Turbo")
    prompt = "a cup of coffee on the table"
    images = omni.generate(prompt)
    images[0].save("coffee.png")

For more usages, please refer to offline inference

Online Serving with OpenAI-Completions API

Please refer to online serving