anodyine/vllm-omni

0 Fork 0

Files

T

Gao Han db81ee9f69 [Docs] optimize and supplement docs system (#86 )

Signed-off-by: Gaohan123 <gaohan19@huawei.com>

2025-11-28 11:03:52 +08:00

529 B

Raw Blame History

Quickstart

This guide will help you quickly get started with vLLM-Omni to perform:

Offline batched inference
Online serving using OpenAI-compatible server

Prerequisites

OS: Linux
Python: 3.12

Installation

Please refer to installation

Offline Inference

Please refer to offline inference

Online Serving with OpenAI-Completions API

Please refer to online serving