[Docs] optimize and supplement docs system (#86)

Signed-off-by: Gaohan123 <gaohan19@huawei.com>
This commit is contained in:
Gao Han authored and GitHub committed 2025-11-28 11:03:52 +08:00
1 parent 38f58064eb
commit db81ee9f69
75 files changed
+2452 -512

No files matched your search

@@ -0,0 +1,4 @@
nav:
- README.md
- gpu.md
- npu.md
@@ -0,0 +1,7 @@
# Installation
vLLM supports the following hardware platforms:
- [GPU](gpu.md)
- [NVIDIA CUDA](gpu.md#nvidia-cuda)
- [Ascend NPU](npu.md)
+49
View File
@@ -0,0 +1,49 @@
# GPU
vLLM-Omni is a Python library that supports the following GPU variants. Select your GPU type to see vendor specific instructions:
[](){ #nvidia-cuda }
=== "NVIDIA CUDA"
--8<-- "docs/getting_started/installation/gpu/cuda.inc.md:installation"
## Requirements
- OS: Linux
- Python: 3.12
!!! note
vLLM-Omni does not support Windows natively.
=== "NVIDIA CUDA"
--8<-- "docs/getting_started/installation/gpu/cuda.inc.md:requirements"
## Set up using Python
### Create a new Python environment
--8<-- "docs/getting_started/installation/python_env_setup.inc.md"
=== "NVIDIA CUDA"
--8<-- "docs/getting_started/installation/gpu/cuda.inc.md:set-up-using-python"
### Pre-built wheels
=== "NVIDIA CUDA"
--8<-- "docs/getting_started/installation/gpu/cuda.inc.md:pre-built-wheels"
[](){ #build-from-source }
### Build wheel from source
=== "NVIDIA CUDA"
--8<-- "docs/getting_started/installation/gpu/cuda.inc.md:build-wheel-from-source"
## Set up using Docker
### Pre-built images
@@ -0,0 +1,73 @@
# --8<-- [start:installation]
vLLM-Omni mainly contains python implementations for framework and models.
# --8<-- [end:installation]
# --8<-- [start:requirements]
- GPU: compute capability 7.0 or higher (e.g., V100, T4, RTX20xx, A100, L4, H100, etc.)
# --8<-- [end:requirements]
# --8<-- [start:set-up-using-python]
vLLM-Omni depends vLLM. So please follow instructions below mainly for vLLM.
!!! note
PyTorch installed via `conda` will statically link `NCCL` library, which can cause issues when vLLM tries to use `NCCL`. See <gh-issue:8420> for more details.
In order to be performant, vLLM has to compile many cuda kernels. The compilation unfortunately introduces binary incompatibility with other CUDA versions and PyTorch versions, even for the same PyTorch version with different building configurations.
Therefore, it is recommended to install vLLM and vLLM-Omni with a **fresh new** environment. If either you have a different CUDA version or you want to use an existing PyTorch installation, you need to build vLLM from source. See [build-from-source-vllm](https://docs.vllm.ai/en/stable/getting_started/installation/gpu/#build-wheel-from-source) for more details.
# --8<-- [start:pre-built-wheels]
## Installation of vLLM
Now we build it based on vLLM v0.11.0. Please install it with command below.
```bash
uv pip install vllm==0.11.0 --torch-backend=auto
```
## Installation of vLLM-Omni
```bash
uv pip install vllm-omni
```
# --8<-- [end:pre-built-wheels]
# --8<-- [start:build-wheel-from-source]
## Installation of vLLM
If you want to check or debug with source code of vLLM, install stable release version of vllm with 0.11.0 from source with pre-built wheel file.
```bash
git clone https://github.com/vllm-project/vllm.git
cd vllm
git checkout v0.11.0
```
Set up environment variables to get pre-built wheels. If there are internet problems, just download the whl file manually. And set VLLM_PRECOMPILED_WHEEL_LOCATION as your local absolute path of whl file.
```bash
export VLLM_PRECOMPILED_WHEEL_LOCATION=https://github.com/vllm-project/vllm/releases/download/v0.11.0/vllm-0.11.0-cp38-abi3-manylinux1_x86_64.whl
```
Install vllm with command below (If you have no existing PyTorch).
```bash
uv pip install --editable .
```
Install vllm with command below (If you already have PyTorch).
```bash
python use_existing_torch.py
uv pip install -r requirements/build.txt
uv pip install --no-build-isolation --editable .
```
## Installation of vLLM-omni
Install additional requirements for vllm-omni
```bash
git clone https://github.com/vllm-project/vllm-omni.git
cd vllm_omni
uv pip install -e .
```
# --8<-- [end:build-wheel-from-source]
+3
View File
@@ -0,0 +1,3 @@
# Ascend-NPU
vLLM-Omni is a Python library that supports the following NPU variants. Select your NPU type to see vendor specific instructions:
@@ -0,0 +1,5 @@
# --8<-- [start:installation]
vLLM-Omni mainly contains python implementations for framework and models.
# --8<-- [end:installation]
@@ -0,0 +1,6 @@
It's recommended to use [uv](https://docs.astral.sh/uv/), a very fast Python environment manager, to create and manage Python environments. Please follow the [documentation](https://docs.astral.sh/uv/#getting-started) to install `uv`. After installing `uv`, you can create a new Python environment using the following commands:
```bash
uv venv --python 3.12 --seed
source .venv/bin/activate
```
+23
View File
@@ -0,0 +1,23 @@
# Quickstart
This guide will help you quickly get started with vLLM-Omni to perform:
- Offline batched inference
- Online serving using OpenAI-compatible server
## Prerequisites
- OS: Linux
- Python: 3.12
## Installation
Please refer to [installation](installation/README.md)
## Offline Inference
Please refer to [offline inference](../user_guide/examples/offline_inference/qwen2_5_omni.md)
## Online Serving with OpenAI-Completions API
Please refer to [online serving](../user_guide/examples/online_serving/README.md)