[Feat] Add NPU Backend support for vLLM-Omni (#89)
Signed-off-by: gcanlin <canlinguosdu@gmail.com>
This commit is contained in:
1 parent
88c00fb271
commit
ec9e04027d
25 files changed
+2932
-85
No files matched your search
@@ -1,3 +1,23 @@
|
||||
# Ascend-NPU
|
||||
# NPU
|
||||
|
||||
vLLM-Omni is a Python library that supports the following NPU variants. Select your NPU type to see vendor specific instructions:
|
||||
vLLM-Omni supports NPU through the vLLM Ascend Plugin (vllm-ascend). This is a community maintained hardware plugin for running vLLM on NPU.
|
||||
|
||||
## Requirements
|
||||
|
||||
- OS: Linux
|
||||
- Python: 3.12
|
||||
|
||||
!!! note
|
||||
vLLM-Omni is currently not natively supported on Windows.
|
||||
|
||||
=== "NPU"
|
||||
|
||||
--8<-- "docs/getting_started/installation/npu/npu.inc.md:requirements"
|
||||
|
||||
## Installation
|
||||
|
||||
### Recommended
|
||||
|
||||
=== "NPU"
|
||||
|
||||
--8<-- "docs/getting_started/installation/npu/npu.inc.md:installation"
|
||||
Reference in new issue
Block a user