[Docs] Add installation guide for Moore Threads (MUSA) GPUs (#2359)
Signed-off-by: Xiaodong Ye <yeahdongcn@gmail.com> Co-authored-by: Canlin Guo <canlinguosdu@gmail.com>
This commit is contained in:
1 parent
08e2e1fc8e
commit
0304c975d5
3 files changed
+86
No files matched your search
@@ -6,4 +6,5 @@ vLLM-Omni supports the following hardware platforms:
|
||||
- [NVIDIA CUDA](gpu.md)
|
||||
- [AMD ROCm](gpu.md)
|
||||
- [Intel XPU](gpu.md)
|
||||
- [MThreads MUSA](gpu.md)
|
||||
- [NPU](npu.md)
|
||||
@@ -22,6 +22,10 @@ vLLM-Omni is a Python library that supports the following GPU variants. The libr
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/xpu.inc.md:requirements"
|
||||
|
||||
=== "MThreads MUSA"
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/musa.inc.md:requirements"
|
||||
|
||||
## Set up using Python
|
||||
|
||||
### Create a new Python environment
|
||||
@@ -44,6 +48,10 @@ Note: Pre-built wheels are currently available for vLLM-Omni 0.11.0rc1, 0.12.0rc
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/xpu.inc.md:pre-built-wheels"
|
||||
|
||||
=== "MThreads MUSA"
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/musa.inc.md:pre-built-wheels"
|
||||
|
||||
[](){ #build-from-source }
|
||||
|
||||
### Build wheel from source
|
||||
@@ -60,6 +68,10 @@ Note: Pre-built wheels are currently available for vLLM-Omni 0.11.0rc1, 0.12.0rc
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/xpu.inc.md:build-wheel-from-source"
|
||||
|
||||
=== "MThreads MUSA"
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/musa.inc.md:build-wheel-from-source"
|
||||
|
||||
## Set up using Docker
|
||||
|
||||
### Pre-built images
|
||||
@@ -76,6 +88,10 @@ Note: Pre-built wheels are currently available for vLLM-Omni 0.11.0rc1, 0.12.0rc
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/xpu.inc.md:pre-built-images"
|
||||
|
||||
=== "MThreads MUSA"
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/musa.inc.md:pre-built-images"
|
||||
|
||||
### Build your own docker image
|
||||
|
||||
=== "AMD ROCm"
|
||||
@@ -85,3 +101,7 @@ Note: Pre-built wheels are currently available for vLLM-Omni 0.11.0rc1, 0.12.0rc
|
||||
=== "Intel XPU"
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/xpu.inc.md:build-docker"
|
||||
|
||||
=== "MThreads MUSA"
|
||||
|
||||
--8<-- "docs/getting_started/installation/gpu/musa.inc.md:build-docker"
|
||||
@@ -0,0 +1,65 @@
|
||||
# --8<-- [start:requirements]
|
||||
|
||||
- GPU: Moore Threads GPU with MUSA SDK installed (validated on MTT S5000)
|
||||
|
||||
# --8<-- [end:requirements]
|
||||
# --8<-- [start:set-up-using-python]
|
||||
|
||||
vLLM-Omni for MUSA requires building from source. Pre-built wheels are not currently available.
|
||||
|
||||
!!! note
|
||||
MUSA platform requires vLLM-MUSA to be installed first.
|
||||
|
||||
# --8<-- [start:pre-built-wheels]
|
||||
|
||||
# --8<-- [end:pre-built-wheels]
|
||||
|
||||
# --8<-- [start:build-wheel-from-source]
|
||||
|
||||
#### Prerequisites
|
||||
|
||||
- **MUSA SDK**: Download from [MUSA SDK Download](https://developer.mthreads.com/sdk/download/musa)
|
||||
- **torchada**: CUDA→MUSA compatibility layer for PyTorch (`pip install torchada`)
|
||||
- **mthreads-ml-py**: MTML Python bindings (`pip install mthreads-ml-py`)
|
||||
- **MATE**: MUSA AI Tensor Engine ([GitHub](https://github.com/MooreThreads/mate))
|
||||
|
||||
#### Installation of vLLM-MUSA
|
||||
|
||||
```bash
|
||||
git clone https://github.com/MooreThreads/vllm-musa.git
|
||||
cd vllm-musa
|
||||
git checkout v0.18.0-dev
|
||||
pip install . --no-build-isolation -v
|
||||
```
|
||||
|
||||
#### Installation of vLLM-Omni
|
||||
|
||||
```bash
|
||||
git clone https://github.com/vllm-project/vllm-omni.git
|
||||
cd vllm-omni
|
||||
VLLM_OMNI_TARGET_DEVICE=musa pip install -e . --no-build-isolation
|
||||
```
|
||||
|
||||
For Gradio demos:
|
||||
|
||||
```bash
|
||||
pip install -e '.[demo]' --no-build-isolation
|
||||
```
|
||||
|
||||
#### Environment Variables
|
||||
|
||||
```bash
|
||||
export MUSA_VISIBLE_DEVICES=0,1
|
||||
export VLLM_WORKER_MULTIPROC_METHOD=spawn
|
||||
export VLLM_MUSA_CUSTOM_OP_USE_NATIVE=false
|
||||
```
|
||||
|
||||
# --8<-- [end:build-wheel-from-source]
|
||||
|
||||
# --8<-- [start:build-docker]
|
||||
|
||||
# --8<-- [end:build-docker]
|
||||
|
||||
# --8<-- [start:pre-built-images]
|
||||
|
||||
# --8<-- [end:pre-built-images]
|
||||
Reference in new issue
Block a user