[ROCm] [CI] Optimize Dockerfilerocm and reduce build time on CI (#885)
Signed-off-by: tjtanaa <tunjian.tan@embeddedllm.com>
This commit is contained in:
1 parent
58efc5dde3
commit
4869311f2f
4 files changed
+47
-34
No files matched your search
@@ -34,8 +34,28 @@ DOCKER_BUILDKIT=1 docker build \
|
||||
|
||||
#### Launch the docker image
|
||||
|
||||
##### Launch with OpenAI API Server
|
||||
|
||||
```
|
||||
docker run -it \
|
||||
docker run --rm \
|
||||
--group-add=video \
|
||||
--ipc=host \
|
||||
--cap-add=SYS_PTRACE \
|
||||
--security-opt seccomp=unconfined \
|
||||
--device /dev/kfd \
|
||||
--device /dev/dri \
|
||||
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||
--env "HF_TOKEN=$HF_TOKEN" \
|
||||
-p 8091:8091 \
|
||||
--ipc=host \
|
||||
vllm-omni-rocm \
|
||||
--model Qwen/Qwen3-Omni-30B-A3B-Instruct --port 8091
|
||||
```
|
||||
|
||||
##### Launch with interactive session for development
|
||||
|
||||
```
|
||||
docker run --rm -it \
|
||||
--network=host \
|
||||
--group-add=video \
|
||||
--ipc=host \
|
||||
@@ -44,8 +64,9 @@ docker run -it \
|
||||
--device /dev/kfd \
|
||||
--device /dev/dri \
|
||||
-v <path/to/model>:/app/model \
|
||||
vllm-omni-rocm \
|
||||
bash
|
||||
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||
--entrypoint bash \
|
||||
vllm-omni-rocm
|
||||
```
|
||||
|
||||
# --8<-- [end:build-docker]
|
||||
@@ -57,8 +78,7 @@ vLLM-Omni offers an official docker image for deployment. These images are built
|
||||
#### Launch vLLM-Omni Server
|
||||
Here's an example deployment command that has been verified on 2 x MI300's:
|
||||
```bash
|
||||
docker run -it \
|
||||
--network=host \
|
||||
docker run --rm \
|
||||
--group-add=video \
|
||||
--ipc=host \
|
||||
--cap-add=SYS_PTRACE \
|
||||
@@ -68,14 +88,15 @@ docker run -it \
|
||||
-v <path/to/model>:/app/model \
|
||||
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||
--env "HF_TOKEN=$HF_TOKEN" \
|
||||
-p 8091:8091 \
|
||||
vllm/vllm-omni-rocm:v0.14.0 \
|
||||
vllm serve --model Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
|
||||
--model Qwen/Qwen3-Omni-30B-A3B-Instruct --omni --port 8091
|
||||
```
|
||||
|
||||
#### Launch an interactive terminal with prebuilt docker image.
|
||||
If you want to run in dev environment you can launch the docker image as follows:
|
||||
```bash
|
||||
docker run -it \
|
||||
docker run --rm -it \
|
||||
--network=host \
|
||||
--group-add=video \
|
||||
--ipc=host \
|
||||
@@ -86,8 +107,8 @@ docker run -it \
|
||||
-v <path/to/model>:/app/model \
|
||||
-v ~/.cache/huggingface:/root/.cache/huggingface \
|
||||
--env "HF_TOKEN=$HF_TOKEN" \
|
||||
vllm/vllm-omni-rocm:v0.14.0 \
|
||||
bash
|
||||
--entrypoint bash \
|
||||
vllm/vllm-omni-rocm:v0.14.0
|
||||
```
|
||||
|
||||
# --8<-- [end:pre-built-images]
|
||||
Reference in new issue
Block a user