Compare commits

..
Author SHA1 Message Date
Roger Wang ed875fa1b2 typo
Signed-off-by: Roger Wang <hey@rogerw.io>
2025-12-03 22:09:30 +00:00
Roger Wang e9e1e77f49 add dockerhub
Signed-off-by: Roger Wang <hey@rogerw.io>
2025-12-03 22:08:42 +00:00
2 changed files with 6 additions and 15 deletions

No files matched your search

-14
View File
@@ -1,14 +0,0 @@
ARG VLLM_BASE_IMAGE=vllm/vllm-openai
ARG VLLM_BASE_TAG=v0.11.0
FROM ${VLLM_BASE_IMAGE}:${VLLM_BASE_TAG}
ARG APP_DIR=/workspace/vllm-omni
WORKDIR ${APP_DIR}
COPY . .
# Install vllm-omni into the same uv-managed Python environment used by the base image.
RUN uv pip install --python "$(python3 -c 'import sys; print(sys.executable)')" --no-cache-dir .
RUN ln -sf /usr/bin/python3 /usr/bin/python
ENTRYPOINT ["vllm", "serve", "--omni"]
+6 -1
View File
@@ -37,4 +37,9 @@ vLLM-Omni is a Python library that supports the following GPU variants. The libr
## Set up using Docker
### Pre-built images
To be released... Stay tuned!
vLLM-Omni offers an official docker image for deployment. These images are built on top of vLLM docker images
and available on Docker Hub as [vllm/vllm-omni](https://hub.docker.com/r/vllm/vllm-omni/tags). The version of vLLM-Omni
indicates which release of vLLM it is based on.
!!! tip
You can use this docker image to serve models the same way you would do in vLLM! To do so, make sure you overwrite the default entrypoint (`vllm serve --omni`) which works only for models supported in the vLLM-Omni project.