Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
1 parent
31c5465e00
commit
7e9cae56fc
5 files changed
+44
-196
No files matched your search
@@ -34,7 +34,7 @@ RUN uv pip install --system --upgrade --force-reinstall --prerelease=allow vllm
|
||||
RUN uv pip uninstall --system nvidia-cublas-cu12
|
||||
# Install vllm-omni into the same uv-managed Python environment used by the base image.
|
||||
# Use bash -c so that $(python3 -c ...) is expanded inside the container.
|
||||
RUN /bin/bash -c 'uv pip install --python "$(python3 -c "import sys; print(sys.executable)")" --no-cache-dir -e ".[dev]"'
|
||||
RUN /bin/bash -c 'VLLM_USE_PRECOMPILED=1 uv pip install --python "$(python3 -c "import sys; print(sys.executable)")" --no-cache-dir -e ".[dev]"'
|
||||
|
||||
|
||||
RUN /bin/bash -c 'FLASHINFER_CUDA_TAG="$(python3 -c "import torch; print((torch.version.cuda or \"12.4\").replace(\".\", \"\"))")" && \
|
||||
|
||||
Reference in new issue
Block a user