Refactor Dockerfile.ci to use precompiled VLLM installation and remove unused parse.py file. Update serving_chat.py to streamline rendering logic and enhance compatibility with new API. Modify OmniInputPreprocessor to directly handle SingletonDictPrompt, improving input processing efficiency.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
tzhouam committed 2026-02-13 02:28:25 +00:00
1 parent 31c5465e00
commit 7e9cae56fc
5 files changed
+44 -196

No files matched your search

+1 -1
View File
@@ -34,7 +34,7 @@ RUN uv pip install --system --upgrade --force-reinstall --prerelease=allow vllm
RUN uv pip uninstall --system nvidia-cublas-cu12
# Install vllm-omni into the same uv-managed Python environment used by the base image.
# Use bash -c so that $(python3 -c ...) is expanded inside the container.
RUN /bin/bash -c 'uv pip install --python "$(python3 -c "import sys; print(sys.executable)")" --no-cache-dir -e ".[dev]"'
RUN /bin/bash -c 'VLLM_USE_PRECOMPILED=1 uv pip install --python "$(python3 -c "import sys; print(sys.executable)")" --no-cache-dir -e ".[dev]"'
RUN /bin/bash -c 'FLASHINFER_CUDA_TAG="$(python3 -c "import torch; print((torch.version.cuda or \"12.4\").replace(\".\", \"\"))")" && \