Update vLLM wheel version in Dockerfile and enhance input processing for omni models. Added functionality to reinject omni-specific fields into processed prompts and improved handling of multimodal audio extraction in serving_chat.py. Adjusted SharedMemoryConnector logic for payload size checks and refined input preprocessing methods for better compatibility with multimodal data.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
tzhouam committed 2026-02-20 08:34:53 +00:00
1 parent 663976b8ca
commit 9869ea007e
8 files changed
+214 -96

No files matched your search

+2 -2
View File
@@ -14,11 +14,11 @@ RUN apt-get update && \
# Install vLLM wheel for the specific commit
# URL format: https://wheels.vllm.ai/<commit_hash>/
# Current vLLM wheel commit: d5fe3f702c2f4392515cdfde6fd0442271c74dda
# Current vLLM wheel commit: 16f24e87975ef4cd2c12879425062913ef62f6fd
# Note: --prerelease=allow is required because nightly/commit wheels use
# pre-release version numbers (e.g. 2.10.0.dev...) which uv ignores by default,
# causing it to fall back to the older stable release from PyPI.
RUN uv pip install --system --upgrade --force-reinstall --prerelease=allow vllm --extra-index-url https://wheels.vllm.ai/d5fe3f702c2f4392515cdfde6fd0442271c74dda
RUN uv pip install --system --upgrade --force-reinstall --prerelease=allow vllm --extra-index-url https://wheels.vllm.ai/16f24e87975ef4cd2c12879425062913ef62f6fd
RUN uv pip uninstall --system nvidia-cublas-cu12