Update Dockerfile for vllm wheel version and enhance SHM connector tests

- Updated Dockerfile to use a new vllm wheel version.
- Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions.
- Added handling for speculative configuration in GPU model runners to delay clearing connector metadata.
- Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability.

Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
tzhouam committed 2026-02-24 05:45:48 +00:00
1 parent e22ae62396
commit 861698f88f
10 files changed
+169 -224

No files matched your search

@@ -64,14 +64,13 @@ def shm_connector():
def test_put_get_inline(shm_connector):
"""Test inline transfer for small data."""
"""Test transfer for small data (currently always uses SHM)."""
data = {"small": "data"}
# Ensure data is smaller than threshold (100 bytes)
success, size, metadata = shm_connector.put("stage_0", "stage_1", "req_1", data)
assert success is True
assert "inline_bytes" in metadata
assert "shm" not in metadata
assert "shm" in metadata
assert "size" in metadata
# Retrieve
retrieved_data, ret_size = shm_connector.get("stage_0", "stage_1", "req_1", metadata)