Update Dockerfile for vllm wheel version and enhance SHM connector tests
- Updated Dockerfile to use a new vllm wheel version. - Modified test_put_get_inline to clarify that it always uses SHM and updated metadata assertions. - Added handling for speculative configuration in GPU model runners to delay clearing connector metadata. - Refactored payload decoding in OmniGPUModelRunner for better clarity and maintainability. Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
1 parent
e22ae62396
commit
861698f88f
10 files changed
+169
-224
No files matched your search
@@ -34,6 +34,7 @@ def _make_runner(multimodal_outputs):
|
||||
runner.use_async_scheduling = False
|
||||
runner.device = torch.device("cpu")
|
||||
runner.supports_mm_inputs = False
|
||||
runner.speculative_config = None
|
||||
return runner
|
||||
|
||||
|
||||
|
||||
Reference in new issue
Block a user