Update vLLM wheel version and adjust GPU memory utilization in CI configuration
- Updated the vLLM wheel commit hash in Dockerfile.ci to a60985b07eaf0fefa4c353ad2309b745707591ed. - Modified the mock shared memory handle name in test_basic_connectors.py for clarity. - Increased GPU memory utilization to 0.9 and adjusted max_num_batched_tokens to 4096 in qwen2_5_omni_ci.yaml to enable testing on H800. - Refactored GPU model runner classes to remove deprecated activate_lora parameter and ensure compatibility with num_active_loras. - Enhanced request handling in the OmniGPUModelRunner class to improve streaming request updates. Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
1 parent
26744bcb35
commit
473d3ecd29
5 files changed
+44
-41
No files matched your search
@@ -86,7 +86,7 @@ def test_put_get_shm(mocker: MockerFixture, shm_connector, monkeypatch: pytest.M
|
||||
data = {"large": "x" * 200}
|
||||
|
||||
# Mock SHM return values
|
||||
mock_handle = {"name": "test_shm", "size": 200}
|
||||
mock_handle = {"name": "req_2", "size": 200}
|
||||
mock_write = mocker.MagicMock(return_value=mock_handle)
|
||||
monkeypatch.setattr("vllm_omni.distributed.omni_connectors.connectors.shm_connector.shm_write_bytes", mock_write)
|
||||
|
||||
|
||||
Reference in new issue
Block a user