[Update] Upgrade vLLM wheel version and enhance metrics handling
- Updated the vLLM wheel to commit 24650715105a26b41c88f4164777b5e1a0f3200b in Dockerfile.ci. - Modified the `calculate_metrics` function to handle `tokenizer` as optional (can be None) in metrics.py. - Adjusted the `benchmark` function to accept `tokenizer` as optional in patch.py. - Enhanced condition checks in OmniARScheduler to ensure valid token generation in omni_ar_scheduler.py. - Updated input processing to include supported tasks in input_processor.py. - Cleaned up unnecessary code in output_processor.py. - Added support for SSL ciphers in the API server configuration in api_server.py. - Enabled speculative decoding in GPU model runners for improved performance in gpu_generation_model_runner.py and gpu_model_runner.py. Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
1 parent
22e01c416b
commit
a252918008
9 files changed
+30
-26
No files matched your search
@@ -223,7 +223,7 @@ async def benchmark(
|
||||
base_url: str,
|
||||
model_id: str,
|
||||
model_name: str,
|
||||
tokenizer: PreTrainedTokenizerBase,
|
||||
tokenizer: PreTrainedTokenizerBase | None,
|
||||
input_requests: list[SampleRequest],
|
||||
logprobs: int | None,
|
||||
request_rate: float,
|
||||
|
||||
Reference in new issue
Block a user