[Bugfix] Remove duplicated config keyword max batch size (#1851)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk> Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
This commit is contained in:
1 parent
4c2293743b
commit
05d8de813e
67 files changed
+226
-159
No files matched your search
@@ -158,7 +158,7 @@ The stage initialized but ran out of memory during processing.
|
||||
|
||||
**Solutions:**
|
||||
1. Reduce `max_num_batched_tokens`
|
||||
2. Reduce `max_batch_size` in runtime config
|
||||
2. Reduce `max_num_seqs` in engine_args
|
||||
3. Lower `gpu_memory_utilization` slightly
|
||||
4. Enable quantization if supported
|
||||
|
||||
|
||||
Reference in new issue
Block a user