[Bugfix] Remove duplicated config keyword max batch size (#1851)
Signed-off-by: tzhouam <tzhouam@connect.ust.hk> Signed-off-by: Zhou Taichang <tzhouam@connect.ust.hk>
This commit is contained in:
1 parent
4c2293743b
commit
05d8de813e
67 files changed
+226
-159
No files matched your search
@@ -967,7 +967,7 @@ hidden_states = hidden_states.reshape(batch_size, seq_len, -1)
|
||||
|
||||
1. **Reduce batch size:**
|
||||
```python
|
||||
omni.generate(prompts=[...], max_batch_size=2)
|
||||
omni.generate(prompts=[...], max_num_seqs=2)
|
||||
```
|
||||
|
||||
2. **Use smaller image size:**
|
||||
|
||||
Reference in new issue
Block a user