[Feature] Support online inference (#64)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
This commit is contained in:
1 parent
bf82e18374
commit
c5a89302f1
52 files changed
+3666
-685
No files matched your search
+3
-3
@@ -40,15 +40,15 @@ Example docstring:
|
||||
```python
|
||||
class OmniLLM:
|
||||
"""Main entry point for vLLM-omni inference.
|
||||
|
||||
|
||||
This class provides a high-level interface for running multi-modal
|
||||
inference with non-autoregressive models.
|
||||
|
||||
|
||||
Args:
|
||||
model: Model name or path
|
||||
stage_configs: Optional stage configurations
|
||||
**kwargs: Additional arguments passed to the engine
|
||||
|
||||
|
||||
Example:
|
||||
>>> llm = OmniLLM(model="Qwen/Qwen2.5-Omni")
|
||||
>>> outputs = llm.generate(prompts="Hello")
|
||||
|
||||
Reference in new issue
Block a user