[Feature] Support online inference (#64)

Signed-off-by: Gaohan123 <gaohan19@huawei.com>
This commit is contained in:
Gao Han authored and GitHub committed 2025-11-17 14:44:10 +08:00
1 parent bf82e18374
commit c5a89302f1
52 files changed
+3666 -685

No files matched your search

+3 -3
View File
@@ -40,15 +40,15 @@ Example docstring:
```python
class OmniLLM:
"""Main entry point for vLLM-omni inference.
This class provides a high-level interface for running multi-modal
inference with non-autoregressive models.
Args:
model: Model name or path
stage_configs: Optional stage configurations
**kwargs: Additional arguments passed to the engine
Example:
>>> llm = OmniLLM(model="Qwen/Qwen2.5-Omni")
>>> outputs = llm.generate(prompts="Hello")