[Feature] Support online inference (#64)
Signed-off-by: Gaohan123 <gaohan19@huawei.com>
This commit is contained in:
1 parent
bf82e18374
commit
c5a89302f1
52 files changed
+3666
-685
No files matched your search
@@ -2,7 +2,7 @@
|
||||
|
||||
This document contains comprehensive API design documentation for all core modules in vLLM-omni. These templates provide a standardized structure for designing and implementing the core, engine, executor, and worker modules.
|
||||
|
||||
## 📋 Module API
|
||||
## 📋 Module API
|
||||
|
||||
### Core Module API
|
||||
**Core module** provides fundamental scheduling, caching, and resource management functionality.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# Architecture Overview
|
||||
# Architecture Overview
|
||||
|
||||
# Introduction
|
||||
|
||||
|
||||
@@ -10,4 +10,3 @@ This section contains design documents and architecture specifications for vLLM-
|
||||
## API Design Documentation
|
||||
|
||||
- [vLLM-omni API Documentation](api_design_doc.md)
|
||||
|
||||
Reference in new issue
Block a user