[Refactor] Declare model-local KV held outside the paged manager (#6171)
Signed-off-by: Yueqian Lin <linyueqian@outlook.com>
This commit is contained in:
1 parent
5a386d789f
commit
5c69fc2a8a
12 files changed
+1283
-3
No files matched your search
@@ -26,6 +26,7 @@ implementation contract; it is not, by itself, a general support claim.
|
||||
- [Async Omni Output Materialization](feature/omni_async_output_materialization.md)
|
||||
- [Runner-to-model Prefill/Decode Phase Contract](feature/preprocess_phase_contract.md)
|
||||
- [Automatic Prefix Caching in Omni Models](feature/prefix_caching.md)
|
||||
- [Model-local KV Caches](feature/model_local_kv_caches.md)
|
||||
- [Realtime AR-Diffusion Sessions](feature/realtime_ar_diffusion.md)
|
||||
|
||||
### Communication
|
||||
|
||||
Reference in new issue
Block a user