[Feat] Support MagCache (#1287)
Signed-off-by: Lancer <maruixiang6688@gmail.com> Co-authored-by: 汪志鹏 <wangzhipeng628@gmail.com> Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>
This commit is contained in:
13 files changed
+1677
-11
No files matched your search
+2
-1
@@ -5,6 +5,7 @@ Cache module for diffusion model inference acceleration.
|
||||
|
||||
This module provides a unified cache backend system for different caching strategies:
|
||||
- TeaCache: Timestep Embedding Aware Cache for adaptive transformer caching
|
||||
- MagCache: Magnitude-based Cache for adaptive transformer caching
|
||||
- cache-dit: DBCache, SCM, and TaylorSeer caching strategies
|
||||
|
||||
Cache backends are instantiated directly via their constructors and configured via OmniDiffusionConfig.
|
||||
@@ -26,9 +27,9 @@ from vllm_omni.diffusion.cache.teacache.backend import TeaCacheBackend
|
||||
|
||||
__all__ = [
|
||||
"CacheBackend",
|
||||
"TeaCacheConfig",
|
||||
"CacheContext",
|
||||
"TeaCacheBackend",
|
||||
"TeaCacheConfig",
|
||||
"apply_teacache_hook",
|
||||
"PromptEmbedCache",
|
||||
"install_prompt_embed_cache",
|
||||
|
||||
Reference in new issue
Block a user