[Feat][Perf][AuK]Support Online Serving and DiT CUDAGraph (#7469)
Signed-off-by: boatman <109857087+sphinxkkkbc@users.noreply.github.com> Signed-off-by: Yueqian Lin <linyueqian@outlook.com> Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com> Co-authored-by: Yueqian Lin <linyueqian@outlook.com>
This commit is contained in:
22 files changed
+1473
-38
No files matched your search
@@ -29,7 +29,7 @@ REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
SERVING_SPEECH = Path("vllm_omni/entrypoints/openai/serving_speech.py")
|
||||
|
||||
# Budgets. These may only go DOWN. See the module docstring.
|
||||
MAX_MODEL_TYPE_BRANCHES = 6
|
||||
MAX_MODEL_TYPE_BRANCHES = 5
|
||||
|
||||
_GUIDANCE = """
|
||||
Move the per-model behaviour into its adapter under
|
||||
|
||||
Reference in new issue
Block a user