[Perf] Bagel KV-ready early forwarding and time step consistency for /v1/chat/completions (#2398)

Signed-off-by: natureofnature <wzliu@connect.hku.hk>
This commit is contained in:
NATURE authored and GitHub committed 2026-04-01 08:50:45 +00:00
1 parent 08cb436d4e
commit d40840b114
7 files changed
+102 -37

No files matched your search

@@ -168,7 +168,6 @@ def main():
params_list = omni.default_sampling_params_list
if args.modality in ("text2img", "img2img"):
params_list[0].max_tokens = 1 # type: ignore
if len(params_list) > 1:
diffusion_params = params_list[1]
diffusion_params.num_inference_steps = args.steps # type: ignore