[Model] Add BAGEL step execution and continuous batching (#6359)

This commit is contained in:
Trigger authored and GitHub committed 2026-09-04 01:36:11 +00:00
1 parent fda3af8bbc
commit e43c787544
24 files changed
+1965 -108

No files matched your search

@@ -150,8 +150,17 @@ that do not belong in the shared contract should be stored in `state.extra`.
Queueing and lifecycle metadata remain in the scheduler's request state.
Current native pipelines that explicitly enable step execution include
Qwen-Image, HunyuanImage3, and Helios. Step execution alone does not imply
continuous-batching support: Qwen-Image accepts batched step states.
Qwen-Image, BAGEL, HunyuanImage3, and Helios. Step execution alone does not
imply continuous-batching support: Qwen-Image accepts batched step states.
BAGEL accepts batched step states for image generation with its default,
think-mode, and single-stage deploy configs. Only the diffusion stage uses the
step protocol in two-stage deployments; explicit single-stage text output
retains complete-request execution. BAGEL schedules exactly
`num_inference_steps` denoising updates, including a one-step request. When a
finished stepwise request is followed by an explicit text fallback wave, the
runner retires its step state and paged diffusion-KV rows before the full
forward dispatch. BAGEL's step path does not currently support sequence
parallelism or a diffusion cache backend.
HunyuanImage3 accepts batched step states only when its resolved self-attention
backend is `TORCH_SDPA`; otherwise it rejects groups larger than one request.
Configure `DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA` or