[Model] Add BAGEL step execution and continuous batching (#6359)

This commit is contained in:
Trigger authored and GitHub committed 2026-09-04 01:36:11 +00:00
1 parent fda3af8bbc
commit e43c787544
24 files changed
+1965 -108

No files matched your search

+10 -2
View File
@@ -94,8 +94,16 @@ validating a model or debugging correctness, then increase it for
multi-request throughput.
Step execution is capability-based, not a generic switch for every diffusion
model. Qwen-Image supports step-wise continuous batching. HunyuanImage3 also
supports it, but only when its resolved self-attention backend is `TORCH_SDPA`;
model. Qwen-Image supports step-wise continuous batching. BAGEL supports
step-wise continuous batching for image generation with `bagel.yaml`,
`bagel_think.yaml`, and `bagel_single_stage.yaml`. In the two-stage topologies,
only the diffusion stage uses step execution; the autoregressive Thinker is
unchanged. Explicit text-output requests in the single-stage topology retain
the complete-request path. BAGEL schedules exactly `num_inference_steps`
denoising updates, so one-step image requests are supported. BAGEL step
execution does not currently support sequence parallelism or a diffusion cache
backend. HunyuanImage3 also supports step execution, but only
when its resolved self-attention backend is `TORCH_SDPA`;
set `DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA` or configure
`diffusion_attention_config.default.backend=TORCH_SDPA` before using
`--max-num-seqs >1`. See the