[Feature] Add AllGather-KV sequence-parallel attention str… (#4968)
Signed-off-by: zhuhh97 <zhuhonghan@huawei.com> Signed-off-by: dph97 <nenbaying@163.com> Signed-off-by: zhuhh97 <zhh@unknown> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: dengyunyang <584797741@qq.com>
This commit is contained in:
33 files changed
+1181
-135
No files matched your search
@@ -473,6 +473,14 @@ class OmniServeCommand(CLISubcommand):
|
||||
help="Ring Sequence Parallelism degree for diffusion models. "
|
||||
"Equivalent to setting DiffusionParallelConfig.ring_degree.",
|
||||
)
|
||||
omni_config_group.add_argument(
|
||||
"--allgather-degree",
|
||||
dest="allgather_degree",
|
||||
type=int,
|
||||
default=None,
|
||||
help="AllGather-KV Sequence Parallelism degree for non-causal diffusion attention. "
|
||||
"Equivalent to setting DiffusionParallelConfig.allgather_degree.",
|
||||
)
|
||||
omni_config_group.add_argument(
|
||||
"--diffusion-quantization-config",
|
||||
type=json.loads,
|
||||
|
||||
Reference in new issue
Block a user