[BugFix] fix possible shape mismatch when using ROPE (#4655)

Signed-off-by: Semmer <semmer@live.cn>
This commit is contained in:
Ting FU authored and GitHub committed 2026-07-07 17:54:39 +08:00
1 parent 31c9c751fc
commit 815abce2ba
2 files changed
+242

No files matched your search

+6
View File
@@ -188,6 +188,12 @@ class RotaryEmbedding(CustomOp):
cos: torch.Tensor,
sin: torch.Tensor,
) -> torch.Tensor:
# All batch elements share the same rotary position encoding.
# Strip the batch dim so the underlying op broadcasts over the batch,
# consistent with forward_cuda / forward_hip / apply_rotary_emb_mindiesd.
if cos.dim() == 3:
cos = cos[0]
sin = sin[0]
return apply_rotary_emb_torch(
x,
cos,