Add OmniMRotaryEmbedding for multimodal position computation
- Introduced OmniMRotaryEmbedding to extend MRotaryEmbedding with methods for computing input positions across various multimodal scenarios, including image, video, and audio inputs. - Updated patch.py to replace the original MRotaryEmbedding with OmniMRotaryEmbedding in the module patching logic. - Created new files for the OmniMRotaryEmbedding implementation and its associated methods. Signed-off-by: [Your Name] <[Your Email]> Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
1 parent
3208afd82b
commit
7aea8e8d91
17 files changed
+744
-57
No files matched your search
@@ -4,13 +4,12 @@ from vllm.inputs.parse import (
|
||||
ParsedStrPrompt,
|
||||
ParsedTextPrompt,
|
||||
ParsedTokensPrompt,
|
||||
SingletonPrompt,
|
||||
)
|
||||
|
||||
from vllm_omni.inputs.data import OmniSingletonPrompt
|
||||
|
||||
|
||||
def parse_singleton_prompt_omni(
|
||||
prompt: OmniSingletonPrompt,
|
||||
prompt: SingletonPrompt,
|
||||
) -> ParsedSingletonPrompt:
|
||||
"""Parse a singleton prompt into a typed parsed prompt.
|
||||
|
||||
|
||||
Reference in new issue
Block a user