- Eliminated the _process_embeds method and related imports from preprocess.py, streamlining the input preprocessing workflow.
- Updated the main processing logic to focus solely on token inputs, enhancing clarity and maintainability.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Updated OmniInputProcessor to streamline input processing, including enhanced handling of prompt embeddings and additional information.
- Introduced OmniTextPrompt and OmniEmbedsInputs to support new input structures with optional metadata.
- Modified input preprocessing methods to accommodate new prompt types and ensure compatibility with multimodal inputs.
- Improved error handling and type checking for data parallel configurations.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Introduced `OmniModelRunnerOutput` and `OmniRequestOutput` classes for handling multimodal outputs.
- Implemented `OmniEngineCoreRequest` and `OmniInputPreprocessor` to manage prompt embeddings and additional information.
- Created `OmniTokensPrompt`, `OmniTokenInputs`, and `OmniEmbedsPrompt` for enhanced token handling.
- Added parsing and preprocessing functions to support new input structures.
- Removed the obsolete `diffusion_engine.py` file as part of the refactor.
tzhouam