- Eliminated redundant numpy import statements to streamline the code.
- Changed the return type of extract_multimodal_outputs from dict to tuple for better clarity on output structure.
- Simplified the assignment of sampled_token_ids by removing the conditional check for "model_stage".
- Ensured consistent handling of sampled_token_ids for improved clarity and maintainability.
- Introduced OmniGPUModelRunner for enhanced GPU model execution with state management.
- Added OmniModelInputForGPU and its builder to support additional information in model inputs.
- Implemented methods for updating request states and handling multimodal inputs.
- Enhanced input preparation for CUDA graph execution and LoRA integration.
Jiangyun Zhu