- Reintroduced the load_chat_template import in api_server.py for better template handling.
- Updated the logic in serving_chat.py to streamline the resolution of chat template content formats, enhancing clarity and functionality.
- Adjusted the application of chat templates to ensure proper handling of tools and chat templates.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Changed the import path for multimodal processing to improve module organization.
- Removed the unused multimodal configuration parameter from the Qwen3OmniMoeThinker initialization, streamlining the class definition.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Removed the unused import of supports_mm_encoder_only from interfaces.
- Updated the dummy run check to utilize the multimodal configuration directly, enhancing clarity and future extensibility.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Updated import statements in chat_utils.py to replace the internal content format type with the public one for better clarity.
- Refactored api_server.py to streamline the chat template loading process by replacing the previous method with a direct function call.
- Enhanced serving_chat.py by reintroducing safe_apply_chat_template functions for better template handling.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Modified the pip install command for vllm to use the correct URL format for direct installation from the specified commit hash, ensuring a more reliable installation process.
- Removed the fallback option for pre-release installation to streamline the installation command.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Updated the import path for AnthropicServingMessages in api_server.py to enhance clarity and maintain consistency with the new module structure.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Changed the pip install command for vllm to download the wheel file directly instead of using an extra index URL, ensuring a more reliable installation process.
- Added a fallback option to install the pre-release version of vllm if the direct installation fails.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Refactored import statements in api_server.py to enhance module organization, including changes to the routing and server setup.
- Updated the method for removing routes to ensure compatibility with the new app structure.
- Adjusted the handling of chat completion requests to improve error responses and streamline processing.
- Modified serving modules to align with the new import paths and enhance code clarity.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Modified the pip install command for vllm to include --upgrade, --force-reinstall flags, and an extra index URL, ensuring the latest version is installed from the specified source during testing.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Modified the pip install command for vllm to include --upgrade and --force-reinstall flags, ensuring the latest version is installed during the CI process.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Added a warning in GPUARWorker and GPUGenerationWorker to indicate that the v2 model runner does not include omni hooks, forcing the use of the v1 model runner.
- Updated import statements in gpu_ar_model_runner.py and gpu_generation_model_runner.py to include necessary functions from parallel_state.
- Removed unused imports from both model runner files to streamline the code.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Deleted the MRotaryEmbedding class and its associated methods from mrope.py to streamline the codebase.
- Removed the get_llm_pos_ids_for_vision function from vision.py as it is no longer needed.
- Updated import statements in qwen2_5_omni.py to reflect the removal of the vision module.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Updated OmniInputProcessor to streamline input processing, including enhanced handling of prompt embeddings and additional information.
- Introduced OmniTextPrompt and OmniEmbedsInputs to support new input structures with optional metadata.
- Modified input preprocessing methods to accommodate new prompt types and ensure compatibility with multimodal inputs.
- Improved error handling and type checking for data parallel configurations.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Changed type hint for stage_connector_spec from 'any' to 'Any' for improved type checking and consistency.
- Ensured compatibility with typing standards in the arg_utils module.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
tzhouam