- Updated the audio tensor extraction logic to access the correct output structure in Qwen2.5 Omni, Qwen3 Omni, and Qwen3 TTS implementations.
- Ensured consistency in handling audio outputs for improved functionality.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Eliminated the _process_embeds method and related imports from preprocess.py, streamlining the input preprocessing workflow.
- Updated the main processing logic to focus solely on token inputs, enhancing clarity and maintainability.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Reintroduced the load_chat_template import in api_server.py for better template handling.
- Updated the logic in serving_chat.py to streamline the resolution of chat template content formats, enhancing clarity and functionality.
- Adjusted the application of chat templates to ensure proper handling of tools and chat templates.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Changed the import path for multimodal processing to improve module organization.
- Removed the unused multimodal configuration parameter from the Qwen3OmniMoeThinker initialization, streamlining the class definition.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Removed the unused import of supports_mm_encoder_only from interfaces.
- Updated the dummy run check to utilize the multimodal configuration directly, enhancing clarity and future extensibility.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Updated import statements in chat_utils.py to replace the internal content format type with the public one for better clarity.
- Refactored api_server.py to streamline the chat template loading process by replacing the previous method with a direct function call.
- Enhanced serving_chat.py by reintroducing safe_apply_chat_template functions for better template handling.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Updated the import path for AnthropicServingMessages in api_server.py to enhance clarity and maintain consistency with the new module structure.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Refactored import statements in api_server.py to enhance module organization, including changes to the routing and server setup.
- Updated the method for removing routes to ensure compatibility with the new app structure.
- Adjusted the handling of chat completion requests to improve error responses and streamline processing.
- Modified serving modules to align with the new import paths and enhance code clarity.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Added a warning in GPUARWorker and GPUGenerationWorker to indicate that the v2 model runner does not include omni hooks, forcing the use of the v1 model runner.
- Updated import statements in gpu_ar_model_runner.py and gpu_generation_model_runner.py to include necessary functions from parallel_state.
- Removed unused imports from both model runner files to streamline the code.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Deleted the MRotaryEmbedding class and its associated methods from mrope.py to streamline the codebase.
- Removed the get_llm_pos_ids_for_vision function from vision.py as it is no longer needed.
- Updated import statements in qwen2_5_omni.py to reflect the removal of the vision module.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Updated OmniInputProcessor to streamline input processing, including enhanced handling of prompt embeddings and additional information.
- Introduced OmniTextPrompt and OmniEmbedsInputs to support new input structures with optional metadata.
- Modified input preprocessing methods to accommodate new prompt types and ensure compatibility with multimodal inputs.
- Improved error handling and type checking for data parallel configurations.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Changed type hint for stage_connector_spec from 'any' to 'Any' for improved type checking and consistency.
- Ensured compatibility with typing standards in the arg_utils module.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Updated OmniARScheduler to wrap output in OmniSchedulerOutput, including transfer metadata for finished requests needing KV cache transfer.
- Enhanced OmniGenerationScheduler to handle routed experts and finish reasons more effectively, ensuring proper output structure.
- Introduced performance statistics handling in both schedulers for improved monitoring.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Introduced mm_encoder_only parameter to OmniModelConfig.
- Added a deprecation warning for the `--convert mm_encoder_only` option, advising users to switch to `--mm-encoder-only`.
Signed-off-by: [Your Name] <your.email@example.com>
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
- Modified the constructor to accept torch.Tensor as an option for prompt_embeds.
- Introduced a new static method _maybe_decode_prompt_embeds to handle the conversion from PromptEmbedsPayload to torch.Tensor.
- Preserved the original prompt_embeds payload for reference.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
tzhouam