Refactor OmniEngineArgs and AsyncOmniEngineArgs to improve compatibility with partial payloads by using getattr for various attributes. Update GPUGenerationModelRunner and OmniGPUModelRunner to replace the activate_lora parameter with num_active_loras for better clarity and backward compatibility. This change enhances the handling of LoRA activation during model runs.
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
1 parent
cd2ffbf2a0
commit
0099eebb3f
4 files changed
+77
-38
No files matched your search
@@ -144,7 +144,6 @@ class Qwen3MoeLLMModel(_Qwen3MoeLLMModel):
|
||||
positions: torch.Tensor,
|
||||
intermediate_tensors: IntermediateTensors | None = None,
|
||||
inputs_embeds: torch.Tensor | None = None,
|
||||
*,
|
||||
capture_layer_indices: Sequence[int] | None = None,
|
||||
return_hidden_states: bool = False,
|
||||
deepstack_input_embeds: IntermediateTensors | None = None,
|
||||
|
||||
Reference in new issue
Block a user