[Fix] Restore MammothModa2 support
Signed-off-by: hsliuustc0106 <hsliuustc0106@users.noreply.github.com>
This commit is contained in:
1 parent
c6df39e9f7
commit
fc5bafb2af
35 files changed
+4671
-13
No files matched your search
@@ -792,9 +792,9 @@ class GPUGenerationModelRunner(OmniGPUModelRunner, OmniConnectorModelRunnerMixin
|
||||
input_ids = self.input_ids.gpu[:num_tokens_padded]
|
||||
inputs_embeds = None
|
||||
|
||||
# Some generation-stage models require model-specific runtime
|
||||
# information (such as image size and conditioning embeddings)
|
||||
# even during the dummy profiling run that vLLM uses to
|
||||
# Some generation-stage models (e.g. MammothModa2DiTPipeline) require
|
||||
# model-specific runtime information (such as image size and conditioning
|
||||
# embeddings) even during the dummy profiling run that vLLM uses to
|
||||
# estimate KV-cache capacity. get_dummy_runtime_additional_information
|
||||
# provides placeholder values of the correct shape so that the profiling
|
||||
# run does not raise an error due to missing inputs.
|
||||
|
||||
Reference in new issue
Block a user