[Model][Feat] Support Minimax-H3 quality grading requests through dynamic loading/unloading with Cache-DiT (#5853)
Signed-off-by: mglyn <1203789601@qq.com>
This commit is contained in:
1 parent
e906f41dc8
commit
d8c35663b4
28 files changed
+1183
-18
No files matched your search
@@ -55,6 +55,10 @@ class StepBatchSamplingParamsKey:
|
||||
true_cfg_scale: float | None = None
|
||||
cfg_normalize: bool = False
|
||||
|
||||
# Request-scoped execution policy. A batch must use one quality mode
|
||||
# because model acceleration hooks are shared by the whole worker batch.
|
||||
quality: str | None = None
|
||||
|
||||
# Output count. Requests with different num_outputs_per_prompt produce
|
||||
# differently shaped outputs and cannot share a batch.
|
||||
num_outputs_per_prompt: int = 1
|
||||
@@ -108,6 +112,11 @@ class RequestBatchSamplingParamsKey:
|
||||
layers: int = 4
|
||||
use_en_prompt: bool = False
|
||||
|
||||
# Request-scoped execution policy. Keep accelerated and reference-path
|
||||
# requests in separate worker batches. ``None`` delegates the default to
|
||||
# the model and must remain distinct from explicit ``lossless``.
|
||||
quality: str | None = None
|
||||
|
||||
# LoRA identity.
|
||||
lora_int_id: int | None = None
|
||||
lora_scale: float = 1.0
|
||||
|
||||
Reference in new issue
Block a user