refactor(inputs): build the Omni renderer as a subclass instead of a proxy
OmniRenderer wrapped the upstream renderer, forwarded unknown attributes via
__getattr__, and re-bound eight BaseRenderer methods (render_cmpl, render_chat,
process_for_engine, _process_enc_dec and their async twins) onto the wrapper
so that they would reach Omni's _process_singleton. That allow-list hid any
concrete-renderer override of those methods and would silently skip the
Omni extras for any future BaseRenderer entry point not on the list.
Make OmniRenderer a mixin and create the renderer as a real subclass of the
upstream class instead:
- omni_renderer_cls(base_cls) builds (and caches) Omni<Base> = (OmniRenderer,
base_cls); only _process_singleton / _process_singleton_async are extended,
everything else is the upstream renderer unchanged.
- build_omni_renderer(vllm_config) resolves the class the way upstream
renderer_from_config does (cached_tokenizer_from_config,
tokenizer_args_from_config, RENDERER_REGISTRY.load_renderer_cls) and
instantiates the Omni subclass; renderer_cls/tokenizer can be passed to
skip the resolution.
- build_stage0_input_processor passes the renderer to InputProcessor up
front instead of replacing it afterwards; the skip-tokenizer branch keeps
_build_token_only_renderer and never resolves a tokenizer.
Rebinding an existing instance's __class__ was rejected: CPython refuses it
for HfRenderer ("object layout differs"), so the subclass is constructed.
Tests: tests/inputs/test_preprocess.py builds the renderer with
omni_renderer_cls and adds checks that the subclass is created once, that a
concrete renderer's render_cmpl override is no longer suppressed, and that
build_omni_renderer resolves like upstream or honours an explicit class;
tests/engine/test_async_omni_engine_stage_init.py asserts the stage-0
processor receives the built subclass instance and that the skip-tokenizer
branch never calls build_omni_renderer. Two pre-existing mypy complaints in
test_preprocess.py are fixed with annotations.
Validation: ruff, pre-commit (incl. mypy) on the four files; the modules
import and omni_renderer_cls(HfRenderer) builds; tests/inputs (32),
tests/engine/test_async_omni_engine_stage_init.py (55) and tests/entrypoints
(548) pass on CPU.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MpXe2LeRrUQGjZfKFaocMZ
Signed-off-by: tzhouam <tzhouam@connect.ust.hk>
This commit is contained in:
1 parent
a58825ed6c
commit
fb94b87d4b
4 files changed
+184
-53
No files matched your search
@@ -54,7 +54,7 @@ from vllm_omni.engine.arg_utils import OmniEngineArgs
|
||||
from vllm_omni.entrypoints.stage_utils import _to_dict, set_stage_devices
|
||||
from vllm_omni.entrypoints.utils import filter_dataclass_kwargs
|
||||
from vllm_omni.inputs.data import OmniDiffusionSamplingParams, OmniSamplingParams
|
||||
from vllm_omni.inputs.preprocess import OmniRenderer
|
||||
from vllm_omni.inputs.preprocess import build_omni_renderer, omni_renderer_cls
|
||||
from vllm_omni.outputs.output_processor import MultimodalOutputProcessor
|
||||
from vllm_omni.platforms import current_omni_platform
|
||||
from vllm_omni.quantization.inc_config import OmniINCConfig
|
||||
@@ -1549,22 +1549,24 @@ class _TokenOnlyRenderer(BaseRenderer):
|
||||
|
||||
|
||||
def _build_token_only_renderer(stage_vllm_config: Any) -> BaseRenderer:
|
||||
return _TokenOnlyRenderer(stage_vllm_config, tokenizer=None)
|
||||
return omni_renderer_cls(_TokenOnlyRenderer)(stage_vllm_config, tokenizer=None)
|
||||
|
||||
|
||||
def build_stage0_input_processor(stage_vllm_config: Any) -> InputProcessor:
|
||||
"""Build the shared stage-0 input processor."""
|
||||
"""Build the shared stage-0 input processor.
|
||||
|
||||
The renderer is the Omni subclass of the upstream renderer class that
|
||||
``renderer_from_config`` would have picked (or of the token-only renderer
|
||||
when the stage skips tokenizer initialization), so upstream's
|
||||
``InputProcessor`` runs unmodified on top of it.
|
||||
"""
|
||||
|
||||
patch_generation_config_if_needed(stage_vllm_config.model_config)
|
||||
if bool(getattr(stage_vllm_config.model_config, "skip_tokenizer_init", False)):
|
||||
input_processor = InputProcessor(
|
||||
vllm_config=stage_vllm_config,
|
||||
renderer=_build_token_only_renderer(stage_vllm_config),
|
||||
)
|
||||
renderer = _build_token_only_renderer(stage_vllm_config)
|
||||
else:
|
||||
input_processor = InputProcessor(vllm_config=stage_vllm_config)
|
||||
input_processor.renderer = OmniRenderer(input_processor.renderer)
|
||||
return input_processor
|
||||
renderer = build_omni_renderer(stage_vllm_config)
|
||||
return InputProcessor(vllm_config=stage_vllm_config, renderer=renderer)
|
||||
|
||||
|
||||
def device_init_lock_path(device_id: int, lock_dir: str = "/tmp") -> str:
|
||||
|
||||
Reference in new issue
Block a user