0 Fork 0

Files

T

tzhouamandClaude Fable 5.1 33de5d8629 Merge vLLM-Omni v0.29.0rc1 into edge-infer/omni-0.29.0

The pair for vLLM v0.29.0: rc1's docs pin vllm==0.29.0 and a static scan of
rc1's vllm.* imports against the v0.29.0 tree finds nothing missing, while
our 0.28-based tree needs seven modules v0.29.0 no longer has
(vllm.inputs.preprocess, entrypoints.launcher, openai.cli_args,
openai.engine.protocol, serve.utils.server_utils, layers.batch_invariant,
models.hunyuan_v1).

Ten files conflicted. Upstream's structure is the base everywhere; the
fork's features are re-attached to it:
- deploy_profile / apply_deploy_profile run before rc1's resolve_omni_config
  pops deploy_config (async_omni_engine, cli/serve); the deploy-profile
  selector and the resolve_model_config_path chain it needs stay in
  entrypoints/utils.py; stage_overrides takes rc1's dict type.
- config_factory keeps the lazy diffusion output-type import; the diffusion
  registry import in rc1's new config/resolver.py is made lazy for the same
  WP6 reason (TTS path must not load diffusion).
- requirements: our core/media split kept; rc1's transformers>=5.13 floor
  and kernels==0.16.1 applied in core.txt / media.txt.
- TTS: rc1's split_granularity and bounded seed plus our output_mode=
  codec_tokens, _SessionInbox cancel handling and codec DELTA accumulation;
  our private _conditioning_cache_salt is superseded by rc1's public
  tts_adapters/base.py::conditioning_cache_salt.
- stage_runtime: rc1's single ExitStack G2 path (lock released before READY)
  replaces our two-path block, with the four worker_phase names the init
  profiler consumes still emitted; _cpu_placed / host admission kept.

Not yet run against vLLM 0.29.0: the tests listed in the W6 record.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-09-18 09:57:20 +08:00

composable_parallel.md

[Docs] Remove stale stage config references (#6270 )

2026-08-20 07:09:12 +08:00

environment_variables.md

Merge vLLM-Omni v0.29.0rc1 into edge-infer/omni-0.29.0

2026-09-18 09:57:20 +08:00

gpu_memory_utilization.md

[Docs] Remove stale stage config references (#6270 )

2026-08-20 07:09:12 +08:00

README.md

[Rebase] Rebase to vLLM 0.29.0 (#7230 )

2026-09-10 12:08:28 +08:00

stage_configs.md

[Model] Qwen3-Omni: register thinker-only pipeline for Instruct serve (#6284 )

2026-08-27 08:21:03 +00:00

README.md

Configuration Options

This section lists the most common options for running vLLM-Omni.

For options within a vLLM Engine, please refer to the vLLM 0.29 configuration guide.

Each model defines fixed topology in a registered PipelineConfig and runtime overrides in a deploy YAML.

For process-level settings shared by the CLI, deploy YAML, workers, and model integrations, see Environment Variables.

For a specific example, see the Qwen2.5-Omni deploy config. The matching frozen pipeline topology lives at vllm_omni/model_executor/models/qwen2_5_omni/pipeline.py.

For an introduction, see Pipeline and deploy configurations.

Memory Configuration

GPU Memory Calculation and Configuration - Guide on how to calculate memory requirements and set up gpu_memory_utilization for optimal performance

Optimization Features

Diffusion Features Overview - Complete overview of all diffusion model features and supported models