[Doc] Add environment variables configuration reference (#6217)

Signed-off-by: Hongsheng Liu <liuhongsheng4@huawei.com>
This commit is contained in:
Hongsheng Liu authored and GitHub committed 2026-08-26 04:58:54 +08:00
1 parent 6639c82a17
commit 780e8d40aa
6 files changed
+873 -6

No files matched your search

+21 -6
View File
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: Apache-2.0 # SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project # SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
# ruff: noqa # ruff: noqa
# code borrowed from https://github.com/pytorch/pytorch/blob/main/torch/utils/collect_env.py # code borrowed from https://github.com/pytorch/pytorch/blob/main/torch/utils/collect_env.py
@@ -494,6 +494,16 @@ def is_xnnpack_available():
def get_env_vars(): def get_env_vars():
try:
from vllm_omni.config.environment_variable_inventory import ENVIRONMENT_VARIABLE_INVENTORY
except Exception:
# Keep environment collection usable when an incomplete/broken Omni
# installation is the reason the user is running this diagnostic.
# Importing an Omni submodule executes package initialization, which
# may fail with RuntimeError, AssertionError, or OSError as well as
# ImportError when vLLM or the accelerator stack is broken.
ENVIRONMENT_VARIABLE_INVENTORY = {}
env_vars = "" env_vars = ""
secret_terms = ("secret", "token", "api", "access", "password") secret_terms = ("secret", "token", "api", "access", "password")
report_prefix = ( report_prefix = (
@@ -507,12 +517,17 @@ def get_env_vars():
"MKL_", "MKL_",
"NVIDIA", "NVIDIA",
) )
for k, v in os.environ.items(): for k, v in sorted(os.environ.items()):
if any(term in k.lower() for term in secret_terms): omni_classification = ENVIRONMENT_VARIABLE_INVENTORY.get(k)
if (omni_classification is not None and omni_classification.redact_value) or any(
term in k.lower() for term in secret_terms
):
continue continue
if k in environment_variables: if (
env_vars = env_vars + "{}={}".format(k, v) + "\n" k in environment_variables
if k.startswith(report_prefix): or (omni_classification is not None and omni_classification.is_public_omni)
or k.startswith(report_prefix)
):
env_vars = env_vars + "{}={}".format(k, v) + "\n" env_vars = env_vars + "{}={}".format(k, v) + "\n"
return env_vars return env_vars
+5
View File
@@ -2,6 +2,11 @@
The CLI for vLLM-Omni inherits from vllm with some additional arguments. The CLI for vLLM-Omni inherits from vllm with some additional arguments.
Environment variables are process configuration rather than CLI-only options.
See the [Environment Variables](../configuration/environment_variables.md)
reference for ownership, precedence, evaluation time, and per-stage `env`
scoping.
## serve ## serve
Starts the vLLM-Omni OpenAI Compatible API server. Starts the vLLM-Omni OpenAI Compatible API server.
+3
View File
@@ -7,6 +7,9 @@ For options within a vLLM Engine, please refer to the [vLLM 0.26 configuration g
Each model defines fixed topology in a registered `PipelineConfig` and runtime Each model defines fixed topology in a registered `PipelineConfig` and runtime
overrides in a deploy YAML. overrides in a deploy YAML.
For process-level settings shared by the CLI, deploy YAML, workers, and model
integrations, see [Environment Variables](./environment_variables.md).
For a specific example, see the [Qwen2.5-Omni deploy config](gh-file:vllm_omni/deploy/qwen2_5_omni.yaml). The matching frozen pipeline topology lives at [vllm_omni/model_executor/models/qwen2_5_omni/pipeline.py](gh-file:vllm_omni/model_executor/models/qwen2_5_omni/pipeline.py). For a specific example, see the [Qwen2.5-Omni deploy config](gh-file:vllm_omni/deploy/qwen2_5_omni.yaml). The matching frozen pipeline topology lives at [vllm_omni/model_executor/models/qwen2_5_omni/pipeline.py](gh-file:vllm_omni/model_executor/models/qwen2_5_omni/pipeline.py).
For an introduction, see [Pipeline and deploy configurations](./stage_configs.md). For an introduction, see [Pipeline and deploy configurations](./stage_configs.md).
+236
View File
@@ -0,0 +1,236 @@
# Environment Variables
Environment variables are process-level configuration shared by the CLI,
deploy YAML, Python APIs, worker processes, and model integrations. They are
therefore documented under **User Guide → Configuration**, rather than under
CLI Reference.
This page describes the public vLLM-Omni environment-variable contract on the
`latest` documentation branch. Use the documentation for your installed
vLLM-Omni release when behavior differs.
!!! important "Public contract"
Only names in [Public vLLM-Omni variables](#public-vllm-omni-variables) are
public Omni configuration. A name found in source code is not automatically
supported: model-specific, benchmark, platform, and internal variables have
separate ownership and lifecycle rules.
## Naming and implementation boundary
New public variables owned by vLLM-Omni must use the `VLLM_OMNI_` prefix.
Existing public names with older prefixes remain supported for compatibility,
but they do not establish naming precedent for new settings. Inherited vLLM
and third-party names retain the prefix chosen by their owning project.
The source [environment-variable inventory](gh-file:vllm_omni/config/environment_variable_inventory.py)
records ownership, migration disposition, and redaction metadata. Unlike
`vllm.envs`, it is not an executable value resolver: existing consumers still
own parsing, defaults, precedence, and evaluation time. The public tables below
describe those live consumer contracts. Stable model or stage behavior should
move to typed configuration, and request-varying behavior should move to a
request schema instead of adding another environment variable.
## Precedence and evaluation time
There is no repository-wide rule that makes environment variables override (or
lose to) every CLI, YAML, or request option. The tables below state precedence
for each public setting. In general:
- request fields should control behavior that can vary per request;
- typed deploy or stage configuration should control stable model and stage
behavior;
- environment variables are appropriate for process launch, platform
integration, emergency diagnostics, and compatibility fallbacks.
An environment variable can be read while a module is imported, when server or
model configuration is constructed, when a worker is spawned, or whenever a
property is accessed. Set import- and startup-time variables before invoking
`vllm serve` or importing vLLM-Omni. Changing the parent shell after workers
start does not update existing worker environments.
The lifecycle labels used below are:
- **Stable** — reviewed public configuration; changes require compatibility
consideration.
- **Experimental** — public for evaluation, but its contract may still evolve.
- **Diagnostic** — supported for investigation or temporary mitigation, not as
a normal tuning interface.
- **Deprecated** — compatibility alias; migrate to the named replacement.
## Public vLLM-Omni variables
### Diffusion backend and cache selection
| Name | Type and default | Applies to and read time | Precedence and invalid values | Lifecycle |
| --- | --- | --- | --- | --- |
| `DIFFUSION_ATTENTION_BACKEND` | Backend name or `auto`; default `auto` (platform selection) | Diffusion stages; read when `OmniDiffusionConfig` is constructed | `diffusion_attention_config.default` wins. An unknown backend fails during backend resolution. | Stable fallback |
| `DIFFUSION_CACHE_BACKEND` | `none`, `cache_dit`, `tea_cache`, `mag_cache`, `step_cache`, `stepcache`, or `step_cache_dit`; default `none` | Diffusion runner startup | Explicit `cache_backend` in config wins. Otherwise this name wins over the deprecated alias. Unsupported values raise `ValueError` during runner setup. | Stable fallback |
| `DIFFUSION_CACHE_ADAPTER` | Same values as `DIFFUSION_CACHE_BACKEND`; default `none` | Diffusion runner startup | Used only when neither explicit `cache_backend` nor `DIFFUSION_CACHE_BACKEND` is set. Unsupported values raise `ValueError`. | Deprecated; use `DIFFUSION_CACHE_BACKEND` |
| `OMNI_DIFFUSION_PROMPT_EMBED_CACHE` | Boolean: `1`, `true`, `yes`, `on`, `0`, `false`, `no`, or `off`; default disabled | Each diffusion runner; resolved during model setup | A recognized environment value overrides the explicit enable setting. An unrecognized value is ignored. | Experimental |
| `OMNI_DIFFUSION_PROMPT_EMBED_CACHE_SIZE` | Positive integer; default `32` entries | Each diffusion runner; resolved during model setup | A valid environment value overrides the explicit cache size. A non-integer logs a warning; a non-positive value is ignored. | Experimental |
| `OMNI_DIFFUSION_SESSION_STATE_MANAGER` | Boolean with the same accepted spellings as the prompt cache; default disabled | Experimental diffusion session manager; model setup | A recognized environment value overrides the explicit enable setting. An unrecognized value is ignored. | Experimental |
| `OMNI_DIFFUSION_SESSION_STATE_MANAGER_MAX_SESSIONS` | Positive integer; default `64` | Experimental diffusion session manager; model setup | A valid environment value overrides the explicit maximum. A non-integer logs a warning; a non-positive value is ignored. | Experimental |
Backend names for `DIFFUSION_ATTENTION_BACKEND` are the members of
`DiffusionAttentionBackendEnum`, such as `FLASH_ATTN`, `TORCH_SDPA`,
`SAGE_ATTN`, `FLASHINFER_ATTN`, and `TRTLLM_ATTN`. Platform support still
depends on the installed kernels and model path.
### Serving and runtime
| Name | Type and default | Applies to and read time | Precedence and invalid values | Lifecycle |
| --- | --- | --- | --- | --- |
| `SPEAKER_SAMPLES_DIR` | Filesystem path; default `~/.cache/vllm-omni/speakers` | Speech server; read when speaker storage initializes | Environment-only setting. The directory is created; filesystem errors propagate. | Stable |
| `SPEAKER_MAX_UPLOADED` | Integer; default `1000` | Speech server; read when speaker storage initializes | Environment-only setting. A non-integer logs a warning and uses `1000`; range is not otherwise validated. | Stable |
| `VLLM_OMNI_INPUT_WAIT_TIMEOUT_S` | Float seconds; default `600`; `<=0` disables | Full-payload input coordinator, not async-chunk transfer; read when the scheduler module imports in each worker | Environment-only setting. A non-float logs a warning and uses `600`. | Stable operational control |
| `VLLM_OMNI_ORCH_MONITOR_PATH` | Filesystem path; default `<current-working-directory>/vllm_omni_orch_monitor_<timestamp>.json` | Orchestrator monitor enabled by `--enable-orch-monitor`; read when the monitor is created | Environment-only path override. Parent directories are created; write errors are logged. | Diagnostic |
| `VLLM_OMNI_VIDEO_SYNC_TIMEOUT` | Float seconds; default `600` | Synchronous Videos API; read when the API server module imports | Environment-only setting. A non-float raises `ValueError` during import. | Experimental |
| `VLLM_VIDEO_ASYNC_CHUNK` | `on` or `off`; default `on` | Streaming video output; read on attribute access | Environment-only setting. Values are trimmed and case-normalized; an invalid value warns once and uses `on`. | Experimental |
| `VLLM_VIDEO_AUDIO_DELTA_MODE` | `fast` or `slow`; default `fast` | Streaming video audio deltas; read on attribute access | Environment-only setting. Values are trimmed and case-normalized; an invalid value warns once and uses `fast`. | Experimental |
### Server storage
Storage names use Pydantic's nested-settings delimiter (`__`). They are read
when `ServerSettings` is initialized, normally during server import/startup.
Invalid integer values raise a Pydantic validation error.
| Name | Type and default | Meaning | Lifecycle |
| --- | --- | --- | --- |
| `VLLM_OMNI_SERVER_STORAGE__PATH` | Path; default `/tmp/storage` | Directory for completed server files. | Stable |
| `VLLM_OMNI_SERVER_STORAGE__FILE_CONCURRENCY` | Integer; default `4` | Maximum concurrent file operations. No additional range validation is currently applied. | Stable |
| `VLLM_OMNI_SERVER_STORAGE__FILE_TTL` | Integer seconds or unset; default unset | Optional lifetime for locally stored files. | Stable |
| `VLLM_OMNI_SERVER_STORAGE__TTL_SWEEP_INTERVAL` | Integer seconds or unset | Sweep frequency. If file TTL is set and this is unset, the effective default is `300`. | Stable |
| `VLLM_OMNI_STORAGE_PATH` | Path | Deprecated alias for `VLLM_OMNI_SERVER_STORAGE__PATH`. The new name wins when both are set. | Deprecated |
| `VLLM_OMNI_STORAGE_MAX_CONCURRENCY` | Integer | Deprecated alias for `VLLM_OMNI_SERVER_STORAGE__FILE_CONCURRENCY`. The new name wins when both are set. | Deprecated |
The deprecated aliases emit `DeprecationWarning`. They do not affect file TTL
settings.
### Quantization diagnostics and performance
| Name | Type and default | Applies to and read time | Precedence and invalid values | Lifecycle |
| --- | --- | --- | --- | --- |
| `VLLM_OMNI_SKIP_NVFP4_NAN_CLAMP` | Boolean truthy spellings: `1`, `true`, `yes`, `on`; default false | ModelOpt NVFP4 compatibility patch; read when `vllm_omni.patch` imports | Environment-only escape hatch. Any other value means false. Set only to diagnose the upstream NaN-scale issue. | Diagnostic and temporary |
| `VLLM_OMNI_USE_QUACK_FP8` | Boolean truthy spellings: `1`, `true`, `yes`, `on`; unset means hardware auto-detection | FP8 scaled matrix multiplication; evaluated when quack capability is selected | A set value overrides auto-detection. Any non-truthy value forces quack off. If quack cannot load, vLLM-Omni warns and falls back to FlashInfer. | Experimental performance control |
`QUACK_CACHE_DIR` is owned by the external quack library and is not an
Omni-owned variable, even though vLLM-Omni supplies a persistent default when
it is unset.
## Per-stage environment
Deploy configurations can set arbitrary environment keys for one stage:
```yaml
stages:
- stage_id: 0
env:
HCCL_IF_BASE_PORT: 23000
MY_MODEL_CACHE: /mnt/model-cache
```
The legacy nested form under `runtime.env` is also accepted. vLLM-Omni converts
keys and values to strings, applies them while the stage engine or child process
is launched, logs keys but not values, and restores the parent process's prior
environment afterward. A child process inherits the scoped values. This keeps
sibling stages from permanently changing one another's parent environment.
This scoping has two consequences:
1. A setting read inside stage/model initialization can see the stage value.
2. A setting already cached by an import that occurred before stage launch
cannot be changed retroactively with stage `env`.
Stage `env` is an escape hatch for process and third-party settings, not a
replacement for typed stage configuration. Do not put request-varying behavior
there. Secret values may reach child processes; logs and diagnostics must emit
their keys only.
## Inherited vLLM variables
vLLM-Omni also reads variables through its aligned vLLM dependency. Refer to
the [vLLM 0.26 environment-variable reference](https://docs.vllm.ai/en/v0.26.0/configuration/env_vars.html)
for their definitions. This includes vLLM launch, cache, logging, plugin, ROCm,
XPU, ModelScope, and FlashInfer workspace settings.
An inherited name does not become Omni-owned because Omni reads or forwards it.
Where behavior differs, the Omni-specific page should document only that
difference. For example, device-visibility values can be further narrowed by a
stage's `devices` setting.
## Platform, external, and secret variables
Hardware runtimes, distributed launchers, and third-party libraries retain
ownership of their variables. Common groups include:
- device visibility (`ASCEND_RT_VISIBLE_DEVICES`, `HIP_VISIBLE_DEVICES`, and
`MUSA_VISIBLE_DEVICES`);
- distributed launch (`MASTER_ADDR`, `MASTER_PORT`, `RANK`, and `WORLD_SIZE`);
- Hugging Face cache and authentication (`HF_HOME`, `HF_TOKEN`, and related
names);
- accelerator libraries such as NCCL, FlashInfer, MindIE, MORI, and quack.
Consult the owning project's documentation for accepted values and support.
`HF_TOKEN`, `HUGGINGFACE_HUB_TOKEN`, and `OPENAI_API_KEY` are explicitly marked
for redaction in the reviewed inventory. Never print their values in environment
collection, examples, bug reports, or logs.
## Model-specific variables
The audit found 57 variables read by a single model or pipeline family. They are
not listed as public usage options here because doing so would turn implementation
escape hatches into an accidental compatibility contract.
Every audited model-specific name has a migration disposition in the
[environment-variable inventory](gh-file:vllm_omni/config/environment_variable_inventory.py):
| Disposition | Count | Required outcome |
| --- | ---: | --- |
| Promote | 33 | Move a stable setting into typed stage or model configuration. |
| Request scope | 6 | Move request-varying behavior into a declared request-option schema. |
| External | 0 | Retain only when a supported third-party library owns the contract. |
| Internalize | 11 | Keep a debug or diagnostic switch out of public documentation and configuration. |
| Deprecate/remove | 7 | Remove a compatibility escape hatch that has no continuing contract. |
The disposition is a migration target, not a statement that the existing
environment switch is stable. Promote or request-scope work should land in
model-owner-reviewed follow-up changes with validation and model-page updates.
The family-sized migration packages and their ownership gate are tracked in
[#6232](https://github.com/vllm-project/vllm-omni/issues/6232).
## Environment collection and troubleshooting
Run the repository collector when preparing a bug report:
```bash
python collect_env.py
```
The environment section includes the safe public Omni allowlist, registered
vLLM variables, and selected platform prefixes. It omits names marked for
redaction and names containing common secret terms. Review the output before
sharing it. A key supplied through stage `env` is not collected merely because
it appears in deploy configuration; the same allowlist and prefix rules apply.
If a setting appears ineffective, first check its read time in the tables
above. Restart processes after changing import- or startup-time values, verify
that the variable reaches the intended worker, and check whether an explicit
typed configuration field takes precedence.
## Benchmark and internal variables
The reviewed inventory marks 20 benchmark variables as transitional while
benchmark configuration moves to CLI/request schemas in
[issue #5376](https://github.com/vllm-project/vllm-omni/issues/5376). They are not
part of the serving/runtime contract.
Internal process-coordination and debug variables are deliberately excluded
from this page. The CI scanner resolves string literals and module-level string
constants used by direct `os` access, local wrapper helpers, aliased `os`
imports, and environment membership checks. Server-storage names generated by
Pydantic settings are checked separately against their model fields.
Names assembled from cross-module metadata and arbitrary keys supplied through
stage `env` cannot be inferred reliably. Contributors must classify those names
explicitly. CI fails when a statically resolved name has no classification or
when a reviewed public name is absent from this page.
+317
View File
@@ -0,0 +1,317 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Drift checks for the reviewed environment-variable inventory."""
import ast
import builtins
from collections import Counter
from pathlib import Path
import pytest
from vllm_omni.config.environment_variable_inventory import (
ENVIRONMENT_VARIABLE_INVENTORY,
EnvironmentVariableCategory,
ModelEnvironmentVariableDisposition,
)
pytestmark = [pytest.mark.core_model, pytest.mark.cpu]
_REPO_ROOT = Path(__file__).parents[2]
_PACKAGE_ROOT = _REPO_ROOT / "vllm_omni"
_INVENTORY_MODULE = _PACKAGE_ROOT / "config" / "environment_variable_inventory.py"
_REFERENCE_PAGE = _REPO_ROOT / "docs" / "configuration" / "environment_variables.md"
def _module_string_constants(tree: ast.Module) -> dict[str, str]:
constants: dict[str, str] = {}
for statement in tree.body:
targets: list[ast.expr] = []
value: ast.expr | None = None
if isinstance(statement, ast.Assign):
targets = statement.targets
value = statement.value
elif isinstance(statement, ast.AnnAssign):
targets = [statement.target]
value = statement.value
if isinstance(value, ast.Constant) and isinstance(value.value, str):
for target in targets:
if isinstance(target, ast.Name):
constants[target.id] = value.value
return constants
def _os_import_aliases(tree: ast.Module) -> tuple[set[str], set[str], set[str]]:
os_names: set[str] = set()
getenv_names: set[str] = set()
environ_names: set[str] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for imported in node.names:
if imported.name == "os":
os_names.add(imported.asname or "os")
elif isinstance(node, ast.ImportFrom) and node.module == "os":
for imported in node.names:
if imported.name == "getenv":
getenv_names.add(imported.asname or imported.name)
elif imported.name == "environ":
environ_names.add(imported.asname or imported.name)
return os_names, getenv_names, environ_names
def _is_environ(node: ast.expr, os_names: set[str], environ_names: set[str]) -> bool:
return (
isinstance(node, ast.Attribute)
and isinstance(node.value, ast.Name)
and node.value.id in os_names
and node.attr == "environ"
) or (isinstance(node, ast.Name) and node.id in environ_names)
def _environment_key_expressions(
node: ast.AST,
os_names: set[str],
getenv_names: set[str],
environ_names: set[str],
) -> list[ast.expr]:
"""Return key expressions used by one direct environment access."""
if isinstance(node, ast.Call) and node.args:
func = node.func
if (
isinstance(func, ast.Attribute)
and isinstance(func.value, ast.Name)
and func.value.id in os_names
and func.attr == "getenv"
) or (isinstance(func, ast.Name) and func.id in getenv_names):
return [node.args[0]]
if (
isinstance(func, ast.Attribute)
and _is_environ(func.value, os_names, environ_names)
and func.attr in {"get", "pop", "setdefault"}
):
return [node.args[0]]
elif isinstance(node, ast.Subscript) and _is_environ(node.value, os_names, environ_names):
return [node.slice]
elif isinstance(node, ast.Compare):
expressions: list[ast.expr] = []
left_values = [node.left, *node.comparators[:-1]]
for left, operator, right in zip(left_values, node.ops, node.comparators):
if isinstance(operator, (ast.In, ast.NotIn)) and _is_environ(right, os_names, environ_names):
expressions.append(left)
return expressions
return []
def _resolve_environment_name(expression: ast.expr, constants: dict[str, str]) -> str | None:
value: str | None = None
if isinstance(expression, ast.Constant) and isinstance(expression.value, str):
value = expression.value
elif isinstance(expression, ast.Name):
value = constants.get(expression.id)
return value or None
def _environment_wrapper_parameters(
tree: ast.Module,
os_names: set[str],
getenv_names: set[str],
environ_names: set[str],
) -> dict[str, set[tuple[str, int | None]]]:
"""Find local helpers that forward one parameter as an environment key."""
wrappers: dict[str, set[tuple[str, int | None]]] = {}
for function in (node for node in tree.body if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef))):
positional = [*function.args.posonlyargs, *function.args.args]
parameter_indexes: dict[str, int | None] = {parameter.arg: index for index, parameter in enumerate(positional)}
parameter_indexes.update({parameter.arg: None for parameter in function.args.kwonlyargs})
forwarded: set[tuple[str, int | None]] = set()
for node in ast.walk(function):
for expression in _environment_key_expressions(node, os_names, getenv_names, environ_names):
if isinstance(expression, ast.Name) and expression.id in parameter_indexes:
forwarded.add((expression.id, parameter_indexes[expression.id]))
if forwarded:
wrappers[function.name] = forwarded
return wrappers
def _environment_accesses(path: Path) -> set[str]:
"""Return statically resolvable names accessed through ``os``."""
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
constants = _module_string_constants(tree)
os_names, getenv_names, environ_names = _os_import_aliases(tree)
wrappers = _environment_wrapper_parameters(tree, os_names, getenv_names, environ_names)
names: set[str] = set()
for node in ast.walk(tree):
for expression in _environment_key_expressions(node, os_names, getenv_names, environ_names):
if name := _resolve_environment_name(expression, constants):
names.add(name)
if isinstance(node, ast.Call) and isinstance(node.func, ast.Name):
for parameter_name, position in wrappers.get(node.func.id, set()):
argument = node.args[position] if position is not None and position < len(node.args) else None
if argument is None:
argument = next(
(keyword.value for keyword in node.keywords if keyword.arg == parameter_name),
None,
)
if argument is not None and (name := _resolve_environment_name(argument, constants)):
names.add(name)
return names
def test_inventory_matches_reviewed_snapshot_counts():
"""Make an inventory expansion an explicit review decision."""
category_counts = Counter(item.category for item in ENVIRONMENT_VARIABLE_INVENTORY.values())
assert category_counts == {
EnvironmentVariableCategory.PUBLIC_OMNI: 22,
EnvironmentVariableCategory.INHERITED_VLLM: 20,
EnvironmentVariableCategory.PLATFORM_EXTERNAL: 27,
EnvironmentVariableCategory.MODEL_SPECIFIC: 57,
EnvironmentVariableCategory.BENCHMARK_TRANSITIONAL: 20,
EnvironmentVariableCategory.INTERNAL: 2,
}
disposition_counts = Counter(
item.model_disposition
for item in ENVIRONMENT_VARIABLE_INVENTORY.values()
if item.category is EnvironmentVariableCategory.MODEL_SPECIFIC
)
assert {disposition: disposition_counts[disposition] for disposition in ModelEnvironmentVariableDisposition} == {
ModelEnvironmentVariableDisposition.PROMOTE: 33,
ModelEnvironmentVariableDisposition.REQUEST_SCOPE: 6,
ModelEnvironmentVariableDisposition.EXTERNAL: 0,
ModelEnvironmentVariableDisposition.INTERNALIZE: 11,
ModelEnvironmentVariableDisposition.DEPRECATE_REMOVE: 7,
}
def test_new_public_omni_names_use_project_prefix():
"""Grandfather legacy names without allowing more prefix exceptions."""
legacy_public_names = {
"DIFFUSION_ATTENTION_BACKEND",
"DIFFUSION_CACHE_ADAPTER",
"DIFFUSION_CACHE_BACKEND",
"OMNI_DIFFUSION_PROMPT_EMBED_CACHE",
"OMNI_DIFFUSION_PROMPT_EMBED_CACHE_SIZE",
"OMNI_DIFFUSION_SESSION_STATE_MANAGER",
"OMNI_DIFFUSION_SESSION_STATE_MANAGER_MAX_SESSIONS",
"SPEAKER_MAX_UPLOADED",
"SPEAKER_SAMPLES_DIR",
"VLLM_VIDEO_ASYNC_CHUNK",
"VLLM_VIDEO_AUDIO_DELTA_MODE",
}
public_names = {
name
for name, item in ENVIRONMENT_VARIABLE_INVENTORY.items()
if item.category is EnvironmentVariableCategory.PUBLIC_OMNI
}
assert {name for name in public_names if not name.startswith("VLLM_OMNI_")} == legacy_public_names
def test_statically_resolvable_environment_accesses_are_classified():
discovered: set[str] = set()
for path in _PACKAGE_ROOT.rglob("*.py"):
discovered.update(_environment_accesses(path))
assert discovered - ENVIRONMENT_VARIABLE_INVENTORY.keys() == set()
def test_model_and_benchmark_inventory_entries_are_still_referenced():
"""Reject transitional rows after their implementation has disappeared."""
package_source = "\n".join(
path.read_text(encoding="utf-8") for path in _PACKAGE_ROOT.rglob("*.py") if path != _INVENTORY_MODULE
)
transitional_categories = {
EnvironmentVariableCategory.MODEL_SPECIFIC,
EnvironmentVariableCategory.BENCHMARK_TRANSITIONAL,
}
stale_names = {
name
for name, item in ENVIRONMENT_VARIABLE_INVENTORY.items()
if item.category in transitional_categories and name not in package_source
}
assert stale_names == set()
def test_environment_scanner_covers_indirection_aliases_membership_and_casing():
expected_by_path = {
"metrics/definitions.py": {
"VLLM_OMNI_BENCH_AUDIO_CHANNELS",
"VLLM_OMNI_BENCH_AUDIO_SAMPLE_RATE",
},
"entrypoints/openai/video_stream_envs.py": {
"VLLM_VIDEO_ASYNC_CHUNK",
"VLLM_VIDEO_AUDIO_DELTA_MODE",
},
"model_executor/models/mimo_audio/mimo_audio.py": {"model_stage"},
"model_executor/models/moss_tts/modeling_moss_tts_local_depth.py": {"MOSS_TTS_DEBUG_STOP"},
"distributed/ray_utils/utils.py": {"RAY_RAYLET_PID"},
}
for relative_path, expected in expected_by_path.items():
assert expected <= _environment_accesses(_PACKAGE_ROOT / relative_path)
def test_generated_server_storage_environment_names_are_classified():
from vllm_omni.config.server_settings import FileBackend
generated_names = {
f"VLLM_OMNI_SERVER_STORAGE__{field_name.upper()}"
for field_name in FileBackend.model_fields
if field_name != "type"
}
assert generated_names <= ENVIRONMENT_VARIABLE_INVENTORY.keys()
def test_public_omni_variables_are_in_the_reference_page():
reference = _REFERENCE_PAGE.read_text(encoding="utf-8")
missing = {
name
for name, item in ENVIRONMENT_VARIABLE_INVENTORY.items()
if item.is_public_omni and f"`{name}`" not in reference
}
assert missing == set()
def test_secret_values_are_marked_for_redaction():
assert {name for name, item in ENVIRONMENT_VARIABLE_INVENTORY.items() if item.redact_value} == {
"HF_TOKEN",
"HUGGINGFACE_HUB_TOKEN",
"OPENAI_API_KEY",
}
def test_collect_env_reports_only_safe_public_omni_values(monkeypatch):
from collect_env import get_env_vars
monkeypatch.setenv("DIFFUSION_CACHE_BACKEND", "tea_cache")
monkeypatch.setenv("HF_TOKEN", "hf-secret")
monkeypatch.setenv("OPENAI_API_KEY", "openai-secret")
monkeypatch.setenv("VLLM_OMNI_REPLICA_ID", "internal-replica")
report = get_env_vars()
assert "DIFFUSION_CACHE_BACKEND=tea_cache" in report.splitlines()
assert "hf-secret" not in report
assert "openai-secret" not in report
assert "VLLM_OMNI_REPLICA_ID" not in report
@pytest.mark.parametrize("error_type", [ImportError, OSError, AssertionError, RuntimeError])
def test_collect_env_survives_inventory_import_error(monkeypatch, error_type):
from collect_env import get_env_vars
real_import = builtins.__import__
def fail_inventory_import(name, *args, **kwargs):
if name == "vllm_omni.config.environment_variable_inventory":
raise error_type("broken vllm-omni installation")
return real_import(name, *args, **kwargs)
monkeypatch.setattr(builtins, "__import__", fail_inventory_import)
monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "0")
assert "CUDA_VISIBLE_DEVICES=0" in get_env_vars().splitlines()
@@ -0,0 +1,291 @@
# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
"""Reviewed inventory of environment variables used by vLLM-Omni.
This module classifies environment-variable names; it does not read them or
make every classified name part of the public configuration contract. In
particular, model-specific variables remain transitional until their recorded
disposition is implemented.
"""
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass
from enum import Enum
from types import MappingProxyType
class EnvironmentVariableCategory(str, Enum):
"""Ownership and documentation boundary for an environment variable."""
PUBLIC_OMNI = "public_omni"
INHERITED_VLLM = "inherited_vllm"
PLATFORM_EXTERNAL = "platform_external"
MODEL_SPECIFIC = "model_specific"
BENCHMARK_TRANSITIONAL = "benchmark_transitional"
INTERNAL = "internal"
class ModelEnvironmentVariableDisposition(str, Enum):
"""Required destination for a model-specific environment variable."""
PROMOTE = "promote"
REQUEST_SCOPE = "request_scope"
EXTERNAL = "external"
INTERNALIZE = "internalize"
DEPRECATE_REMOVE = "deprecate_remove"
@dataclass(frozen=True)
class EnvironmentVariableClassification:
"""Classification metadata for one statically identifiable name."""
name: str
category: EnvironmentVariableCategory
model_disposition: ModelEnvironmentVariableDisposition | None = None
redact_value: bool = False
@property
def is_public_omni(self) -> bool:
return self.category is EnvironmentVariableCategory.PUBLIC_OMNI
# New entries must use the VLLM_OMNI_ prefix. The regression test explicitly
# grandfathers the older public names below.
_PUBLIC_OMNI = (
"DIFFUSION_ATTENTION_BACKEND",
"DIFFUSION_CACHE_ADAPTER",
"DIFFUSION_CACHE_BACKEND",
"OMNI_DIFFUSION_PROMPT_EMBED_CACHE",
"OMNI_DIFFUSION_PROMPT_EMBED_CACHE_SIZE",
"OMNI_DIFFUSION_SESSION_STATE_MANAGER",
"OMNI_DIFFUSION_SESSION_STATE_MANAGER_MAX_SESSIONS",
"SPEAKER_MAX_UPLOADED",
"SPEAKER_SAMPLES_DIR",
"VLLM_OMNI_INPUT_WAIT_TIMEOUT_S",
"VLLM_OMNI_ORCH_MONITOR_PATH",
"VLLM_OMNI_SERVER_STORAGE__FILE_CONCURRENCY",
"VLLM_OMNI_SERVER_STORAGE__FILE_TTL",
"VLLM_OMNI_SERVER_STORAGE__PATH",
"VLLM_OMNI_SERVER_STORAGE__TTL_SWEEP_INTERVAL",
"VLLM_OMNI_SKIP_NVFP4_NAN_CLAMP",
"VLLM_OMNI_STORAGE_MAX_CONCURRENCY",
"VLLM_OMNI_STORAGE_PATH",
"VLLM_OMNI_USE_QUACK_FP8",
"VLLM_OMNI_VIDEO_SYNC_TIMEOUT",
"VLLM_VIDEO_ASYNC_CHUNK",
"VLLM_VIDEO_AUDIO_DELTA_MODE",
)
_INHERITED_VLLM = (
"CUDA_HOME",
"CUDA_VISIBLE_DEVICES",
"LOCAL_RANK",
"NO_COLOR",
"VLLM_ALLOW_LONG_MAX_MODEL_LEN",
"VLLM_BATCH_INVARIANT",
"VLLM_CACHE_ROOT",
"VLLM_DISABLE_LOG_LOGO",
"VLLM_FLASHINFER_WORKSPACE_BUFFER_SIZE",
"VLLM_HTTP_TIMEOUT_KEEP_ALIVE",
"VLLM_LOGGING_COLOR",
"VLLM_MOONCAKE_BOOTSTRAP_PORT",
"VLLM_PLUGINS",
"VLLM_ROCM_USE_AITER",
"VLLM_ROCM_USE_AITER_RMSNORM",
"VLLM_TUNED_CONFIG_FOLDER",
"VLLM_USE_MODELSCOPE",
"VLLM_USE_OINK_OPS",
"VLLM_WORKER_MULTIPROC_METHOD",
"VLLM_XPU_USE_SAMPLER_KERNEL",
)
_PLATFORM_EXTERNAL = (
"ASCEND_LAUNCH_BLOCKING",
"ASCEND_RT_VISIBLE_DEVICES",
"FLASHINFER_DISABLE_VERSION_CHECK",
"HF_HOME",
"HF_HUB_DISABLE_XET",
"HF_TOKEN",
"HIP_VISIBLE_DEVICES",
"HUGGINGFACE_HUB_TOKEN",
"MASTER_ADDR",
"MASTER_PORT",
"MINDIE_SD_FA_TYPE",
"MORI_RDMA_DEVICES",
"MUSA_VISIBLE_DEVICES",
"MV2_COMM_WORLD_LOCAL_RANK",
"NCCL_ASYNC_ERROR_HANDLING",
"OMPI_COMM_WORLD_LOCAL_RANK",
"OPENAI_API_KEY",
"PYTHONPATH",
"QUACK_CACHE_DIR",
"RANK",
"RANK_ID",
"RAY_RAYLET_PID",
"RDMA_DEVICE_NAME",
"SAGE_ATTN_TENSOR_LAYOUT",
"VLLM_LOCAL_RANK",
"WORLD_SIZE",
"XDG_CACHE_HOME",
)
_MODEL_PROMOTE = (
"AR_DIFFUSION_KV_SCRATCH_BLOCKS_PER_BRANCH",
"COSMOS3_SOUND_TOKENIZER_CONFIG_PATH",
"COSMOS3_SOUND_TOKENIZER_PATH",
"COSYVOICE3_ESTIMATOR_ONNX",
"COSYVOICE3_TRT",
"COSYVOICE3_TRT_CACHE",
"DYNIN_CONFIG_PATH",
"DYNIN_MAGVIT_REMOTE_CODE_LOCAL_FILES_ONLY",
"DYNIN_MAGVIT_REMOTE_CODE_REPO_ID",
"DYNIN_MAGVIT_REMOTE_CODE_REVISION",
"DYNIN_REMOTE_CODE_LOCAL_FILES_ONLY",
"DYNIN_REMOTE_CODE_REPO_ID",
"DYNIN_REMOTE_CODE_REVISION",
"HIGGS_AUDIO_TOKENIZER_PATH",
"INTERNVLA_A1_COSMOS_DECODER_PATH",
"INTERNVLA_A1_COSMOS_DIR",
"INTERNVLA_A1_COSMOS_ENCODER_PATH",
"INTERNVLA_A1_PROCESSOR_DIR",
"MIMO_AUDIO_TOKENIZER_CUDA_GRAPH",
"MIMO_AUDIO_TOKENIZER_DEVICE",
"MIMO_AUDIO_TOKENIZER_PATH",
"MING_CFM_CUDAGRAPH",
"MINICPMO_TOKEN2WAV_TRT",
"MINICPMO_TOKEN2WAV_TRT_CACHE",
"MINICPMO_TOKEN2WAV_TRT_DTYPE",
"NEMOTRON_VOICECHAT_LLM_PATH",
"OMNIVOICE_CUDA_GRAPH",
"OMNIVOICE_TF32",
"VLLM_GEPARD_CHUNK_FRAMES",
"VLLM_GEPARD_FIRST_CHUNK_FRAMES",
"VLLM_GEPARD_LOOKBACK_FRAMES",
"VLLM_OMNI_FISH_KVCACHE_ATTN",
"VLLM_OMNI_SANA_WM_STAGE1_TEXT_ENCODER",
)
_MODEL_REQUEST_SCOPE = (
"DOTS_TTS_DIT_NUM_STEPS",
"GR00T_NOISE_SEED",
"STEP_AUDIO2_DEFAULT_PROMPT_WAV",
"VLLM_GEPARD_END_FADE_MS",
"VLLM_GEPARD_END_SILENCE_MS",
"VLLM_GEPARD_GREEDY",
)
# No audited model variable is currently owned by a supported third-party
# contract. Keep the disposition explicit so future reviews cannot silently
# fold an externally owned name into "promote".
_MODEL_EXTERNAL: tuple[str, ...] = ()
_MODEL_INTERNALIZE = (
"DOTS_TTS_BETA_TRACE",
"DYNIN_S2U_VENDOR_ROOT",
"DZ_PHASE_TIMING",
"MING_TTS_STAGE1_FINAL_LOG",
"MOSS_TTS_DEBUG_STOP",
"NEMOTRON_VOICECHAT_DEBUG_FUNCTION_TIMELINE",
"VLLM_OMNI_QWEN3_CODE2WAV_BATCH_STATS",
"VLLM_OMNI_QWEN3_CODE2WAV_BATCH_STATS_LOG_EVERY",
"VLLM_OMNI_QWEN3_CODE2WAV_CUDAGRAPH_STATS",
"VLLM_OMNI_QWEN3_CODE2WAV_CUDAGRAPH_STATS_FILE",
"VLLM_OMNI_QWEN3_CODE2WAV_CUDAGRAPH_STATS_LOG_EVERY",
)
_MODEL_DEPRECATE_REMOVE = (
"HIGGS_AUDIO_V2_TOKENIZER_PATH",
"ROSVOT_SOURCE_DIR",
"VLLM_OMNI_LINGBOT_ACTION_ROOT",
"VLLM_OMNI_VOXCPM_CODE_PATH",
"VLLM_USE_FLASHINFER_MOE_FP16",
"XCODEC1_PATH",
"model_stage",
)
_BENCHMARK_TRANSITIONAL = (
"DAILY_OMNI_EXTRACT_MODE",
"DAILY_OMNI_SAVE_EVAL_ITEMS",
"MAX_NUM_FRAMES",
"SEED_TTS_EVAL_DEVICE",
"SEED_TTS_HF_WHISPER_MODEL",
"SEED_TTS_SIM_DEVICE",
"SEED_TTS_SIM_EVAL",
"SEED_TTS_UTMOS_EVAL",
"SEED_TTS_UTMOS_HF_REPO",
"SEED_TTS_UTMOS_JIT_FILE",
"SEED_TTS_WAVLM_MAX_SECONDS",
"SEED_TTS_WAVLM_MIN_SAMPLES",
"SEED_TTS_WAVLM_MODEL",
"SEED_TTS_WER_EVAL",
"SEED_TTS_WER_SAVE_AUDIO_DIR",
"SEED_TTS_WER_SAVE_ITEMS",
"VLLM_DAILY_OMNI_MEDIA_REPO",
"VLLM_OMNI_BENCH_AUDIO_CHANNELS",
"VLLM_OMNI_BENCH_AUDIO_CONTINUITY_THRESHOLD_S",
"VLLM_OMNI_BENCH_AUDIO_SAMPLE_RATE",
)
_INTERNAL = (
"VLLM_OMNI_DLO_DP_WAVE_TIMEOUT",
"VLLM_OMNI_REPLICA_ID",
)
_REDACTED = frozenset(
{
"HF_TOKEN",
"HUGGINGFACE_HUB_TOKEN",
"OPENAI_API_KEY",
}
)
def _build_inventory() -> Mapping[str, EnvironmentVariableClassification]:
inventory: dict[str, EnvironmentVariableClassification] = {}
def add(
names: tuple[str, ...],
category: EnvironmentVariableCategory,
disposition: ModelEnvironmentVariableDisposition | None = None,
) -> None:
for name in names:
if name in inventory:
raise ValueError(f"Environment variable {name!r} is classified more than once")
inventory[name] = EnvironmentVariableClassification(
name=name,
category=category,
model_disposition=disposition,
redact_value=name in _REDACTED,
)
add(_PUBLIC_OMNI, EnvironmentVariableCategory.PUBLIC_OMNI)
add(_INHERITED_VLLM, EnvironmentVariableCategory.INHERITED_VLLM)
add(_PLATFORM_EXTERNAL, EnvironmentVariableCategory.PLATFORM_EXTERNAL)
add(_MODEL_PROMOTE, EnvironmentVariableCategory.MODEL_SPECIFIC, ModelEnvironmentVariableDisposition.PROMOTE)
add(
_MODEL_REQUEST_SCOPE,
EnvironmentVariableCategory.MODEL_SPECIFIC,
ModelEnvironmentVariableDisposition.REQUEST_SCOPE,
)
add(_MODEL_EXTERNAL, EnvironmentVariableCategory.MODEL_SPECIFIC, ModelEnvironmentVariableDisposition.EXTERNAL)
add(
_MODEL_INTERNALIZE,
EnvironmentVariableCategory.MODEL_SPECIFIC,
ModelEnvironmentVariableDisposition.INTERNALIZE,
)
add(
_MODEL_DEPRECATE_REMOVE,
EnvironmentVariableCategory.MODEL_SPECIFIC,
ModelEnvironmentVariableDisposition.DEPRECATE_REMOVE,
)
add(_BENCHMARK_TRANSITIONAL, EnvironmentVariableCategory.BENCHMARK_TRANSITIONAL)
add(_INTERNAL, EnvironmentVariableCategory.INTERNAL)
return MappingProxyType(dict(sorted(inventory.items())))
# All reviewed environment-variable names, indexed by exact name. This is
# classification metadata, not the executable value registry used by
# ``vllm.envs``.
ENVIRONMENT_VARIABLE_INVENTORY: Mapping[str, EnvironmentVariableClassification] = _build_inventory()