[Misc] fix yaml config (#320)
Signed-off-by: Lancer <maruixiang6688@gmail.com>
This commit is contained in:
1 parent
06c2b7f6e4
commit
31fedcfa2b
5 files changed
+13
-13
No files matched your search
@@ -41,7 +41,7 @@ stage_args:
|
||||
model_stage: thinker
|
||||
model_arch: Qwen2_5OmniForConditionalGeneration # The model implementation registered in model_executor/models/registry.py
|
||||
worker_cls: vllm_omni.worker.gpu_ar_worker.GPUARWorker # The specific worker used
|
||||
scheduler_cls: vllm_omni.core.sched.scheduler.OmniScheduler # The specific scehduler used
|
||||
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler # The specific scehduler used
|
||||
gpu_memory_utilization: 0.8 # The gpu memory allocation for the stage within a single chip
|
||||
enforce_eager: true # Now we only support eager mode
|
||||
trust_remote_code: true # Needed by huggingface config parsing
|
||||
@@ -67,7 +67,7 @@ stage_args:
|
||||
model_stage: talker
|
||||
model_arch: Qwen2_5OmniForConditionalGeneration
|
||||
worker_cls: vllm_omni.worker.gpu_ar_worker.GPUARWorker
|
||||
scheduler_cls: vllm_omni.core.sched.scheduler.OmniScheduler
|
||||
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
|
||||
gpu_memory_utilization: 0.8
|
||||
enforce_eager: true
|
||||
trust_remote_code: true
|
||||
@@ -92,8 +92,8 @@ stage_args:
|
||||
engine_args:
|
||||
model_stage: code2wav
|
||||
model_arch: Qwen2_5OmniForConditionalGeneration
|
||||
worker_cls: vllm_omni.worker.gpu_diffusion_worker.GPUDiffusionWorker
|
||||
scheduler_cls: vllm_omni.core.sched.diffusion_scheduler.DiffusionScheduler
|
||||
worker_cls: vllm_omni.worker.gpu_generation_worker.GPUGenerationWorker
|
||||
scheduler_cls: vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler
|
||||
gpu_memory_utilization: 0.15
|
||||
enforce_eager: true
|
||||
trust_remote_code: true
|
||||
@@ -171,11 +171,11 @@ The model architecture class name that is registered in `model_executor/models/r
|
||||
|
||||
#### `engine_args.worker_cls`
|
||||
|
||||
The specific worker class to use for this stage. This determines how the model computations are executed. Examples include `vllm_omni.worker.gpu_ar_worker.GPUARWorker` for autoregressive stages and `vllm_omni.worker.gpu_diffusion_worker.GPUDiffusionWorker` for diffusion-based stages.
|
||||
The specific worker class to use for this stage. This determines how the model computations are executed. Examples include `vllm_omni.worker.gpu_ar_worker.GPUARWorker` for autoregressive stages and `vllm_omni.worker.gpu_generation_worker.GPUGenerationWorker` for diffusion-based stages.
|
||||
|
||||
#### `engine_args.scheduler_cls`
|
||||
|
||||
The scheduler class to use for this stage. The scheduler manages request queuing, batching, and execution order. Examples include `vllm_omni.core.sched.scheduler.OmniScheduler` for standard stages and `vllm_omni.core.sched.diffusion_scheduler.DiffusionScheduler` for diffusion stages.
|
||||
The scheduler class to use for this stage. The scheduler manages request queuing, batching, and execution order. Examples include `vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler` for standard stages and `vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler` for diffusion stages.
|
||||
|
||||
#### `engine_args.gpu_memory_utilization`
|
||||
|
||||
|
||||
@@ -60,8 +60,8 @@ stage_args:
|
||||
engine_args:
|
||||
model_stage: code2wav
|
||||
model_arch: Qwen2_5OmniForConditionalGeneration
|
||||
worker_cls: vllm_omni.worker.gpu_diffusion_worker.GPUDiffusionWorker
|
||||
scheduler_cls: vllm_omni.core.sched.diffusion_scheduler.DiffusionScheduler
|
||||
worker_cls: vllm_omni.worker.gpu_generation_worker.GPUGenerationWorker
|
||||
scheduler_cls: vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler
|
||||
gpu_memory_utilization: 0.15
|
||||
enforce_eager: true
|
||||
trust_remote_code: true
|
||||
|
||||
@@ -121,7 +121,7 @@ Only specific types of PRs will be reviewed. The PR title is prefixed appropriat
|
||||
- `[Model]` for adding a new model or improving an existing model. Model name should appear in the title.
|
||||
- `[Frontend]` For changes on the vLLM-Omni frontend (e.g., OpenAI API server, `OmniLLM` class, etc.)
|
||||
- `[Kernel]` for changes affecting CUDA kernels or other compute kernels.
|
||||
- `[Core]` for changes in the core vLLM-Omni logic (e.g., `OmniProcessor`, `OmniScheduler`, etc.)
|
||||
- `[Core]` for changes in the core vLLM-Omni logic (e.g., `OmniProcessor`, `OmniARScheduler`, etc.)
|
||||
- `[Hardware][Vendor]` for hardware-specific changes. Vendor name should appear in the prefix, such as [Ascend] for Ascend NPUs.
|
||||
- `[Misc]` for PRs that do not fit the above categories. Please use this sparingly.
|
||||
|
||||
|
||||
@@ -14,7 +14,7 @@ from vllm.v1.spec_decode.metrics import SpecDecodingStats
|
||||
|
||||
class OmniARScheduler(VLLMScheduler):
|
||||
"""
|
||||
OmniScheduler: Scheduler for vLLM-Omni multimodal processing.
|
||||
OmniARScheduler: Scheduler for vLLM-Omni multimodal processing.
|
||||
|
||||
This scheduler extends vLLM's scheduler to support multimodal and
|
||||
non-autoregressive processing with additional fields and methods
|
||||
|
||||
@@ -13,7 +13,7 @@ stage_args:
|
||||
model_stage: thinker
|
||||
model_arch: Qwen3OmniMoeForConditionalGeneration
|
||||
worker_cls: vllm_omni.worker.gpu_ar_worker.GPUARWorker
|
||||
scheduler_cls: vllm_omni.core.sched.scheduler.OmniScheduler
|
||||
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
|
||||
gpu_memory_utilization: 0.6
|
||||
enforce_eager: true
|
||||
trust_remote_code: true
|
||||
@@ -45,7 +45,7 @@ stage_args:
|
||||
model_stage: talker
|
||||
model_arch: Qwen3OmniMoeForConditionalGeneration
|
||||
worker_cls: vllm_omni.worker.gpu_ar_worker.GPUARWorker
|
||||
scheduler_cls: vllm_omni.core.sched.scheduler.OmniScheduler
|
||||
scheduler_cls: vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler
|
||||
gpu_memory_utilization: 0.3
|
||||
enforce_eager: true
|
||||
trust_remote_code: true
|
||||
@@ -80,7 +80,7 @@ stage_args:
|
||||
model_stage: code2wav
|
||||
model_arch: Qwen3OmniMoeForConditionalGeneration
|
||||
worker_cls: vllm_omni.worker.gpu_generation_worker.GPUGenerationWorker
|
||||
scheduler_cls: vllm_omni.core.sched.generation_scheduler.GenerationScheduler
|
||||
scheduler_cls: vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler
|
||||
enforce_eager: true
|
||||
trust_remote_code: true
|
||||
enable_prefix_caching: false
|
||||
|
||||
Reference in new issue
Block a user