anodyine/vllm-omni
1
0 Fork 0
Code Issues 4 Pull Requests 2 Actions Packages Projects Releases Wiki Activity
Files
e4c47a2569f96c59977a86070dcecaa7b37ca521
vllm-omni/docs/api/README.md
T

184 lines
11 KiB
Markdown
Raw Normal View History

[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
# Summary
[Misc] Fix docs and update image edit example with cache-dit (#318)
2025-12-15 21:43:28 +08:00
## Entry Points
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
[Misc] Fix docs and update image edit example with cache-dit (#318)
2025-12-15 21:43:28 +08:00
Main entry points for vLLM-Omni inference and serving.
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
[Feat] Rename AsyncOmniLLM -> AsyncOmni (#103)
2025-11-30 22:26:33 +08:00
- [vllm_omni.entrypoints.async_omni.AsyncOmni][]
[Core] Refactor CFG companion tracker and use in Orchestrator (#2623)
2026-04-17 16:13:00 +08:00
- [vllm_omni.engine.cfg_companion_tracker.CfgCompanionTracker][]
[Doc]Fix - Align with repo. (#1176)
2026-02-03 18:11:50 +08:00
- [vllm_omni.entrypoints.cli.benchmark.base.OmniBenchmarkSubcommandBase][]
- [vllm_omni.entrypoints.cli.benchmark.main.OmniBenchmarkSubcommand][]
- [vllm_omni.entrypoints.cli.benchmark.serve.OmniBenchmarkServingSubcommand][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
- [vllm_omni.entrypoints.cli.serve.OmniServeCommand][]
[Feature] Support output modalities control per request (#298)
2025-12-17 17:19:56 +08:00
- [vllm_omni.entrypoints.client_request_state.ClientRequestState][]
[diffusion] add doc and fix qwen-image (#96)
2025-11-28 18:07:44 +08:00
- [vllm_omni.entrypoints.omni.Omni][]
[Entrypoint][Refactor] vLLM-Omni Entrypoint Refactoring (#1908)
2026-03-18 17:33:53 +08:00
- [vllm_omni.entrypoints.omni_base.OmniBase][]
[Doc] Sync and fix. (#2110)
2026-03-24 11:20:50 +08:00
- [vllm_omni.entrypoints.pd_utils.PDDisaggregationMixin][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
[Core] Graduate MiniCPM-o 4.5 and PersonaPlex full-duplex serving out of experimental (#6196)
2026-09-05 16:20:21 +08:00
## Clients
Client libraries that talk to a running vLLM-Omni server (the full-duplex Realtime client and its per-model session presets).
- [vllm_omni.clients.duplex.AudioDelta][]
- [vllm_omni.clients.duplex.AudioFormat][]
- [vllm_omni.clients.duplex.ConnectionResumed][]
- [vllm_omni.clients.duplex.DuplexClient][]
- [vllm_omni.clients.duplex.DuplexClientError][]
- [vllm_omni.clients.duplex.DuplexConnectionError][]
- [vllm_omni.clients.duplex.DuplexEvent][]
- [vllm_omni.clients.duplex.DuplexProtocolError][]
- [vllm_omni.clients.duplex.DuplexSessionClosedError][]
- [vllm_omni.clients.duplex.ErrorEvent][]
- [vllm_omni.clients.duplex.EventCollector][]
- [vllm_omni.clients.duplex.ListenDecision][]
- [vllm_omni.clients.duplex.ReconnectPolicy][]
- [vllm_omni.clients.duplex.ResponseCreated][]
- [vllm_omni.clients.duplex.ResponseDone][]
- [vllm_omni.clients.duplex.ResponseHandle][]
- [vllm_omni.clients.duplex.SessionClosed][]
- [vllm_omni.clients.duplex.SessionConfig][]
- [vllm_omni.clients.duplex.SessionCreated][]
- [vllm_omni.clients.duplex.SessionExpired][]
- [vllm_omni.clients.duplex.SessionResumed][]
- [vllm_omni.clients.duplex.SpeakDecision][]
- [vllm_omni.clients.duplex.TextDelta][]
- [vllm_omni.clients.duplex.TranscriptDelta][]
- [vllm_omni.clients.duplex.WebSocketTransport][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
## Inputs
Input data structures for multi-modal inputs.
[Doc] Sync and fix. (#2110)
2026-03-24 11:20:50 +08:00
- [vllm_omni.inputs.data.OmniCustomPrompt][]
[Doc]Fix - Align with repo. (#1176)
2026-02-03 18:11:50 +08:00
- [vllm_omni.inputs.data.OmniDiffusionSamplingParams][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
- [vllm_omni.inputs.data.OmniEmbedsPrompt][]
[Doc]Fix - Align with repo. (#1176)
2026-02-03 18:11:50 +08:00
- [vllm_omni.inputs.data.OmniTextPrompt][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
- [vllm_omni.inputs.data.OmniTokenInputs][]
- [vllm_omni.inputs.data.OmniTokensPrompt][]
- [vllm_omni.inputs.preprocess.OmniInputPreprocessor][]
## Engine
Engine classes for offline and online inference.
[diffusion] add doc and fix qwen-image (#96)
2025-11-28 18:07:44 +08:00
- [vllm_omni.diffusion.diffusion_engine.DiffusionEngine][]
[skip ci][Doc] add design docs for async chunk in qwen3-omni (#962)
2026-02-27 10:46:13 +08:00
- [vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.BufferAllocator][]
- [vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.ManagedBuffer][]
- [vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.MooncakeAgentMetadata][]
- [vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.MooncakeTransferEngineConnector][]
- [vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.QueryRequest][]
- [vllm_omni.distributed.omni_connectors.connectors.mooncake_transfer_engine_connector.QueryResponse][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
- [vllm_omni.engine.AdditionalInformationEntry][]
- [vllm_omni.engine.AdditionalInformationPayload][]
[Bugfix] Remove mandatory flash-attn dependency and optimzie docs (#113)
2025-11-30 18:20:44 +08:00
- [vllm_omni.engine.OmniEngineCoreOutput][]
- [vllm_omni.engine.OmniEngineCoreOutputs][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
- [vllm_omni.engine.OmniEngineCoreRequest][]
- [vllm_omni.engine.PromptEmbedsPayload][]
- [vllm_omni.engine.arg_utils.OmniEngineArgs][]
[Doc] Sync and fix. (#2110)
2026-03-24 11:20:50 +08:00
- [vllm_omni.engine.async_omni_engine.AsyncOmniEngine][]
- [vllm_omni.engine.orchestrator.Orchestrator][]
- [vllm_omni.engine.orchestrator.OrchestratorRequestState][]
- [vllm_omni.engine.stage_engine_core_client.StageEngineCoreClient][]
- [vllm_omni.engine.stage_init_utils.StageMetadata][]
- [vllm_omni.engine.stage_init_utils.StartedLlmStage][]
[Refactor][OutputProcessor 1/8]: clean up multimodal payload accumulation and move payload logic into MultimodalPayload & directory refactor (#4980)
2026-07-15 15:50:19 +08:00
- [vllm_omni.outputs.mm_outputs.MultimodalCompletionOutput][]
- [vllm_omni.outputs.mm_outputs.MultimodalPayload][]
- [vllm_omni.outputs.output_modality.OutputModality][]
- [vllm_omni.outputs.output_modality.TensorAccumulationStrategy][]
- [vllm_omni.outputs.output_processor.MultimodalOutputProcessor][]
- [vllm_omni.outputs.output_processor.OmniRequestState][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
## Core
Core scheduling and caching components.
[Bugfix] Fix ci bug for qwen2.5-omni (#230)
2025-12-08 02:43:48 +08:00
- [vllm_omni.core.sched.omni_ar_scheduler.OmniARScheduler][]
- [vllm_omni.core.sched.omni_generation_scheduler.OmniGenerationScheduler][]
[Feature]Support async computation and communication across stages by chunks (#727)
2026-01-26 11:21:24 +08:00
- [vllm_omni.core.sched.output.OmniCachedRequestData][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
- [vllm_omni.core.sched.output.OmniNewRequestData][]
[Doc]Fix - Align with repo. (#1176)
2026-02-03 18:11:50 +08:00
- [vllm_omni.core.sched.output.OmniSchedulerOutput][]
[Doc] Sync and fix. (#2110)
2026-03-24 11:20:50 +08:00
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.cfm.BASECFM][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.cfm.CausalConditionalCFM][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.cfm.CausalMaskedDiffWithDiT][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.cfm.ConditionalCFM][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.CausalConv1d][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.CausalConv1dUpsample][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.CausalConvRNNF0Predictor][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.CausalHiFTGenerator][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.HiFTGenerator][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.SineGen][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.SineGen2][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.Snake][]
- [vllm_omni.model_executor.models.cosyvoice3.code2wav_core.hifigan.SourceModuleHnNSF][]
[Model] Support Qwen3-TTS model series (#895)
2026-01-22 16:57:37 +08:00
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.DistributedGroupResidualVectorQuantization][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.DistributedResidualVectorQuantization][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.EuclideanCodebook][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.VectorQuantization][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.vq.core_vq.preprocess][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
[Misc] Fix docs and update image edit example with cache-dit (#318)
2025-12-15 21:43:28 +08:00
## Configuration
Configuration classes.
- [vllm_omni.config.model.OmniModelConfig][]
[Doc] Sync and fix. (#2110)
2026-03-24 11:20:50 +08:00
- [vllm_omni.config.stage_config.StageConfig][]
- [vllm_omni.config.stage_config.StageConfigFactory][]
- [vllm_omni.config.stage_config.StageType][]
[Misc] Fix docs and update image edit example with cache-dit (#318)
2025-12-15 21:43:28 +08:00
- [vllm_omni.diffusion.cache.teacache.config.TeaCacheConfig][]
- [vllm_omni.distributed.omni_connectors.utils.config.ConnectorSpec][]
- [vllm_omni.distributed.omni_connectors.utils.config.OmniTransferConfig][]
[Doc] Sync and fix. (#2110)
2026-03-24 11:20:50 +08:00
- [vllm_omni.model_executor.models.fish_speech.configuration_fish_speech.FishSpeechConfig][]
- [vllm_omni.model_executor.models.fish_speech.configuration_fish_speech.FishSpeechFastARConfig][]
- [vllm_omni.model_executor.models.fish_speech.configuration_fish_speech.FishSpeechSlowARConfig][]
[Docs] update async chunk docs diagram [skip ci] (#1530)
2026-02-28 10:42:50 +08:00
- [vllm_omni.model_executor.models.mimo_audio.config_mimo_audio.MiMoAudioConfig][]
- [vllm_omni.model_executor.models.mimo_audio.config_mimo_audio.MiMoAudioTokenizerConfig][]
[Model] Support Qwen3-TTS model series (#895)
2026-01-22 16:57:37 +08:00
- [vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts.Qwen3TTSConfig][]
- [vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts.Qwen3TTSSpeakerEncoderConfig][]
- [vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts.Qwen3TTSTalkerCodePredictorConfig][]
- [vllm_omni.model_executor.models.qwen3_tts.configuration_qwen3_tts.Qwen3TTSTalkerConfig][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_12hz.configuration_qwen3_tts_tokenizer_v2.Qwen3TTSTokenizerV2Config][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_12hz.configuration_qwen3_tts_tokenizer_v2.Qwen3TTSTokenizerV2DecoderConfig][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1Config][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1DecoderBigVGANConfig][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1DecoderConfig][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1DecoderDiTConfig][]
- [vllm_omni.model_executor.models.qwen3_tts.tokenizer_25hz.configuration_qwen3_tts_tokenizer_v1.Qwen3TTSTokenizerV1EncoderConfig][]
[Refactor] Migrate and clean up TTS configs: CosyVoice3, OmniVoice, VoxCPM (#3338)
2026-05-17 07:32:34 +08:00
- [vllm_omni.transformers_utils.configs.cosyvoice3.CosyVoice3Config][]
[Fix] Restore MammothModa2 support (#6694)
2026-08-28 08:32:32 +08:00
- [vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Config][]
- [vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Qwen2_5_VLConfig][]
- [vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Qwen2_5_VLTextConfig][]
- [vllm_omni.transformers_utils.configs.mammoth_moda2.Mammothmoda2Qwen2_5_VLVisionConfig][]
[Refactor] Migrate and clean up TTS configs: CosyVoice3, OmniVoice, VoxCPM (#3338)
2026-05-17 07:32:34 +08:00
- [vllm_omni.transformers_utils.configs.omnivoice.OmniVoiceConfig][]
- [vllm_omni.transformers_utils.configs.voxcpm2.VoxCPM2Config][]
[Misc] Fix docs and update image edit example with cache-dit (#318)
2025-12-15 21:43:28 +08:00
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
## Workers
Worker classes and model runners for distributed inference.
[Doc]Fix - Align with repo. (#1176)
2026-02-03 18:11:50 +08:00
- [vllm_omni.diffusion.worker.diffusion_model_runner.DiffusionModelRunner][]
[skip ci][Doc] add design docs for async chunk in qwen3-omni (#962)
2026-02-27 10:46:13 +08:00
- [vllm_omni.diffusion.worker.diffusion_worker.CustomPipelineWorkerExtension][]
[Doc]Fix - Align with repo. (#1176)
2026-02-03 18:11:50 +08:00
- [vllm_omni.diffusion.worker.diffusion_worker.DiffusionWorker][]
- [vllm_omni.diffusion.worker.diffusion_worker.WorkerProc][]
[skip ci][Doc] add design docs for async chunk in qwen3-omni (#962)
2026-02-27 10:46:13 +08:00
- [vllm_omni.diffusion.worker.diffusion_worker.WorkerWrapperBase][]
[Refactor]: clean up diffusion scheduler interfaces (#5216)
2026-07-22 14:05:49 +08:00
- [vllm_omni.diffusion.worker.utils.StepRequestState][]
[Doc] Sync and fix. (#2110)
2026-03-24 11:20:50 +08:00
- [vllm_omni.diffusion.worker.utils.RunnerOutput][]
[Doc]Fix - Align with repo. (#1176)
2026-02-03 18:11:50 +08:00
- [vllm_omni.platforms.npu.worker.npu_ar_model_runner.ExecuteModelState][]
- [vllm_omni.platforms.npu.worker.npu_ar_model_runner.NPUARModelRunner][]
- [vllm_omni.platforms.npu.worker.npu_ar_worker.NPUARWorker][]
- [vllm_omni.platforms.npu.worker.npu_generation_model_runner.NPUGenerationModelRunner][]
- [vllm_omni.platforms.npu.worker.npu_generation_worker.NPUGenerationWorker][]
- [vllm_omni.platforms.npu.worker.npu_model_runner.OmniNPUModelRunner][]
- [vllm_omni.platforms.xpu.worker.xpu_ar_model_runner.XPUARModelRunner][]
- [vllm_omni.platforms.xpu.worker.xpu_ar_worker.XPUARWorker][]
- [vllm_omni.platforms.xpu.worker.xpu_generation_model_runner.XPUGenerationModelRunner][]
- [vllm_omni.platforms.xpu.worker.xpu_generation_worker.XPUGenerationWorker][]
[Doc] Sync (#1216)
2026-02-11 06:00:43 +08:00
- [vllm_omni.worker.base.OmniGPUWorkerBase][]
[Doc] Refine the document. (#403)
2025-12-22 21:29:40 +08:00
- [vllm_omni.worker.gpu_ar_model_runner.ExecuteModelState][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
- [vllm_omni.worker.gpu_ar_model_runner.GPUARModelRunner][]
- [vllm_omni.worker.gpu_ar_worker.GPUARWorker][]
[Bugfix] Remove mandatory flash-attn dependency and optimzie docs (#113)
2025-11-30 18:20:44 +08:00
- [vllm_omni.worker.gpu_generation_model_runner.GPUGenerationModelRunner][]
- [vllm_omni.worker.gpu_generation_worker.GPUGenerationWorker][]
[Doc] Sync (#1216)
2026-02-11 06:00:43 +08:00
- [vllm_omni.worker.gpu_memory_utils.parse_cuda_visible_devices][]
[Doc] Sync and fix. (#2110)
2026-03-24 11:20:50 +08:00
- [vllm_omni.worker.gpu_model_runner.CUDAGraphWrapper][]
[Docs] optimize and supplement docs system (#86)
2025-11-28 11:03:52 +08:00
- [vllm_omni.worker.gpu_model_runner.OmniGPUModelRunner][]
[Doc]Fix - Align with repo. (#1176)
2026-02-03 18:11:50 +08:00
- [vllm_omni.worker.mixins.OmniWorkerMixin][]
Reference in new issue Copy Permalink