[Doc]Fixed some spelling errors (#199)
Signed-off-by: princepride <wangzhipeng628@gmail.com>
This commit is contained in:
3 files changed
+6
-6
No files matched your search
@@ -16,7 +16,7 @@ vLLM-Omni is a multi-modality extension for vLLM that supports non-autoregressiv
|
||||
|
||||
API Server --> OmniLLM/AsyncOmni (New, including multi engines) --> LLMEngine/AsyncLLM --> Engine Core
|
||||
--> Scheduler (New one for DiT) --> Executor (New one for diffusers) --> Worker (New one for DiT)
|
||||
--> ModelRunner (New one for AR hiddenstate, New one for DiT) --> RequestState --> OutputProcessoer (New one for final multimodal output)
|
||||
--> ModelRunner (New one for AR hiddenstate, New one for DiT) --> RequestState --> OutputProcessor (New one for final multimodal output)
|
||||
|
||||
## Core Components (Ordered by Data Flow)
|
||||
|
||||
@@ -137,7 +137,7 @@ class OmniLLM(LLM):
|
||||
prompt_str, engine_request, tokenization_kwargs = self._process_stage_inputs(stage_config, **stage_args)
|
||||
|
||||
# Add inputs to Engine
|
||||
stage_engine.add_request(requesy_id, prompt_str, tokenization_kwargs)
|
||||
stage_engine.add_request(request_id, prompt_str, tokenization_kwargs)
|
||||
# Run Engine
|
||||
stage_output = stage_engine.step()
|
||||
|
||||
@@ -157,7 +157,7 @@ class OmniLLM(LLM):
|
||||
return self._process_ar_inputs(**stage_args)
|
||||
elif stage_config.engine_type == "DiT":
|
||||
return self._process_dit_inputs(**stage_args)
|
||||
else
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
def _process_dit_inputs(**stage_args)-> Any:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Installation
|
||||
|
||||
vLLM supports the following hardware platforms:
|
||||
vLLM-Omni supports the following hardware platforms:
|
||||
|
||||
- [GPU](gpu.md)
|
||||
- [NVIDIA CUDA](gpu.md)
|
||||
|
||||
+2
-2
@@ -2,7 +2,7 @@
|
||||
|
||||
> Q: How many chips do I need to infer a model in vLLM-Omni?
|
||||
|
||||
A: Now, we support natively disaggregated deployment for different model stages within a model. There is a restriction that one chip can only has one AutoRegressive model stage. This is because the unified KV cache management of vLLM. Stages of other types can coexist with in a chip. The restriction will be resolved in later version.
|
||||
A: Now, we support natively disaggregated deployment for different model stages within a model. There is a restriction that one chip can only have one AutoRegressive model stage. This is because the unified KV cache management of vLLM. Stages of other types can coexist within a chip. The restriction will be resolved in later version.
|
||||
|
||||
> Q: When trying to run examples, I encounter error about backend of librosa or soundfile. How to solve it?
|
||||
|
||||
@@ -14,7 +14,7 @@ sudo apt install ffmpeg
|
||||
|
||||
> Q: I encounter some bugs or CI problems, which is urgent. How can I solve it?
|
||||
|
||||
A: At first, you can check current [issues](https://github.com/vllm-project/vllm-omni/issues) to find possible solutions. If non of these satisfy your demand and it is urgent, please find these [volunteers](https://docs.vllm.ai/projects/vllm-omni/en/latest/community/volunteers/) for help.
|
||||
A: At first, you can check current [issues](https://github.com/vllm-project/vllm-omni/issues) to find possible solutions. If none of these satisfy your demand and it is urgent, please find these [volunteers](https://docs.vllm.ai/projects/vllm-omni/en/latest/community/volunteers/) for help.
|
||||
|
||||
> Q: Does vLLM-Omni support AWQ or any other quantization?
|
||||
|
||||
|
||||
Reference in new issue
Block a user