[CI]Remove DreamID-Omni and MagiHuman support (#6357)
Signed-off-by: wangyu <410167048@qq.com> Signed-off-by: [Your Name] <your.email@example.com> Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
1 parent
40dcfdbb43
commit
aeb348d8f8
31 files changed
+26
-6957
No files matched your search
@@ -16,20 +16,19 @@ vllm_omni/diffusion/models/wan2_2/
|
||||
|
||||
The transformer is loaded separately via `weights_sources` + `load_weights()`. Non-transformer components (VAE, text encoder) are loaded in `__init__` via `from_pretrained()`.
|
||||
|
||||
### Custom model with external deps (e.g., DreamID-Omni)
|
||||
### Custom model with external deps
|
||||
|
||||
```
|
||||
vllm_omni/diffusion/models/dreamid_omni/
|
||||
vllm_omni/diffusion/models/<name>/
|
||||
├── __init__.py # Exports pipeline only
|
||||
├── pipeline_dreamid_omni.py # Pipeline: loads ALL weights in __init__ via custom helpers
|
||||
├── fusion.py # Custom fusion architecture (video + audio cross-attention)
|
||||
└── wan2_2.py # Re-implemented Wan backbone with split API
|
||||
├── pipeline_<name>.py # Pipeline: loads ALL weights in __init__ via custom helpers
|
||||
└── <backbone>.py # Custom transformer / fusion architecture
|
||||
|
||||
examples/offline_inference/x_to_video_audio/
|
||||
└── download_dreamid_omni.py # Downloads weights from 3 HF repos + clones code repo
|
||||
examples/offline_inference/<task>/
|
||||
└── download_<name>.py # Downloads weights and/or clones code repo
|
||||
```
|
||||
|
||||
All weights loaded eagerly in `__init__`. `load_weights()` is a no-op. External dependency (`dreamid_omni` package) imported with try/except.
|
||||
All weights loaded eagerly in `__init__`. `load_weights()` is a no-op. External dependency imported with try/except.
|
||||
|
||||
### Custom model with ported code (e.g., BAGEL)
|
||||
|
||||
@@ -76,7 +75,7 @@ self.weights_sources = [
|
||||
]
|
||||
```
|
||||
|
||||
### Pattern 3: Fully custom loading (DreamID-Omni)
|
||||
### Pattern 3: Fully custom loading
|
||||
|
||||
```
|
||||
init → load ALL weights eagerly via custom helpers → load_weights() = no-op
|
||||
@@ -108,8 +107,8 @@ Standard diffusers `model_index.json`:
|
||||
Custom model `model_index.json` (minimal):
|
||||
```json
|
||||
{
|
||||
"_class_name": "DreamIDOmniPipeline",
|
||||
"fusion": "DreamID-Omni/dreamid_omni.safetensors"
|
||||
"_class_name": "YourCustomPipeline",
|
||||
"fusion": "weights/model.safetensors"
|
||||
}
|
||||
```
|
||||
|
||||
@@ -117,7 +116,7 @@ The only **required** field is `_class_name` — it must match a key in `_DIFFUS
|
||||
|
||||
## External Dependency Management
|
||||
|
||||
### Git clone + .pth injection (DreamID-Omni pattern)
|
||||
### Git clone + .pth injection
|
||||
|
||||
```python
|
||||
def download_dependency():
|
||||
@@ -163,7 +162,7 @@ The engine checks `isinstance(pipeline, SupportImageInput)` at startup to config
|
||||
|
||||
Diffusers models use `config.json` in each subfolder. Custom models often use:
|
||||
|
||||
**Module-level config dicts** (DreamID-Omni):
|
||||
**Module-level config dicts**:
|
||||
```python
|
||||
VIDEO_CONFIG = {
|
||||
"patch_size": [1, 2, 2], "model_type": "ti2v",
|
||||
@@ -181,7 +180,7 @@ vae_cfg = bagel_cfg.get("vae_config", {})
|
||||
|
||||
## Custom Architecture Patterns
|
||||
|
||||
### Split forward API (DreamID-Omni)
|
||||
### Split forward API
|
||||
|
||||
When a fusion model needs to interleave blocks from two backbones:
|
||||
|
||||
|
||||
Reference in new issue
Block a user