144 lines
4.0 KiB
Markdown
144 lines
4.0 KiB
Markdown
# API Design Documentation
|
|
|
|
This document contains comprehensive API design documentation for all core modules in vLLM-omni. These templates provide a standardized structure for designing and implementing the core, engine, executor, and worker modules.
|
|
|
|
## 📋 Module API
|
|
|
|
### Core Module API
|
|
**Core module** provides fundamental scheduling, caching, and resource management functionality.
|
|
|
|
**Key Components:**
|
|
- Request scheduling and prioritization
|
|
- DiT cache management for diffusion models
|
|
- Resource allocation and coordination
|
|
- Inter-module communication
|
|
|
|
### Engine Module API
|
|
**Engine module** handles model loading, inference execution, and output processing.
|
|
|
|
**Key Components:**
|
|
- Model loading and initialization
|
|
- Inference execution for AR and diffusion models
|
|
- Input preprocessing for various modalities
|
|
- Output postprocessing and formatting
|
|
|
|
### Executor Module API
|
|
**Executor module** coordinates and manages request execution across different engines and workers.
|
|
|
|
**Key Components:**
|
|
- Request routing and load balancing
|
|
- Execution pipeline coordination
|
|
- Worker management and task distribution
|
|
- Error handling and recovery
|
|
|
|
### Worker Module API
|
|
**Worker module** provides the actual execution environment for model inference.
|
|
|
|
**Key Components:**
|
|
- Model execution and inference
|
|
- GPU resource management
|
|
- Request batching and processing
|
|
- Performance optimization
|
|
|
|
|
|
## 🔧 Implementation Guidelines
|
|
|
|
### Code Standards
|
|
- Use type hints for all methods
|
|
- Include comprehensive docstrings
|
|
- Follow PEP 8 style guidelines
|
|
- Use async/await for I/O operations
|
|
|
|
### Error Handling
|
|
- Define specific exception types
|
|
- Provide meaningful error messages
|
|
- Include error recovery strategies
|
|
- Log errors appropriately
|
|
|
|
### Configuration
|
|
- Use dataclasses for configuration
|
|
- Provide sensible defaults
|
|
- Include validation methods
|
|
- Support environment variables
|
|
|
|
### Testing
|
|
- Write unit tests for all public methods
|
|
- Include integration tests
|
|
- Test error conditions
|
|
- Validate configuration options
|
|
|
|
## 🚀 Getting Started
|
|
|
|
### For Developers
|
|
|
|
1. **Choose a Module**
|
|
- Pick the module you want to work on
|
|
- Read the corresponding API documentation
|
|
- Understand the responsibilities and interfaces
|
|
|
|
2. **Set Up Development Environment**
|
|
```bash
|
|
# Clone the repository
|
|
git clone https://github.com/vllm-project/vllm-omni.git
|
|
cd vllm-omni
|
|
|
|
# Install dependencies
|
|
pip install -e ".[dev]"
|
|
|
|
# Set up pre-commit hooks
|
|
pre-commit install
|
|
```
|
|
|
|
3. **Start Implementation**
|
|
- Create the module directory
|
|
- Implement base classes first
|
|
- Add concrete implementations
|
|
- Write tests as you go
|
|
|
|
### For Contributors
|
|
|
|
1. **Review API Documentation**
|
|
- Read through all module APIs
|
|
- Understand the overall architecture
|
|
- Identify areas for improvement
|
|
|
|
2. **Propose Changes**
|
|
- Create issues for API changes
|
|
- Discuss in pull requests
|
|
- Update documentation accordingly
|
|
|
|
3. **Submit Contributions**
|
|
- Follow the coding standards
|
|
- Include tests for new features
|
|
- Update documentation
|
|
- Submit pull requests
|
|
|
|
## 🤝 Contributing
|
|
|
|
We welcome contributions to improve the API documentation and implementation. Please:
|
|
|
|
1. **Follow the Template Structure**
|
|
- Use the [api template](https://github.com/vllm-project/vllm-omni/docs/contributing/design_documents/api_design_template.md) as a guide
|
|
- Maintain consistency across modules
|
|
- Include all required sections
|
|
|
|
2. **Provide Working Examples**
|
|
- Test all code examples
|
|
- Include both basic and advanced usage
|
|
- Show integration patterns
|
|
|
|
3. **Keep Documentation Updated**
|
|
- Update docs when APIs change
|
|
- Version control documentation changes
|
|
- Maintain backward compatibility notes
|
|
|
|
## 📝 Notes
|
|
|
|
- All API documentation should be kept up-to-date with implementation
|
|
- Code examples should be tested and working
|
|
- Configuration options should be validated
|
|
- Error handling should be comprehensive
|
|
- Performance considerations should be documented
|
|
|
|
For questions or suggestions about the API documentation, please open an issue or start a discussion in the repository.
|