236 lines
6.9 KiB
Markdown
236 lines
6.9 KiB
Markdown
# vLLM-omni API Documentation
|
|
|
|
This directory contains comprehensive API design documentation for all core modules in vLLM-omni. These templates provide a standardized structure for designing and implementing the core, engine, executor, and worker modules.
|
|
|
|
## 📋 API Design Templates
|
|
|
|
### 1. [API Design Template](API_DESIGN_TEMPLATE.md)
|
|
**Master template** for creating API documentation for any module in vLLM-omni. Use this as a starting point for new modules.
|
|
|
|
### 2. [Core Module API](core_module_api.md)
|
|
**Core module** provides fundamental scheduling, caching, and resource management functionality.
|
|
|
|
**Key Components:**
|
|
- Request scheduling and prioritization
|
|
- DiT cache management for diffusion models
|
|
- Resource allocation and coordination
|
|
- Inter-module communication
|
|
|
|
### 3. [Engine Module API](engine_module_api.md)
|
|
**Engine module** handles model loading, inference execution, and output processing.
|
|
|
|
**Key Components:**
|
|
- Model loading and initialization
|
|
- Inference execution for AR and diffusion models
|
|
- Input preprocessing for various modalities
|
|
- Output postprocessing and formatting
|
|
|
|
### 4. [Executor Module API](executor_module_api.md)
|
|
**Executor module** coordinates and manages request execution across different engines and workers.
|
|
|
|
**Key Components:**
|
|
- Request routing and load balancing
|
|
- Execution pipeline coordination
|
|
- Worker management and task distribution
|
|
- Error handling and recovery
|
|
|
|
### 5. [Worker Module API](worker_module_api.md)
|
|
**Worker module** provides the actual execution environment for model inference.
|
|
|
|
**Key Components:**
|
|
- Model execution and inference
|
|
- GPU resource management
|
|
- Request batching and processing
|
|
- Performance optimization
|
|
|
|
## 🎯 How to Use These Templates
|
|
|
|
### For New Module Development
|
|
|
|
1. **Start with the Master Template**
|
|
- Copy `API_DESIGN_TEMPLATE.md`
|
|
- Rename to `[module_name]_api.md`
|
|
- Follow the structure exactly
|
|
|
|
2. **Fill in Module-Specific Details**
|
|
- Update the module overview
|
|
- Define core classes and interfaces
|
|
- Specify public API methods
|
|
- Add configuration options
|
|
- Define error handling strategy
|
|
- Provide usage examples
|
|
|
|
3. **Review and Validate**
|
|
- Ensure all sections are complete
|
|
- Verify code examples work
|
|
- Check for consistency with other modules
|
|
- Validate configuration options
|
|
|
|
### For Existing Module Updates
|
|
|
|
1. **Update API Documentation**
|
|
- Modify existing API files
|
|
- Add new methods and classes
|
|
- Update examples and configuration
|
|
- Maintain backward compatibility notes
|
|
|
|
2. **Version Control**
|
|
- Track changes in git
|
|
- Use clear commit messages
|
|
- Tag major API changes
|
|
|
|
## 📚 Template Structure
|
|
|
|
Each API template follows this standardized structure:
|
|
|
|
### 1. Module Overview
|
|
- **Purpose**: What the module does
|
|
- **Responsibilities**: Key responsibilities
|
|
- **Dependencies**: Required modules
|
|
- **Integration Points**: How it connects with other modules
|
|
|
|
### 2. Core Classes/Interfaces
|
|
- **Base Classes**: Abstract base classes
|
|
- **Implementation Classes**: Concrete implementations
|
|
- **Data Structures**: Key data models
|
|
|
|
### 3. Public API Methods
|
|
- **Initialization**: Constructor and setup
|
|
- **Core Operations**: Main functionality
|
|
- **Configuration**: Settings management
|
|
- **Lifecycle Management**: Start/stop/cleanup
|
|
- **Monitoring**: Status and metrics
|
|
|
|
### 4. Configuration
|
|
- **Configuration Classes**: Dataclasses for settings
|
|
- **Required Parameters**: Must-have settings
|
|
- **Optional Parameters**: Optional settings with defaults
|
|
- **Validation**: Parameter validation rules
|
|
|
|
### 5. Error Handling
|
|
- **Custom Exceptions**: Module-specific errors
|
|
- **Error Codes**: Standardized error codes
|
|
- **Recovery Strategies**: Error recovery approaches
|
|
|
|
### 6. Examples
|
|
- **Basic Usage**: Simple usage examples
|
|
- **Advanced Usage**: Complex scenarios
|
|
- **Integration Examples**: Multi-module usage
|
|
|
|
## 🔧 Implementation Guidelines
|
|
|
|
### Code Standards
|
|
- Use type hints for all methods
|
|
- Include comprehensive docstrings
|
|
- Follow PEP 8 style guidelines
|
|
- Use async/await for I/O operations
|
|
|
|
### Error Handling
|
|
- Define specific exception types
|
|
- Provide meaningful error messages
|
|
- Include error recovery strategies
|
|
- Log errors appropriately
|
|
|
|
### Configuration
|
|
- Use dataclasses for configuration
|
|
- Provide sensible defaults
|
|
- Include validation methods
|
|
- Support environment variables
|
|
|
|
### Testing
|
|
- Write unit tests for all public methods
|
|
- Include integration tests
|
|
- Test error conditions
|
|
- Validate configuration options
|
|
|
|
## 🚀 Getting Started
|
|
|
|
### For Developers
|
|
|
|
1. **Choose a Module**
|
|
- Pick the module you want to work on
|
|
- Read the corresponding API documentation
|
|
- Understand the responsibilities and interfaces
|
|
|
|
2. **Set Up Development Environment**
|
|
```bash
|
|
# Clone the repository
|
|
git clone https://github.com/hsliuustc0106/vllm-omni.git
|
|
cd vllm-omni
|
|
|
|
# Install dependencies
|
|
pip install -r requirements-dev.txt
|
|
|
|
# Set up pre-commit hooks
|
|
pre-commit install
|
|
```
|
|
|
|
3. **Start Implementation**
|
|
- Create the module directory
|
|
- Implement base classes first
|
|
- Add concrete implementations
|
|
- Write tests as you go
|
|
|
|
### For Contributors
|
|
|
|
1. **Review API Documentation**
|
|
- Read through all module APIs
|
|
- Understand the overall architecture
|
|
- Identify areas for improvement
|
|
|
|
2. **Propose Changes**
|
|
- Create issues for API changes
|
|
- Discuss in pull requests
|
|
- Update documentation accordingly
|
|
|
|
3. **Submit Contributions**
|
|
- Follow the coding standards
|
|
- Include tests for new features
|
|
- Update documentation
|
|
- Submit pull requests
|
|
|
|
## 📖 Additional Resources
|
|
|
|
### Architecture Overview
|
|
- [vLLM-omni Architecture Design](../architecture/vLLM-omni%20arch%20design%20doc.md)
|
|
- [Component Categorization](../../ARCHITECTURE_CATEGORIZATION.md)
|
|
|
|
### Development Guides
|
|
- [Development Setup](../../README.md#development)
|
|
- [Testing Guidelines](../../tests/README.md)
|
|
- [Contributing Guidelines](../../CONTRIBUTING.md)
|
|
|
|
### Examples
|
|
- [Basic Examples](../../examples/basic/)
|
|
- [Advanced Examples](../../examples/advanced/)
|
|
- [Multimodal Examples](../../examples/multimodal/)
|
|
|
|
## 🤝 Contributing
|
|
|
|
We welcome contributions to improve the API documentation and implementation. Please:
|
|
|
|
1. **Follow the Template Structure**
|
|
- Use the master template as a guide
|
|
- Maintain consistency across modules
|
|
- Include all required sections
|
|
|
|
2. **Provide Working Examples**
|
|
- Test all code examples
|
|
- Include both basic and advanced usage
|
|
- Show integration patterns
|
|
|
|
3. **Keep Documentation Updated**
|
|
- Update docs when APIs change
|
|
- Version control documentation changes
|
|
- Maintain backward compatibility notes
|
|
|
|
## 📝 Notes
|
|
|
|
- All API documentation should be kept up-to-date with implementation
|
|
- Code examples should be tested and working
|
|
- Configuration options should be validated
|
|
- Error handling should be comprehensive
|
|
- Performance considerations should be documented
|
|
|
|
For questions or suggestions about the API documentation, please open an issue or start a discussion in the repository.
|