commit 79104a29649c8f1a39b7dddc9e694bcf6ef83d70 Author: Deploy Bot Date: Thu Jul 17 16:08:27 2025 -0400 Initial commit: Add documentation and move to docs/ directory diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..f6d7ce2 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,190 @@ +# Dokku Docker Compose Plugin Architecture + +## Overview + +This document describes the architecture of the Dokku Docker Compose plugin, which enables importing Docker Compose files into Dokku while maintaining compatibility with existing Dokku plugins and features. + +## High-Level Architecture + +```mermaid +graph TD + A[Docker Compose File] --> B[Parser] + B --> C[Service Graph] + C --> D[Transformer] + D --> E[Dokku Resources] + E --> F[Dokku Core] + G[Plugins] --> F + H[CLI] --> B + H --> D + H --> F +``` + +## Core Components + +### 1. Parser +- **Responsibility**: Parse and validate Docker Compose files +- **Input**: `docker-compose.yml` (v2/v3) +- **Output**: Normalized service definitions +- **Features**: + - YAML parsing with schema validation + - Environment variable interpolation + - Extension point for custom validators + +### 2. Service Graph +- **Responsibility**: Model service dependencies and relationships +- **Components**: + - Dependency resolver + - Cycle detection + - Topological sorter +- **Features**: + - Builds directed acyclic graph (DAG) of services + - Determines execution order + - Identifies independent services for parallel processing + +### 3. Transformer +- **Responsibility**: Convert Docker Compose concepts to Dokku resources +- **Components**: + - Resource mappers (networks, volumes, etc.) + - Plugin detector + - Environment variable processor +- **Features**: + - Handles Docker-specific configurations + - Integrates with Dokku plugins + - Manages resource naming and references + +### 4. Executor +- **Responsibility**: Coordinate the creation/update of Dokku resources +- **Components**: + - Operation planner + - Dependency manager + - Rollback handler +- **Features**: + - Transactional updates + - Parallel execution where possible + - Automatic rollback on failure + +## Data Flow + +1. **Initialization**: + - Load and parse `docker-compose.yml` + - Resolve environment variables + - Validate against schema + +2. **Analysis**: + - Build service dependency graph + - Detect plugin integrations + - Plan operations + +3. **Execution**: + - Create/update resources in dependency order + - Handle plugin integrations + - Configure networking and storage + +4. **Verification**: + - Run health checks + - Verify service availability + - Perform post-deployment tasks + +## Key Design Decisions + +### 1. Plugin Integration +- **Approach**: Use Dokku's plugin system for service management +- **Rationale**: Leverage existing, well-tested implementations +- **Implementation**: + - Detect compatible plugins based on image names + - Map Docker Compose configurations to plugin commands + - Fall back to container-based deployment when no plugin is available + +### 2. State Management +- **Approach**: Store minimal state in Dokku's configuration +- **Rationale**: Ensure consistency and enable recovery +- **Implementation**: + - Store original compose file checksum + - Track resource mappings + - Maintain version history + +### 3. Error Handling +- **Approach**: Fail fast with clear error messages +- **Rationale**: Prevent partial or inconsistent states +- **Implementation**: + - Input validation + - Pre-flight checks + - Transactional updates with rollback + +## Integration Points + +### Dokku Core +- App management +- Network configuration +- Volume management +- Process management + +### External Systems +- Docker Registry (for image pulls) +- Docker Daemon (for container operations) +- System Package Manager (for plugin installation) + +## Security Considerations + +1. **Input Validation**: + - Sanitize all inputs + - Validate against schema + - Restrict file system access + +2. **Secret Management**: + - Handle environment variables securely + - Support Dokku's secret management + - Avoid logging sensitive data + +3. **Access Control**: + - Respect file permissions + - Run with least privilege + - Validate user permissions + +## Performance Considerations + +1. **Parallel Processing**: + - Process independent services concurrently + - Limit concurrency to prevent resource exhaustion + +2. **Caching**: + - Cache parsed compose files + - Store intermediate build artifacts + +3. **Optimizations**: + - Lazy loading of resources + - Incremental updates + - Batch operations where possible + +## Monitoring and Observability + +1. **Logging**: + - Structured logging + - Different log levels + - Operation context + +2. **Metrics**: + - Operation duration + - Resource usage + - Success/failure rates + +3. **Tracing**: + - Request tracing + - Dependency tracking + - Performance analysis + +## Future Extensions + +1. **Plugin System**: + - Custom transformers + - Additional validators + - Post-processing hooks + +2. **Export Functionality**: + - Generate compose files from existing Dokku apps + - Support for different output formats + +3. **Advanced Features**: + - Blue/green deployments + - Canary releases + - A/B testing support diff --git a/docs/EXAMPLE_PROJECTS.md b/docs/EXAMPLE_PROJECTS.md new file mode 100644 index 0000000..7fea914 --- /dev/null +++ b/docs/EXAMPLE_PROJECTS.md @@ -0,0 +1,527 @@ +# Example Projects for Testing + +This document lists various projects with Docker Compose files of increasing complexity, which will be used to test the Dokku Docker Compose plugin. + +## Difficulty: Beginner (Single Service) + +### 1. Nginx Web Server +**Description**: Basic Nginx web server with a custom HTML page +**Key Features**: +- Single service +- Volume mount for static content +- Port mapping + +```yaml +version: '3.8' +services: + web: + image: nginx:alpine + ports: + - "8080:80" + volumes: + - ./html:/usr/share/nginx/html +``` + +### 2. Redis Cache +**Description**: Simple Redis instance +**Key Features**: +- Single service +- Volume for data persistence +- Custom configuration + +```yaml +version: '3.8' +services: + redis: + image: redis:alpine + ports: + - "6379:6379" + volumes: + - redis_data:/data + command: redis-server --requirepass yourpassword + +volumes: + redis_data: +``` + +## Difficulty: Intermediate (Multiple Services) + +### 3. WordPress with MySQL +**Description**: WordPress with a separate MySQL database +**Key Features**: +- Multi-service +- Environment variables +- Volume mounts +- Service dependencies + +```yaml +version: '3.8' +services: + wordpress: + image: wordpress:latest + ports: + - "8000:80" + environment: + WORDPRESS_DB_HOST: db + WORDPRESS_DB_USER: wordpress + WORDPRESS_DB_PASSWORD: wordpress + WORDPRESS_DB_NAME: wordpress + depends_on: + - db + volumes: + - wordpress_data:/var/www/html + + db: + image: mysql:5.7 + environment: + MYSQL_DATABASE: wordpress + MYSQL_USER: wordpress + MYSQL_PASSWORD: wordpress + MYSQL_RANDOM_ROOT_PASSWORD: '1' + volumes: + - db_data:/var/lib/mysql + +volumes: + wordpress_data: + db_data: +``` + +### 4. Node.js App with MongoDB +**Description**: Simple Node.js application with MongoDB +**Key Features**: +- Multi-service +- Build context +- Environment variables +- Network configuration + +```yaml +version: '3.8' +services: + app: + build: . + ports: + - "3000:3000" + environment: + - NODE_ENV=development + - MONGODB_URI=mongodb://mongo:27017/mydb + depends_on: + - mongo + networks: + - app-network + + mongo: + image: mongo:latest + volumes: + - mongo_data:/data/db + networks: + - app-network + +networks: + app-network: + driver: bridge + +volumes: + mongo_data: +``` + +## Difficulty: Advanced (Complex Applications) + +### 5. Nextcloud +**Description**: Self-hosted productivity platform +**Key Features**: +- Multiple interconnected services +- Database and caching +- Volume mounts for persistence +- Environment configuration +- Health checks +- Complex networking + +```yaml +version: '3.8' +services: + db: + image: mariadb:10.6 + command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW + restart: always + environment: + - MYSQL_ROOT_PASSWORD=db_root_password + - MYSQL_PASSWORD=db_password + - MYSQL_DATABASE=nextcloud + - MYSQL_USER=nextcloud + volumes: + - db:/var/lib/mysql + + redis: + image: redis:alpine + restart: always + + app: + image: nextcloud:apache + restart: always + ports: + - 8080:80 + links: + - db + - redis + environment: + - MYSQL_PASSWORD=db_password + - MYSQL_DATABASE=nextcloud + - MYSQL_USER=nextcloud + - MYSQL_HOST=db + - REDIS_HOST=redis + - OVERWRITEPROTOCOL=https + volumes: + - nextcloud:/var/www/html + depends_on: + - db + - redis + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:80/status.php"] + interval: 30s + timeout: 10s + retries: 3 + +volumes: + db: + nextcloud: +``` + +### 6. Plane (Project Management) +**Description**: Open-source project management tool +**Key Features**: +- Multiple microservices +- Database and cache +- File storage +- Background workers +- Environment variables +- Health checks + +```yaml +version: '3.8' +services: + web: + image: planepowers/plane-web:latest + environment: + - NEXT_PUBLIC_API_BASE_URL=${API_BASE_URL} + - NEXT_PUBLIC_WS_BASE_URL=${WS_BASE_URL} + ports: + - "3000:3000" + depends_on: + - api + restart: unless-stopped + + api: + image: planepowers/plane-api:latest + environment: + - DATABASE_URL=${DATABASE_URL} + - REDIS_URL=${REDIS_URL} + - JWT_SECRET=${JWT_SECRET} + - CORS_ORIGINS=${CORS_ORIGINS} + depends_on: + - db + - redis + restart: unless-stopped + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8000/health/"] + interval: 30s + timeout: 10s + retries: 3 + + worker: + image: planepowers/plane-worker:latest + environment: + - DATABASE_URL=${DATABASE_URL} + - REDIS_URL=${REDIS_URL} + depends_on: + - db + - redis + restart: unless-stopped + + db: + image: postgres:13-alpine + environment: + - POSTGRES_USER=${DB_USER} + - POSTGRES_PASSWORD=${DB_PASSWORD} + - POSTGRES_DB=${DB_NAME} + volumes: + - postgres_data:/var/lib/postgresql/data + restart: unless-stopped + + redis: + image: redis:alpine + command: redis-server --requirepass ${REDIS_PASSWORD} + volumes: + - redis_data:/data + restart: unless-stopped + + minio: + image: minio/minio:latest + volumes: + - minio_data:/data + environment: + - MINIO_ROOT_USER=${MINIO_ACCESS_KEY} + - MINIO_ROOT_PASSWORD=${MINIO_SECRET_KEY} + command: server /data + restart: unless-stopped + +volumes: + postgres_data: + redis_data: + minio_data: +``` + +## Difficulty: Expert (Enterprise-Grade) + +### 7. GitLab CE +**Description**: Complete DevOps platform +**Key Features**: +- Multiple interconnected services +- Database, cache, and search +- File storage +- Background processing +- Complex networking +- Resource limits +- Health checks + +```yaml +version: '3.8' +services: + gitlab: + image: 'gitlab/gitlab-ce:latest' + restart: always + hostname: 'gitlab.example.com' + environment: + GITLAB_OMNIBUS_CONFIG: | + external_url 'https://gitlab.example.com' + gitlab_rails['gitlab_shell_ssh_port'] = 2222 + ports: + - '80:80' + - '443:443' + - '2222:22' + volumes: + - '$GITLAB_HOME/config:/etc/gitlab' + - '$GITLAB_HOME/logs:/var/log/gitlab' + - '$GITLAB_HOME/data:/var/opt/gitlab' + shm_size: '256m' + healthcheck: + test: ['CMD', '/opt/gitlab/bin/gitlab-healthcheck', '--fail'] + interval: 30s + timeout: 10s + retries: 3 + start_period: 5m +``` + +### 8. Jitsi Meet +**Description**: Secure, Simple and Scalable Video Conferences +**Key Features**: +- Multiple real-time services +- WebRTC configuration +- TURN/STUN servers +- Authentication +- Custom networking +- Environment configuration + +```yaml +version: '3.8' +services: + # Frontend + web: + image: jitsi/web:latest + ports: + - '${HTTP_PORT}:80' + - '${HTTPS_PORT}:443' + volumes: + - '${CONFIG}/web:/config' + - '${CONFIG}/web/letsencrypt:/etc/letsencrypt' + - '${CONFIG}/transcripts:/usr/share/jitsi-meet/transcripts' + environment: + - ENABLE_AUTH + - ENABLE_RECORDING + - ENABLE_LETSCENCRYPT + - LETSENCRYPT_DOMAIN + - LETSENCRYPT_EMAIL + - PUBLIC_URL + - XMPP_DOMAIN + - XMPP_AUTH_DOMAIN + - XMPP_GUEST_DOMAIN + - XMPP_MUC_DOMAIN + - XMPP_INTERNAL_MUC_DOMAIN + - XMPP_MODULES + - XMPP_MUC_MODULES + - XMPP_INTERNAL_MUC_MODULES + - JVB_ADDRESS + - ENABLE_COLIBRI_WEBSOCKET + - ENABLE_XMPP_WEBSOCKET + - COLIBRI_REST_ENDPOINT + - ENABLE_LOBBY + - ENABLE_IPV6 + networks: + meet.jitsi: + meet.jitsi.2: + ipv4_address: 172.28.1.1 + restart: unless-stopped + + # XMPP server + prosody: + image: jitsi/prosody:latest + volumes: + - '${CONFIG}/prosody/config:/config' + - '${CONFIG}/prosody/prosody-plugins-custom:/prosody-plugins-custom' + environment: + - AUTH_TYPE + - ENABLE_AUTH + - ENABLE_GUESTS + - GLOBAL_MODULES + - GLOBAL_CONFIG + - LDAP_URL + - LDAP_BASE + - LDAP_BINDDN + - LDAP_BINDPW + - LDAP_FILTER + - LDAP_AUTH_METHOD + - LDAP_VERSION + - LDAP_USE_TLS + - LDAP_TLS_CIPHERS + - LDAP_TLS_CHECK_PEER + - LDAP_TLS_CACERT_FILE + - LDAP_TLS_CACERT_DIR + - LDAP_START_TLS + - XMPP_DOMAIN + - XMPP_AUTH_DOMAIN + - XMPP_GUEST_DOMAIN + - XMPP_MUC_DOMAIN + - XMPP_INTERNAL_MUC_DOMAIN + - XMPP_MODULES + - XMPP_MUC_MODULES + - XMPP_INTERNAL_MUC_MODULES + - XMPP_RECORDER_DOMAIN + - JICOFO_COMPONENT_SECRET + - JICOFO_AUTH_USER + - JVB_AUTH_USER + - JIGASI_XMPP_USER + - JIBRI_XMPP_USER + - JIBRI_RECORDER_USER + - JIBRI_RECORDING_DIR + - JIBRI_FINALIZE_RECORDING_SCRIPT_PATH + - JIBRI_XMPP_PASSWORD + - JIBRI_RECORDER_PASSWORD + - JWT_APP_ID + - JWT_APP_SECRET + - JWT_ACCEPTED_ISSUERS + - JWT_ACCEPTED_AUDIENCES + - JWT_ASAP_KEYSERVER + - JWT_ALLOW_EMPTY + - JWT_AUTH_TYPE + - JWT_TOKEN_AUTH_MODULE + - LOG_LEVEL + - TZ + networks: + meet.jitsi: + meet.jitsi.2: + ipv4_address: 172.28.1.2 + restart: unless-stopped + + # Video bridge + jvb: + image: jitsi/jvb:latest + ports: + - '${JVB_PORT}:${JVB_PORT}/udp' + - '${JVB_TCP_PORT}:${JVB_TCP_PORT}' + volumes: + - '${CONFIG}/jvb:/config' + environment: + - DOCKER_HOST_ADDRESS + - XMPP_AUTH_DOMAIN + - XMPP_INTERNAL_MUC_DOMAIN + - XMPP_SERVER + - JVB_AUTH_USER + - JVB_AUTH_PASSWORD + - JVB_BREWERY_MUC + - JVB_PORT + - JVB_TCP_PORT + - JVB_TCP_ENABLED + - JVB_TCP_PORT + - JVB_STUN_SERVERS + - JVB_ENABLE_APIS + - JVB_WS_DOMAIN + - JVB_WS_SERVER_ID + - JVB_WS_REGION + - PUBLIC_URL + - TZ + depends_on: + - prosody + networks: + meet.jitsi: + meet.jitsi.2: + ipv4_address: 172.28.1.3 + restart: unless-stopped + + # Jicofo - the conference focus component + jicofo: + image: jitsi/jicofo:latest + volumes: + - '${CONFIG}/jicofo:/config' + environment: + - ENABLE_AUTH + - XMPP_DOMAIN + - XMPP_AUTH_DOMAIN + - XMPP_INTERNAL_MUC_DOMAIN + - XMPP_SERVER + - JICOFO_COMPONENT_SECRET + - JICOFO_AUTH_USER + - JICOFO_AUTH_PASSWORD + - JICOFO_RESERVATION_REST_BASE_URL + - JVB_BREWERY_MUC + - JIGASI_BREWERY_MUC + - JIGASI_RECORDER_BREWERY_MUC + - JIBRI_BREWERY_MUC + - JIBRI_PENDING_TIMEOUT + - TZ + depends_on: + - prosody + networks: + meet.jitsi: + meet.jitsi.2: + ipv4_address: 172.28.1.4 + restart: unless-stopped + +networks: + meet.jitsi: + meet.jitsi.2: + driver: bridge + driver_opts: + com.docker.network.bridge.name: meet.jitsi.2 + enable_ipv6: true + ipam: + driver: default + config: + - subnet: 172.28.0.0/16 + gateway: 172.28.0.1 + - subnet: 2001:db8:4001:1::/64 + gateway: 2001:db8:4001::1 +``` + +## Testing Strategy + +1. **Start Simple**: Begin with the basic examples and ensure they work +2. **Progressive Complexity**: Move to more complex examples +3. **Edge Cases**: Test with different configurations and edge cases +4. **Real-world Scenarios**: Test with actual applications like Nextcloud and Plane +5. **Performance Testing**: Measure resource usage and optimize as needed + +## Adding New Examples + +When adding new examples, please: +1. Categorize by difficulty level +2. Include a brief description +3. List key features +4. Ensure the example is complete and runnable +5. Add any necessary environment variables or setup instructions + +## Contributing + +Contributions of additional examples are welcome! Please ensure that any contributed examples: +- Are well-documented +- Follow best practices +- Include all necessary configuration +- Are tested and working diff --git a/docs/PLUGIN_STYLEGUIDE.md b/docs/PLUGIN_STYLEGUIDE.md new file mode 100644 index 0000000..8348237 --- /dev/null +++ b/docs/PLUGIN_STYLEGUIDE.md @@ -0,0 +1,494 @@ +# Dokku Plugin Style Guide + +This document outlines the best practices and conventions for developing high-quality Dokku plugins, based on the patterns used in popular official plugins like dokku-postgres, dokku-redis, dokku-letsencrypt, and others. + +## Plugin Types + +Dokku plugins generally fall into three main categories, each with its own patterns and considerations: + +### 1. Service Plugins (e.g., dokku-postgres, dokku-redis) +- Manage long-running services (databases, caches, etc.) +- Include full lifecycle management (create, destroy, backup, restore) +- Handle data persistence and migrations +- Support linking to applications +- Examples: PostgreSQL, Redis, MySQL, MongoDB + +### 2. Utility Plugins (e.g., dokku-letsencrypt, dokku-http-auth) +- Add specific functionality to Dokku +- Often integrate with external services +- May run on schedules or triggers +- Examples: SSL certificates, authentication, monitoring + +### 3. Core Plugins (e.g., dokku-postgres, dokku-redis) +- Provide fundamental infrastructure services +- Often bundled with Dokku +- Handle core functionality like networking, storage, etc. +- Examples: Network, Storage, Scheduler + +## Table of Contents + +1. [Project Structure](#project-structure) +2. [File Organization](#file-organization) +3. [Code Style](#code-style) +4. [Testing](#testing) +5. [Documentation](#documentation) +6. [Versioning](#versioning) +7. [CI/CD](#cicd) +8. [Common Patterns](#common-patterns) +9. [Best Practices](#best-practices) + +## Project Structure + +Dokku plugins follow a consistent structure that varies slightly based on the plugin type. Here's a comprehensive directory structure that covers all plugin types: + +``` +. +├── .github/ # GitHub-specific files +│ ├── workflows/ # GitHub Actions workflows +│ │ ├── ci.yml # Continuous Integration +│ │ └── release.yml # Release automation +│ └── dependabot.yml # Dependency updates +│ +├── docs/ # Documentation +│ ├── README.md # Main documentation +│ ├── commands/ # Command references +│ └── examples/ # Usage examples +│ +├── scripts/ # Utility scripts (optional) +│ └── setup_*.sh # Setup/configuration scripts +│ +├── subcommands/ # Plugin subcommands (one file per subcommand) +│ ├── create # Service creation +│ ├── destroy # Service teardown +│ ├── backup # Data backup +│ └── ... # Other commands +│ +├── tests/ # Test files +│ ├── unit/ # Unit tests +│ ├── integration/ # Integration tests +│ ├── test_helper.bash # Test utilities +│ └── setup.sh # Test setup +│ +├── templates/ # Configuration templates (optional) +│ └── *.conf.sigil # Sigil templates for configs +│ +├── .editorconfig # Editor configuration +├── .gitignore # Git ignore rules +├── Dockerfile # Development container +├── Makefile # Build and test tasks +├── plugin.toml # Plugin metadata +├── commands # Plugin command definitions +├── common-functions # Shared shell functions +├── help-functions # Help text generation +└── README.md # Project overview +``` + +### Key Files Explained: + +1. **plugin.toml** + - Required for all plugins + - Defines plugin metadata and configuration + - Example: + ```toml + [plugin] + description = "Postgres service plugin" + version = "1.0.0" + + [plugin.config] + # Default configuration values + "docker-options" = "--restart=on-failure:10" + ``` + +2. **commands** + - Defines the main plugin commands + - Maps commands to their implementations + - Example: + ``` + postgres:create postgres-create + postgres:destroy postgres-destroy + ``` + +3. **subcommands/** + - One file per subcommand + - Naming: `plugin-command` (e.g., `postgres-create`) + - Each file should be executable and include help text + +4. **tests/** + - BATS test files + - Mirror the command structure + - Include both unit and integration tests + +## File Organization + +### Core Files + +1. **plugin.toml** + - Contains plugin metadata and configuration + - Defines plugin name, version, and dependencies + - Example: + ```toml + [plugin] + description = "Postgres service plugin" + version = "1.15.0" + [plugin.config] + "docker-options" = "--restart=on-failure:10" + ``` + +2. **Makefile** + - Standard build and test targets + - Common targets: `build`, `test`, `lint`, `install` + - Example: + ```makefile + .PHONY: test + test: + @bats tests/ + ``` + +3. **Dockerfile** + - Defines the development environment + - Includes all necessary build tools and dependencies + +### Subcommands + +- Each subcommand is a separate file in the `subcommands/` directory +- Naming convention: `subcommands/-` +- Example: `subcommands/service-create`, `subcommands/service-logs` + +### Documentation + +- Comprehensive README.md with: + - Installation instructions + - Basic usage examples + - Available commands + - Configuration options + - Troubleshooting + +## Code Style + +### Shell Scripting + +- Use `#!/usr/bin/env bash` for portability +- Follow [Google's Shell Style Guide](https://google.github.io/styleguide/shellguide.html) +- Use `local` for function-scoped variables +- Quote all variables to prevent word splitting +- Use `[[` for tests instead of `[` +- Example: + ```bash + #!/usr/bin/env bash + + set -eo pipefail + + my_function() { + local var1="value1" + local var2="value2" + + if [[ "$var1" == "$var2" ]]; then + echo "Values are equal" + fi + } + ``` + +### Error Handling + +- Use `set -eo pipefail` at the start of scripts +- Provide meaningful error messages +- Use `trap` for cleanup operations +- Example: + ```bash + cleanup() { + rm -f "$TEMP_FILE" + } + trap cleanup EXIT + ``` + +## Testing + +### Test Structure + +- Use [bats-core](https://github.com/bats-core/bats-core) for testing +- Organize tests by functionality +- Follow naming convention: `tests/_test.bats` +- Example test file: + ```bash + #!/usr/bin/env bats + + load 'test_helper/bats-support/load' + load 'test_helper/bats-assert/load' + + setup() { + # Test setup code + } + + teardown() { + # Test teardown code + } + + @test "service create should succeed" { + run dokku postgres:create test-db + assert_success + assert_output --partial "PostgreSQL container created: test-db" + } + ``` + +### Test Coverage + +- Test all subcommands +- Include error cases +- Test edge cases +- Mock external dependencies +- Use test fixtures for complex data + +## Documentation + +### Command Help + +- Include help text for all commands +- Use `dokku :help ` pattern +- Document all options and arguments +- Example: + ```bash + help() { + cat < [options] + + Create a new PostgreSQL service. + + Options: + --config-opt KEY=VALUE Set configuration option + --image IMAGE Use custom Docker image + EOF + } + ``` + +### Man Pages + +- Include man pages for complex commands +- Use `ronn` or similar tool to generate man pages from markdown +- Example: `docs/man/dokku-postgres-create.1.ronn` + +## Versioning + +- Follow [Semantic Versioning](https://semver.org/) +- Update version in `plugin.toml` +- Create GitHub releases with changelog +- Tag releases with `v` prefix (e.g., `v1.0.0`) + +## CI/CD + +### GitHub Actions + +- Run tests on push and pull requests +- Lint shell scripts with ShellCheck +- Build and test on multiple platforms +- Example workflow: + ```yaml + name: CI + on: [push, pull_request] + jobs: + test: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - name: Run tests + run: make test + - name: Lint shell scripts + run: shellcheck -x scripts/* + ``` + +### Release Process + +1. Update version in `plugin.toml` +2. Update `CHANGELOG.md` +3. Create git tag +4. Push tag to trigger release +5. GitHub Actions will publish the release + +## Common Patterns + +### Service Management + +```bash +# Start service +dokku postgres:start my-db + +# Stop service +dokku postgres:stop my-db + +# View logs +dokku postgres:logs my-db +``` + +### Data Management + +```bash +# Create backup +dokku postgres:backup my-db + +# Restore from backup +dokku postgres:restore my-db < backup.dump + +# Export data +dokku postgres:export my-db > data.sql +``` + +### Linking Services + +```bash +# Link service to app +dokku postgres:link my-db my-app + +# Unlink service +dokku postgres:unlink my-db my-app +``` + +## Best Practices + +### Core Principles +1. **Idempotency**: All commands should be idempotent (running them multiple times has the same effect as running them once) +2. **Error Handling**: + - Provide clear, actionable error messages + - Include error codes for programmatic handling + - Clean up resources on failure +3. **Documentation**: + - Document all commands with examples + - Keep README and help text up-to-date + - Include troubleshooting guides + +### Code Quality +4. **Testing**: + - Maintain high test coverage (aim for 80%+) + - Test edge cases and error conditions + - Include both unit and integration tests +5. **Security**: + - Never log sensitive data + - Validate all inputs + - Follow principle of least privilege +6. **Performance**: + - Optimize for speed and resource usage + - Minimize external command calls + - Cache expensive operations when possible + +### Plugin Architecture +7. **Compatibility**: + - Maintain backward compatibility within major versions + - Use feature detection for optional functionality + - Document version requirements +8. **Logging**: + - Use appropriate log levels (DEBUG, INFO, WARN, ERROR) + - Include timestamps and context in logs + - Support structured logging for machine processing +9. **Resource Management**: + - Clean up temporary files and resources + - Handle signals and shutdown gracefully + - Implement timeouts for long-running operations + +### Plugin-Specific Guidelines +10. **Service Plugins**: + - Implement full lifecycle management + - Support backup/restore operations + - Handle data persistence and migrations +11. **Utility Plugins**: + - Focus on single responsibility + - Support dry-run mode + - Include validation for all inputs +12. **Core Plugins**: + - Follow established patterns from official plugins + - Document all configuration options + - Include upgrade paths for existing users + +### Development Workflow +13. **Version Control**: + - Use semantic versioning + - Create meaningful commit messages + - Use feature branches and pull requests +14. **CI/CD**: + - Automate testing and releases + - Run linters and static analysis + - Generate changelogs automatically +15. **Community**: + - Document contribution guidelines + - Include a code of conduct + - Be responsive to issues and PRs + +## Example Plugins + +### Service Plugin: dokku-postgres +- **Repository**: [dokku/dokku-postgres](https://github.com/dokku/dokku-postgres) +- **Key Features**: + - Full database lifecycle management + - Backup/restore functionality + - User and permission management + - SSL configuration +- **Notable Patterns**: + - Well-structured subcommands + - Comprehensive test suite + - Detailed documentation + +### Utility Plugin: dokku-letsencrypt +- **Repository**: [dokku/dokku-letsencrypt](https://github.com/dokku/dokku-letsencrypt) +- **Key Features**: + - Automatic SSL certificate management + - Certificate renewal + - Nginx configuration +- **Notable Patterns**: + - Cron-based scheduling + - Template-based configuration + - Clean error handling + +### Core Plugin: dokku-redis +- **Repository**: [dokku/dokku-redis](https://github.com/dokku/dokku-redis) +- **Key Features**: + - Redis instance management + - Data persistence + - Service linking +- **Notable Patterns**: + - Consistent command structure + - Integration with Dokku networking + - Clear user feedback + +## Plugin Development Checklist + +When developing a new Dokku plugin, ensure you've covered these aspects: + +### Core Functionality +- [ ] Implement all required lifecycle hooks +- [ ] Support all standard commands (create, destroy, info, etc.) +- [ ] Handle errors gracefully +- [ ] Validate all inputs + +### User Experience +- [ ] Provide clear help text for all commands +- [ ] Include usage examples +- [ ] Document configuration options +- [ ] Support common environment variables + +### Testing +- [ ] Write unit tests for core functions +- [ ] Include integration tests +- [ ] Test error conditions +- [ ] Verify plugin uninstallation + +### Documentation +- [ ] Complete README +- [ ] Command reference +- [ ] Installation instructions +- [ ] Upgrade guides + +### Deployment +- [ ] CI/CD pipeline +- [ ] Version management +- [ ] Release process +- [ ] Changelog generation + +## Contributing + +1. Fork the repository +2. Create a feature branch +3. Make your changes +4. Add tests +5. Update documentation +6. Submit a pull request + +## License + +This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. diff --git a/docs/TESTING.md b/docs/TESTING.md new file mode 100644 index 0000000..f7013eb --- /dev/null +++ b/docs/TESTING.md @@ -0,0 +1,565 @@ +# Testing Strategy for Dokku Docker Compose Plugin + +## Test Types + +### 1. Plugin Integration Tests +**Purpose**: Test integration with various Dokku plugins +**Location**: `tests/integration/plugins/` +**What to test**: +- Plugin detection and selection +- Configuration mapping from docker-compose to plugin +- Service linking with plugins +- Fallback behavior when plugins are not available +- Version-specific plugin features + +### 2. Unit Tests +**Purpose**: Test individual functions in isolation +**Tools**: [bats-core](https://github.com/bats-core/bats-core) (Bash Automated Testing System) +**Location**: `tests/unit/` +**What to test**: +- Small, pure functions +- Input validation +- Output formatting +- Error conditions + +Example test file structure: +``` +tests/unit/ + ├── test_helpers.bats + ├── test_validation.bats + └── test_parser.bats +``` + +### 3. Integration Tests +**Purpose**: Test interactions between components +**Tools**: bats-core, Docker-in-Docker (DinD) +**Location**: `tests/integration/` +**What to test**: +- Component interactions +- File system operations +- Command execution +- Docker API interactions + +### 4. End-to-End (E2E) Tests +**Purpose**: Test the plugin in a real Dokku environment +**Tools**: +- [Dokku Test Suite](http://dokku.viewdocs.io/dokku/development/testing/) +- Docker Compose test environments +**Location**: `tests/e2e/` +**What to test**: +- Full plugin commands +- Real Dokku app creation +- Network and volume handling +- Multi-service scenarios + +## Test Environment + +### Plugin-Specific Test Setup +For testing with different Dokku plugins, we'll use a matrix approach: + +```yaml +# .github/workflows/test-plugins.yml +name: Test with Dokku Plugins + +on: [push, pull_request] + +jobs: + test-plugins: + runs-on: ubuntu-latest + strategy: + matrix: + plugin: [postgres, redis, memcached, mysql, mariadb, mongo] + steps: + - uses: actions/checkout@v3 + - name: Set up Dokku + run: | + wget https://raw.githubusercontent.com/dokku/dokku/v0.30.0/bootstrap.sh + sudo DOKKU_TAG=v0.30.0 bash bootstrap.sh + - name: Install ${{ matrix.plugin }} plugin + run: | + sudo dokku plugin:install https://github.com/dokku/dokku-${{ matrix.plugin }}.git ${{ matrix.plugin }} + - name: Run plugin tests + run: | + PLUGIN=${{ matrix.plugin }} bats tests/integration/plugins/ +``` + +### Prerequisites +- Docker +- Docker Compose +- bats-core (`brew install bats-core` on macOS) +- yq (`brew install yq`) +- jq (`brew install jq`) + +### Setup +1. Clone the repository +2. Run `make test-deps` to install test dependencies +3. Run `make test` to execute all tests + +## Test Structure + +### Test Data +Sample Docker Compose files for testing: +``` +testdata/ + ├── simple/ + │ ├── docker-compose.yml + │ └── expected/ + ├── multi-service/ + │ ├── docker-compose.yml + │ └── expected/ + └── edge-cases/ + ├── invalid-yaml.yml + └── unsupported-version.yml +``` + +### Test Helpers +Create reusable test utilities in `tests/test_helper.bash`: +```bash +#!/usr/bin/env bash + +setup() { + load 'test_helper/bats-support/load' + load 'test_helper/bats-assert/load' + + # Set up test environment + TEST_DIR="$(mktemp -d)" + cd "$TEST_DIR" || exit 1 +} + +teardown() { + # Clean up test environment + rm -rf "$TEST_DIR" +} + +# Helper functions... +``` + +## Writing Plugin Integration Tests + +### Example Plugin Test +```bash +@test "import with postgres plugin" { + # Setup + local test_dir="$(mktemp -d)" + cd "$test_dir" || exit 1 + + # Create test compose file + cat > docker-compose.yml </dev/null + rm -rf "$test_dir" +} +``` + +### Testing Plugin Installation Instructions + +```bash +@test "show installation instructions when plugin is missing" { + # Setup + local test_dir="$(mktemp -d)" + cd "$test_dir" || exit 1 + + # Create test compose file + cat > docker-compose.yml </dev/null + rm -rf "$test_dir" +} + +@test "continue with --skip-plugin flag" { + # Setup + local test_dir="$(mktemp -d)" + cd "$test_dir" || exit 1 + + # Create test compose file + cat > docker-compose.yml </dev/null + rm -rf "$test_dir" +} + +### Testing Fallback Behavior +```bash +@test "fallback to container when plugin not available" { + # Setup + local test_dir="$(mktemp -d)" + cd "$test_dir" || exit 1 + + # Create test compose file + cat > docker-compose.yml </dev/null + rm -rf "$test_dir" +} +``` + +## Writing Tests + +### Example Unit Tests for Plugin Detection + +```bash +@test "get_plugin_installation_command returns correct command" { + # Test with postgres + run get_plugin_installation_command postgres + assert_success + assert_output "dokku plugin:install https://github.com/dokku/dokku-postgres.git postgres" + + # Test with redis + run get_plugin_installation_command redis + assert_success + assert_output "dokku plugin:install https://github.com/dokku/dokku-redis.git redis" +} + +@test "check_plugin_installed detects missing plugin" { + # Mock dokku command for missing plugin + dokku() { + if [[ "$1" == "plugin:list" ]]; then + echo " 00_dokku-standard 0.30.0 enabled dokku core standard plugin" + return 0 + fi + return 1 + } + export -f dokku + + run check_plugin_installed postgres + assert_failure + assert_output --partial "postgres plugin is not installed" +} + +@test "generate_skip_plugins_option generates correct flags" { + # Test single plugin + run generate_skip_plugins_option "postgres" + assert_success + assert_output "--skip-plugin=postgres" + + # Test multiple plugins + run generate_skip_plugins_option "postgres redis" + assert_success + assert_output "--skip-plugin=postgres --skip-plugin=redis" +} +``` + +### Example Plugin Unit Test +```bash +@test "detect_postgres_plugin" { + # Mock dokku command + dokku() { + if [[ "$1" == "plugin:list" ]]; then + echo " 00_dokku-standard 0.30.0 enabled dokku core standard plugin" + echo " postgres 1.15.0 enabled dokku postgres service plugin" + return 0 + fi + return 1 + } + export -f dokku + + # Run test + run detect_plugin 'postgres:13' + + # Assert + assert_success + assert_output "postgres" +} + +@test "map_postgres_config" { + # Setup test data + local service_config=$(cat < docker-compose.yml <- + --privileged + --network=host + + steps: + - uses: actions/checkout@v3 + + - name: Set up Docker + run: | + docker --version + docker-compose --version + + - name: Install dependencies + run: | + sudo apt-get update + sudo apt-get install -y bats jq + sudo pip install yq + + - name: Run unit tests + run: make test-unit + + - name: Run integration tests + run: make test-integration + + - name: Run e2e tests + run: make test-e2e +``` + +## Test Coverage + +### Coverage Reporting +Use `kcov` for code coverage: +```bash +# Install kcov +brew install kcov + +# Run tests with coverage +make coverage +``` + +### Coverage Reports +- HTML reports in `coverage/` +- Publish to codecov.io or similar service + +## Best Practices + +1. **Test Isolation**: Each test should be independent +2. **Descriptive Names**: Use clear, descriptive test names +3. **Minimal Fixtures**: Keep test data minimal and focused +4. **Test Edge Cases**: Include tests for error conditions +5. **Fast Feedback**: Keep tests fast for quick iteration + +## Debugging Tests + +Run a single test file: +```bash +bats tests/unit/test_parser.bats +``` + +Run with debug output: +```bash +bats --tap tests/ +``` + +## Performance Testing + +For large compose files, add performance benchmarks: +```bash +@test "import handles large compose files quickly" { + # Generate large compose file + # ... + + # Time the import + run time dokku docker-compose:import large-compose.yml + + # Assert it completes within time limit + assert_success + assert_output --regexp 'real\s+0:[0-4]\d\.[0-9]s' # Less than 5 seconds +} +``` + +## Mocking + +For testing without real Dokku/Docker: + +1. Create mocks in `tests/mocks/` +2. Override commands in test setup: + ```bash + dokku() { + echo "[MOCK] dokku $*" >> "$TEST_MOCK_LOG" + case "$1" in + apps:exists) + return 1 # App doesn't exist + ;; + --version) + echo "0.30.0" + ;; + *) + echo "Unexpected dokku command: $*" + return 1 + ;; + esac + } + export -f dokku + ``` + +## Test Maintenance + +1. Update tests when adding new features +2. Review test failures carefully +3. Keep test data up-to-date +4. Document test cases in pull requests diff --git a/docs/requirements.md b/docs/requirements.md new file mode 100644 index 0000000..434721c --- /dev/null +++ b/docs/requirements.md @@ -0,0 +1,368 @@ +# Dokku Docker Compose Plugin Requirements + +## Overview +This document defines the requirements for a Dokku plugin that imports `docker-compose.yml` files and creates corresponding Dokku applications for each service. The plugin will support both simple deployments and complex multi-service applications with dependencies. + +### Definitions +- **Compose File**: A YAML file (typically `docker-compose.yml`) defining services, networks, and volumes +- **Service**: A single container definition within the compose file +- **Dokku App**: A deployed application instance in Dokku +- **Plugin**: The `dokku-docker-compose` plugin being specified +- **Resource Limits**: CPU, memory, and other constraints applied to containers + +## Core Functionality + +### 1. Plugin Initialization +- [ ] Create a new Dokku plugin named `docker-compose` +- [ ] Register plugin commands under `dokku docker-compose:*` +- [ ] Implement basic help and version information + +### 2. Docker Compose File Parsing +- [ ] Support reading and parsing `docker-compose.yml` (and `docker-compose.yaml`) +- [ ] Support both Compose v2 and v3 file formats +- [ ] Validate required fields in the compose file +- [ ] Handle YAML anchors and references + +### 3. Service to Dokku App Mapping +- [ ] For each service in the compose file, create a corresponding Dokku app with these transformations: + - Convert service names to lowercase + - Replace underscores (`_`) with hyphens (`-`) + - Remove any other invalid characters (anything not matching `[a-z0-9-]`) + - Truncate to 30 characters while preserving uniqueness +- [ ] Support custom app name prefixes/suffixes via command-line flags: + - `--prefix `: Add prefix to all generated app names + - `--suffix `: Add suffix to all generated app names + - `--separator `: Character to separate prefix/suffix from name (default: `-`) +- [ ] Ensure generated app names are unique within the Dokku instance + +### 4. Resource Configuration +- [ ] Map Docker Compose resources to Dokku equivalents with the following specifications: + - **CPU/Memory Limits**: + - Convert `deploy.resources.limits.cpus` to Dokku CPU shares + - Convert `deploy.resources.limits.memory` to Dokku memory limits + - Support both v2 (`mem_limit`) and v3 (`deploy.resources`) syntax + - **Environment Variables**: + - Transfer all `environment` and `env_file` entries to Dokku config + - Support variable substitution in environment values + - Preserve the order of variable definitions + - **Volumes**: + - Map `volumes` to Dokku persistent storage + - Support both named volumes and host paths + - Convert volume permissions (e.g., `:ro`, `:rw`) + - **Networks**: + - Create Dokku networks matching compose networks + - Handle `aliases` and `ipv4_address`/`ipv6_address` + - Support custom network drivers and options + - **Ports**: + - Map `ports` to Dokku proxy configuration + - Support both short (`"8080:80"`) and long syntax + - Handle protocol specification (tcp/udp) + - **Health Checks**: + - Convert `healthcheck` to Dokku CHECKS + - Support all healthcheck parameters (test, interval, timeout, retries) + - **Restart Policies**: + - Map `restart` to Dokku process management + - Support all restart policies (no, always, on-failure, unless-stopped) + +### 5. Build and Deployment +- [ ] Support building from Dockerfile if specified +- [ ] Support building from image if specified +- [ ] Handle build arguments and context paths +- [ ] Support multi-stage builds + +### 6. Service Dependencies +- [ ] Detect and handle service dependencies (depends_on) with the following behaviors: + +#### Create Operation +- [ ] **Dependency Resolution**: + - Build a dependency graph from `depends_on` and service links + - Detect and report circular dependencies + - Support all `depends_on` conditions: `service_started`, `service_healthy`, `service_completed_successfully` + +- [ ] **Execution Order**: + - Process services in dependency order (dependencies first) + - Parallelize independent services when possible + - Respect `depends_on` conditions before starting dependent services + +- [ ] **Error Handling**: + - If a dependency fails to start, stop and clean up dependent services + - Provide clear error messages about dependency failures + - Support `--ignore-dependency-errors` flag to continue on dependency failures + +#### Update Operation +- [ ] **Dependency Analysis**: + - Compare current and desired states + - Identify services that need updating based on configuration changes + - Determine if dependency changes require recreation of dependent services + +- [ ] **Update Strategy**: + - Default to rolling updates (one service at a time) + - Support `--parallel` flag for concurrent updates of independent services + - Preserve data volumes unless explicitly configured otherwise + +- [ ] **Rollback**: + - Maintain previous service versions for rollback + - Support `--rollback` flag to revert to previous state + - Automatically roll back if health checks fail after update + +#### Destroy Operation +- [ ] **Dependency Handling**: + - Process services in reverse dependency order (dependents first) + - Prevent destruction of services that are dependencies of other services unless `--force` is specified + - Support `--cascade` flag to automatically destroy all dependents + +- [ ] **Data Management**: + - By default, preserve volumes to prevent data loss + - Support `--volumes` flag to remove associated volumes + - Prompt for confirmation before destroying services with persistent data + +- [ ] **Cleanup**: + - Remove all service links and dependencies + - Clean up any temporary resources + - Remove network configurations if no longer referenced + +### 7. Network Configuration +- [ ] Create necessary Dokku networks +- [ ] Map Docker Compose networks to Dokku networks +- [ ] Handle custom network configurations + +### 8. Volume Management +- [ ] Map Docker Compose volumes to Dokku volumes +- [ ] Support named volumes and bind mounts +- [ ] Handle volume permissions + +### 9. Environment Variables +- [ ] Transfer environment variables from compose file to Dokku config +- [ ] Handle environment file includes +- [ ] Support variable substitution + +### 10. Health Checks +- [ ] Convert Docker Compose healthcheck to Dokku checks +- [ ] Support custom health check configurations + +### 11. Dokku Plugin Integration +- [ ] Detect and utilize existing Dokku plugins for common services with the following behavior: + - **Plugin Detection**: + - Match service image names against known plugin patterns (e.g., `postgres:*` → `dokku-postgres`) + - Support custom plugin mappings via configuration + - **Installation Handling**: + - Check for required plugins using `dokku plugin:installed` + - If missing, provide exact installation command: + ``` + dokku plugin:install https://github.com/dokku/dokku-.git + ``` + - Support `--skip-plugin=` to bypass installation + - **Service Configuration**: + - Map environment variables to plugin-specific configuration + - Handle version specifications from image tags + - Support plugin-specific parameters via labels or environment variables + - **Service Linking**: + - Automatically link services using `dokku :link` + - Set appropriate environment variables in the target app + - **Supported Plugins**: + - **Databases**: + - [ ] postgres (`postgres:*`) + - [ ] mysql (`mysql:*`) + - [ ] mariadb (`mariadb:*`) + - [ ] mongo (`mongo:*`) + - [ ] redis (`redis:*`) + - **Caches**: + - [ ] redis (`redis:*`) + - [ ] memcached (`memcached:*`) + - **Search**: + - [ ] elasticsearch (`elasticsearch:*`) + - **Message Brokers**: + - [ ] rabbitmq (`rabbitmq:*`) + - **Object Storage**: + - [ ] minio (`minio/minio:*`) + - **Key-Value Stores**: + - [ ] etcd (`etcd:*`) + +### 12. Logging and Monitoring +- [ ] Provide detailed logging of the import process +- [ ] Support dry-run mode +- [ ] Generate a summary report after import + +## Non-Functional Requirements + +### 1. Performance +- [ ] Handle compose files with up to 50 services +- [ ] Process each service in parallel where possible +- [ ] Provide real-time progress feedback including: + - Current operation + - Progress percentage + - Estimated time remaining + - Resource usage +- [ ] Complete processing of a 10-service compose file in under 2 minutes on average hardware + +### 2. Error Handling +- [ ] Comprehensive error messages +- [ ] Graceful rollback on failure +- [ ] Input validation + +### 3. Security +- [ ] Handle sensitive information appropriately +- [ ] Validate input to prevent injection attacks +- [ ] Follow Dokku's security best practices + +### 4. Documentation +- [ ] Comprehensive CLI documentation covering: + - All commands and options + - Environment variables + - Configuration files + - Exit codes and their meanings +- [ ] Examples for: + - Basic single-service deployment + - Multi-service application with dependencies + - Custom resource allocation + - Plugin integration examples +- [ ] Troubleshooting guide covering: + - Common error messages and resolutions + - Debugging techniques + - Log file locations + - Recovery procedures + +## Update Process + +### 1. Change Detection +- [ ] **Configuration Comparison**: + - Compare current and desired states for each service + - Track changes in: + - Image versions + - Environment variables + - Resource limits + - Volume mounts + - Network configurations + - Health checks + - Dependencies +- [ ] **Change Classification**: + - **No Change**: No action needed + - **Configuration Change**: Update service configuration + - **Image Change**: Rebuild/redeploy service + - **Dependency Change**: May require recreation of dependent services + - **Breaking Change**: Requires explicit confirmation or flag + +### 2. Update Strategies +- [ ] **Default Strategy (Rolling Update)**: + - Update one service at a time + - Wait for health checks to pass before proceeding + - Automatic rollback on failure + - Minimal downtime + +- [ ] **Parallel Update**: + - Update multiple independent services concurrently + - Controlled by `--parallel` flag with optional concurrency limit + - Example: `--parallel=3` for up to 3 concurrent updates + +- [ ] **Recreate Strategy**: + - Stop old containers before starting new ones + - Triggered by `--recreate` flag + - Useful for changes requiring complete service restart + +### 3. State Management +- [ ] **State Storage**: + - Store original docker-compose.yml in Dokku app config + - Track checksums of configuration files + - Maintain version history of applied configurations + +- [ ] **Dry Run**: + - Support `--dry-run` flag to preview changes + - Show what would be updated without making changes + - Include change summary and impact analysis + +### 4. Handling Specific Changes +- [ ] **Image Updates**: + - Pull new images + - Recreate containers with new images + - Support `--pull` flag to force image refresh + +- [ ] **Configuration Changes**: + - Update environment variables + - Adjust resource limits + - Apply new health check configurations + - Restart services if needed + +- [ ] **Volume Changes**: + - Preserve existing volumes by default + - Support `--recreate-volumes` for volume recreation + - Handle volume permission changes + +- [ ] **Network Changes**: + - Update network configurations + - Handle IP address changes + - Manage network aliases + +### 5. Rollback Mechanism +- [ ] **Automatic Rollback**: + - Trigger on health check failures + - Configurable timeout for health checks + - Limit number of rollback attempts + +- [ ] **Manual Rollback**: + - Support `--rollback` flag to revert to previous version + - Show rollback preview before executing + - Support rollback to specific version + +### 6. Dependency Handling +- [ ] **Dependency Analysis**: + - Rebuild dependency graph + - Identify services affected by changes + - Determine update order based on dependencies + +- [ ] **Dependent Updates**: + - Update dependent services when needed + - Support `--no-dependents` to skip dependent updates + - Show impact analysis before proceeding + +## Future Enhancements + +### 1. Export Functionality +- [ ] Export existing Dokku apps to docker-compose format + +### 2. Update Support +- [ ] Update existing Dokku apps from modified compose files + +### 3. Plugin System +- [ ] Allow extensions for custom service types +- [ ] Support for Docker Compose extensions + +### 4. Integration Tests +- [ ] Test suite with various docker-compose configurations +- [ ] CI/CD pipeline + +## Dependencies + +### Required +- Docker +- Docker Compose +- Dokku +- Bash (for the plugin) +- yq (for YAML parsing in bash) + +### Optional +- jq (for advanced JSON processing) +- parallel (for faster processing) + +## Implementation Notes +- The plugin should be written in Bash to match Dokku's existing plugin system +- Use temporary directories for build contexts +- Implement proper cleanup of temporary resources +- Follow Dokku's plugin development guidelines +- Include comprehensive logging for debugging + +## Usage Examples + +### Basic Import +```bash +dokku docker-compose:import /path/to/docker-compose.yml +``` + +### Import with Custom Prefix +```bash +dokku docker-compose:import --prefix myapp- /path/to/docker-compose.yml +``` + +### Dry Run +```bash +dokku docker-compose:import --dry-run /path/to/docker-compose.yml +``` diff --git a/docs/todo.md b/docs/todo.md new file mode 100644 index 0000000..2f9ebed --- /dev/null +++ b/docs/todo.md @@ -0,0 +1,174 @@ +# Dokku Docker Compose Plugin - Implementation Plan + +## Phase 1: Project Setup +- [ ] Initialize Git repository +- [ ] Set up project structure +- [ ] Create basic plugin files + - [ ] `plugin.toml` - Plugin metadata + - [ ] `commands` - Main plugin commands + - [ ] `functions` - Core functions + - [ ] `tests` - Test directory +- [ ] Add README.md with basic usage + +## Phase 2: Core Functionality + +### 2.1 Basic Plugin Structure +- [ ] Implement plugin initialization +- [ ] Create help command +- [ ] Add version information +- [ ] Set up logging system + +### 2.2 Docker Compose Parser +- [ ] Add YAML parsing with `yq` +- [ ] Implement validation for compose file +- [ ] Support for both v2 and v3 compose formats +- [ ] Handle YAML anchors and references + +### 2.3 Dokku Plugin Integration +- [ ] Create service detection system for Dokku plugins +- [ ] Implement plugin installation check +- [ ] Add support for core plugins (postgres, redis, etc.) +- [ ] Map Docker Compose config to Dokku plugin config +- [ ] Handle service linking with plugins +- [ ] Implement fallback to container when plugin is not available + +### 2.4 Dokku App Creation +- [ ] Create function to generate valid Dokku app names +- [ ] Implement basic app creation +- [ ] Add error handling for existing apps +- [ ] Implement dry-run mode + +## Phase 3: Service Configuration + +### 3.1 Plugin Service Configuration +- [ ] Implement plugin-specific configuration mapping +- [ ] Handle plugin version specifications +- [ ] Configure resource limits for plugin services +- [ ] Set up backup policies for data services +- [ ] Implement plugin service health checks + +### 3.2 Basic Container Service Configuration +- [ ] Map service images to Dokku apps +- [ ] Handle environment variables +- [ ] Configure container resources (CPU, memory) +- [ ] Set up build context for Dockerfile builds + +### 3.3 Networking +- [ ] Create and configure Dokku networks +- [ ] Map container ports +- [ ] Handle service links and depends_on +- [ ] Configure DNS settings + +### 3.3 Storage +- [ ] Create and map volumes +- [ ] Handle bind mounts +- [ ] Configure volume permissions + +## Phase 4: Plugin-Specific Features + +### 4.1 Database Plugins +- [ ] Handle database initialization +- [ ] Support for database extensions +- [ ] Implement backup/restore functionality +- [ ] Support for read replicas + +### 4.2 Cache Plugins +- [ ] Configure memory limits +- [ ] Set up eviction policies +- [ ] Implement persistence options + +## Phase 5: Advanced Features + +### 4.1 Build System +- [ ] Support Dockerfile builds +- [ ] Handle build arguments +- [ ] Support multi-stage builds +- [ ] Implement build caching + +### 4.2 Health Checks +- [ ] Convert Docker healthchecks to Dokku checks +- [ ] Configure liveness/readiness probes +- [ ] Implement custom health check commands + +### 4.3 Logging and Monitoring +- [ ] Set up log aggregation +- [ ] Configure monitoring +- [ ] Implement log rotation + +## Phase 6: Testing and Documentation + +### 6.1 Testing +- [ ] Unit tests for core functions +- [ ] Integration tests with sample compose files +- [ ] Test error conditions +- [ ] Performance testing + +### 6.2 Documentation +- [ ] Complete CLI documentation +- [ ] Examples directory +- [ ] Troubleshooting guide +- [ ] Contribution guidelines + +## Phase 7: Polish and Release + +### 7.1 Error Handling +- [ ] Comprehensive error messages +- [ ] Graceful rollback on failure +- [ ] Input validation + +### 7.2 Security +- [ ] Secure handling of sensitive data +- [ ] Input sanitization +- [ ] Permission checks + +### 7.3 Performance +- [ ] Optimize for large compose files +- [ ] Add parallel processing where possible +- [ ] Implement progress indicators + +## Phase 8: Future Enhancements (Post-MVP) + +### 8.1 Export Functionality +- [ ] Export Dokku apps to compose format +- [ ] Support for app updates + +### 8.2 Plugin System +- [ ] Extension points for custom service types +- [ ] Support for Compose extensions + +### 8.3 CI/CD Integration +- [ ] GitHub Actions workflow +- [ ] Automated testing +- [ ] Release automation + +## Getting Started + +### Prerequisites +- Docker +- Docker Compose +- Dokku +- yq (YAML processor) +- Bash 4.0+ + +### Development Setup +```bash +# Clone the repository +git clone +cd dokku-docker-compose + +# Install development dependencies +make setup + +# Run tests +make test +``` + +## Contributing +1. Fork the repository +2. Create a feature branch +3. Make your changes +4. Add tests +5. Submit a pull request + +## License +[Specify License]