Server Manager - Docker + Traefik Infrastructure Toolkit

Technology Stack

Purpose and Philosophy

The Server Manager project is a comprehensive infrastructure toolkit designed to simplify the deployment and management of containerized applications on a production server. The core philosophy centers on making server management boring to maintain and exciting to extend by eliminating manual certificate management, port configuration, and repetitive deployment tasks.

At its heart, this project embraces the infrastructure-as-code approach where all server configurations are version-controlled, documented, and reproducible. The system leverages Traefik v3 as an intelligent reverse proxy that automatically discovers Docker containers and configures routing based on container labels, eliminating the need for manual nginx configuration or complex load balancer setups.

Core Design Principles

The project follows several fundamental principles that ensure reliability, security, and maintainability:

1. Zero-Touch SSL: All applications automatically receive SSL certificates through Let's Encrypt's HTTP-01 challenge without manual intervention. When a new container starts with proper Traefik labels, the proxy automatically requests and provisions certificates.

1. Configuration-Driven Deployments: The project uses a template-based approach where deploying a new application requires only editing a .env file. The docker-compose.yml files remain generic and reusable across different applications, reducing configuration drift and human error.

1. Network Isolation: Applications run on isolated Docker networks. The proxy network connects containers to Traefik for external access, while internal networks (like internal for databases) ensure sensitive services remain inaccessible from the outside world.

1. Secret Management: The project implements a two-branch Git strategy separating public code from secrets. The main branch contains only configuration templates and application code, while the private branch tracks actual environment variables and credentials. A pre-push hook prevents accidental secret leaks.

1. Documentation-First: Every component, from the Traefik setup to individual application templates, includes comprehensive documentation explaining the what, why, and how. The documentation serves as both a user guide and a reference for future maintenance.

Architecture Deep Dive

The Traefik Foundation

Traefik serves as the centerpiece of the infrastructure, functioning as both a reverse proxy and a dynamic service discovery mechanism. Unlike traditional nginx configurations that require manual editing and service restarts, Traefik watches the Docker daemon for new containers and automatically configures routes based on labels.

# traefik-setup/traefik.yml - Core configuration
api:
  dashboard: true

entryPoints:
  web:
    address: ":80"
    http:
      redirections:
        entryPoint:
          to: websecure
          scheme: https

  websecure:
    address: ":443"

certificatesResolvers:
  myresolver:
    acme:
      email: [email protected]
      storage: acme.json
      httpChallenge:
        entryPoint: web

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false

The configuration defines two entry points: web for HTTP (which automatically redirects to HTTPS) and websecure for HTTPS. The exposedByDefault: false setting is a critical security feature that prevents containers from being accidentally exposed to the internet unless explicitly configured with traefik.enable=true.

Traffic Flow Architecture

The complete request flow demonstrates how requests traverse from end users to containerized applications:

User's Browser
       │
       ▼
Cloudflare (DNS Resolution + SSL Termination)
       │
       ▼
Server IP: 203.0.113.30
       │
       ▼
Port 80 (HTTP) → Traefik → Auto-redirect to HTTPS
Port 443 (HTTPS) → Traefik → SSL Termination
       │
       ▼
Traefik inspects Host header
       │
       ├── tf.namani.in → Dashboard (api@internal)
       ├── blog.namani.in → Ghost Container (:2368)
       ├── c.aatmanova.in → WordPress Container (:80)
       └── [any].namani.in → Respective Container

This architecture enables hosting multiple applications on a single server with different domains, each receiving automatic SSL without additional configuration. Cloudflare provides an additional layer of DDoS protection and CDN caching for static content.

Application Templates

The project provides reusable templates for common application types:

#### Single-Container Template (blog-example)

This template works with any single-container Docker application:

# applications/blog-example/docker-compose.yml
services:
  app:
    image: ${APP_IMAGE}
    container_name: ${CONTAINER_NAME}
    restart: always
    networks:
      - proxy
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.${ROUTER_NAME}.rule=Host(`${APP_DOMAIN}`)"
      - "traefik.http.routers.${ROUTER_NAME}.entrypoints=websecure"
      - "traefik.http.routers.${ROUTER_NAME}.tls.certresolver=myresolver"
      - "traefik.http.services.${ROUTER_NAME}.loadbalancer.server.port=${APP_PORT}"

networks:
  proxy:
    external: true

Users only edit the .env file to configure:

• APP_DOMAIN: The domain for the application
• ROUTER_NAME: Unique identifier for Traefik routing
• APP_IMAGE: Docker image to run
• APP_PORT: Internal container port
• CONTAINER_NAME: Friendly name for docker ps

#### Multi-Container Template (WordPress)

The WordPress template demonstrates handling database-backed applications:

# applications/wordpress/docker-compose.yml
services:
  app:
    image: ${WP_IMAGE}
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: ${DB_USER}
      WORDPRESS_DB_PASSWORD: ${DB_PASSWORD}
    networks:
      - proxy
      - internal
    labels:
      - "traefik.enable=true"
      # ... Traefik labels

  db:
    image: ${DB_IMAGE}
    environment:
      MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD}
      MYSQL_DATABASE: ${DB_NAME}
    networks:
      - internal  # Only accessible internally

networks:
  proxy:
    external: true
  internal:
    name: ${INTERNAL_NETWORK_NAME}

This architecture ensures the database container is completely isolated from external access while WordPress connects to both networks to serve traffic and communicate with the database.

Key Features and Functionality

1. Automatic SSL Certificate Management

The system uses Let's Encrypt's automated certificate authority to provision free SSL certificates:

1. Container starts with tls.certresolver=myresolver label
2. Traefik detects the new domain from Host() rule
3. Let's Encrypt validates domain ownership via HTTP-01 challenge on port 80
4. Certificate is issued and stored in acme.json
5. HTTPS becomes available automatically

This happens entirely automatically with no manual intervention. Certificates are automatically renewed before expiration, and the system gracefully handles multiple domains.

2. Traefik Dashboard with Basic Auth

The Traefik dashboard provides real-time insights into routing, services, and middleware:

# Dashboard routing with Basic Auth
labels:
  - "traefik.http.routers.dashboard.rule=Host(`tf.namani.in`)"
  - "traefik.http.routers.dashboard.entrypoints=websecure"
  - "traefik.http.routers.dashboard.service=api@internal"
  - "traefik.http.routers.dashboard.middlewares=auth"
  - "traefik.http.routers.dashboard.tls.certresolver=myresolver"
  - "traefik.http.middlewares.auth.basicauth.users=${DASHBOARD_AUTH}"

The dashboard is protected by HTTP Basic Authentication using htpasswd-formatted credentials. The password hash is stored in .env with doubled dollar signs ($$) to escape Docker Compose's variable interpolation.

3. Network Architecture

The shared proxy network enables Traefik to communicate with all containers:

# Create the network once
docker network create proxy

All applications connect to this external network, allowing Traefik to discover and route to them. Internal networks like internal provide additional isolation for databases and backend services.

4. Environment-Driven Configuration

The entire setup is driven by environment variables:

# traefik-setup/.env
TRAEFIK_DOMAIN=tf.namani.in
[email protected]
DASHBOARD_AUTH=admin:$$apr1$$VDlqiA.Q$$HZsnUXm2oIU1ZXZQ8enzb0
DOCKER_API_VERSION=1.44

This approach allows the same docker-compose.yml to be reused across different environments by simply changing the .env file.

5. Production Server Information

The project manages a live production server:

6. Git-Based Secret Management

The project implements a sophisticated branching strategy for security:

• `main` branch (public): Contains only non-sensitive configuration templates and application code. Safe to push to GitHub.
• `private` branch (local): Tracks actual .env files with real credentials. Never pushed to remote.

A pre-push Git hook provides automated protection:

• Blocks attempts to push the private branch
• Scans commits for forbidden file patterns (.env, .key, .pem, id_rsa)
• Provides manual verification step before any push

7. Troubleshooting Infrastructure

The documentation includes a comprehensive troubleshooting reference:

8. Homelab Compatibility

The documentation thoroughly covers homelab scenarios:

VPS with Public IP: Works exactly as configured. Direct access to ports 80/443 enables Let's Encrypt HTTP-01 challenge.

Homelab with Port Forwarding: Works identically to VPS. Forward ports 80/443 from router to the server.

Homelab behind CGNAT: Does NOT work with standard setup. Let's Encrypt cannot reach the HTTP challenge endpoint.

Homelab with Cloudflare Tunnel: Works as an alternative. The tunnel creates an outbound connection to Cloudflare, bypassing the need for incoming port access.

The documentation provides a detailed comparison between Cloudflare Tunnel and Tailscale for homelab scenarios, concluding that both can run simultaneously with zero interference.

Security Implementation

Network Isolation

Applications follow the principle of least privilege:

• Web-facing containers connect to proxy network
• Databases connect only to isolated internal networks
• Traefik has read-only access to Docker socket (/var/run/docker.sock:ro)

SSL/TLS Security

• All traffic automatically upgrades from HTTP to HTTPS
• Let's Encrypt certificates provide industry-standard encryption
• Cloudflare SSL mode set to "Full" or "Full (Strict)"
• Certificate storage file (acme.json) has restrictive permissions (600)

Dashboard Protection

• Traefik dashboard hidden behind Basic Auth
• Credentials stored as htpasswd hash, not plaintext
• Separate user from application users
• Access limited to specific domain

Secret Handling

• No secrets in version control
• Environment-specific .env files
• Branch-based secret isolation
• Automated pre-push validation

Deployment and Operations

Quick Start Process

Deploying a new application follows a simple workflow:

# 1. Copy the template
cp -r applications/blog-example /opt/my-blog

# 2. Edit ONLY the .env file
nano /opt/my-blog/.env

# 3. Upload to server and start
scp -r /opt/my-blog [email protected]:/opt/
ssh [email protected] "cd /opt/my-blog && docker compose up -d"

Traefik automatically discovers the new container within seconds and begins routing traffic.

Common Deployment Targets

The template supports numerous applications:

Maintenance Operations

Regular server maintenance includes:

# View Traefik logs
docker logs -f traefik

# Restart after config changes
cd /opt/traefik
docker compose up -d --force-recreate

# Check container status
docker ps

# Inspect network
docker network inspect proxy

# View certificates
cat acme.json | python3 -m json.tool

Documentation Strategy

The project maintains comprehensive documentation across multiple files:

• `README.md`: Project overview, quick start guide, structure explanation
• `traefik-setup/Documentation.md`: Complete Traefik technical reference with 850+ lines
• `traefik-setup/info.md`: Server credentials and connection details
• `applications/*/README.md`: Per-application deployment instructions
• `ai-rules.md`: Guidelines for AI agents working on the repository

This documentation-first approach ensures the infrastructure remains maintainable over time, even as the original author returns after extended periods.

Lessons Learned and Best Practices

What Works Well

1. Template-Driven Deployments: Editing only .env files dramatically reduces configuration errors and deployment time.

1. Comprehensive Documentation: The 850+ line Traefik documentation proves invaluable for troubleshooting and onboarding.

1. Secret Branching Strategy: The two-branch approach provides strong protection against accidental secret exposure while maintaining version control benefits.

1. Network Isolation: Separate proxy and internal networks provide defense-in-depth for database containers.

1. Auto-Discovery: Traefik's container label-based discovery eliminates the need for manual routing configuration.

Areas for Enhancement

1. Monitoring Integration: Could benefit from centralized logging (Loki/Graylog) and metrics (Prometheus/Grafana).

1. Backup Automation: Database backups could be automated with scheduled jobs rather than manual intervention.

1. Container Updates: No automated update mechanism for base images. Currently requires manual docker compose pull.

1. Multiple Environments: The system manages production directly. Development/staging environments would improve reliability.

1. Load Balancing: Currently single-server. Could scale horizontally with Docker Swarm or Kubernetes.

Future Expansion

The infrastructure is designed to support growth in several directions:

• Orchestration: Could migrate to Docker Swarm or Kubernetes for multi-node clustering
• Service Mesh: Could add Istio or Linkerd for advanced traffic management
• GitOps: Could implement ArgoCD or Flux for declarative infrastructure
• Monitoring: Could deploy Prometheus stack for comprehensive observability
• Container Registry: Could set up private registry for custom images

Conclusion

The Server Manager project represents a mature, production-ready infrastructure approach that balances automation with control. By leveraging Traefik's dynamic configuration and Docker's containerization, it achieves the goal of making infrastructure boring to maintain and exciting to extend.

The comprehensive documentation, security-conscious branching strategy, and template-driven deployment model make this a robust foundation for any self-hosted application portfolio. Whether running a simple blog or complex multi-service architecture, this infrastructure toolkit provides the reliability and flexibility needed for modern production workloads.

The project demonstrates that effective server management doesn't require complex tooling—it requires thoughtful design, clear documentation, and consistent patterns that can be understood and maintained over time.

Operational Procedures and Best Practices

Adding New Applications to the Infrastructure

When deploying a new application to the server, follow this systematic approach to ensure consistency and maintainability:

1. Select the appropriate template based on the application architecture. Use the single-container template for stateless applications like Nginx, Ghost, or Grafana. Use the multi-container template for applications requiring databases like WordPress or custom backends.

1. Configure the environment file with all required variables including the domain name, router identifier, container name, image reference, and internal port. Ensure the router name is unique across all deployed applications to prevent routing conflicts.

1. Verify network configuration by ensuring the application connects to the proxy network for external access and any additional internal networks for service-to-service communication.

1. Test locally if possible before deploying to production. Use docker compose up -d in development mode with verbose logging to verify the configuration.

1. Deploy to production using SCP to transfer files to the server, then execute the container start command. Monitor Traefik logs to confirm successful discovery and routing configuration.

1. Verify SSL certificate issuance by checking the acme.json file or observing the Traefik dashboard. SSL should provision automatically within seconds of container startup.

Monitoring and Alerting

The infrastructure provides several monitoring capabilities through the Traefik dashboard:

• Router Status: View all active routes and their health
• Service Metrics: Check load balancer distribution and response times
• Middleware Performance: Analyze rate limiting and authentication overhead
• Certificate Expiration: Monitor SSL certificate renewal status

For production deployments, consider integrating external monitoring solutions like Uptime Kuma (deployed via this very infrastructure) to provide external availability checks and notification mechanisms beyond the internal Traefik metrics.

Backup and Recovery

Database containers in this infrastructure use named Docker volumes:

volumes:
  - db_data:/var/lib/mysql

To back up database data, use Docker volume backup procedures:

# Create a backup container
docker run --rm -v mydb_data:/data -v $(pwd):/backup alpine tar czf /backup/backup.tar.gz -C /data .

For the configuration itself, maintain regular Git commits with clear commit messages describing infrastructure changes. The branching strategy ensures production configurations are preserved in the private branch while development iterations remain in the main branch.

Performance Optimization

Several configuration options can improve application performance:

• Cloudflare Caching: Enable Cloudflare's CDN for static content by setting appropriate Page Rules
• Container Resource Limits: Add resource constraints to prevent any single container from consuming excessive CPU or memory
• Connection Pooling: For database-backed applications, configure appropriate connection pool sizes
• Static Asset Handling: Consider deploying dedicated static file servers or CDNs for high-traffic assets

Security Hardening Recommendations

While the current setup provides a solid security foundation, consider these enhancements for production environments:

1. Rate Limiting: Implement Traefik middleware for rate limiting to prevent abuse
2. IP Whitelisting: Add IP-based access control for administrative interfaces
3. WAF Integration: Leverage Cloudflare's Web Application Firewall for additional protection
4. Container Scanning: Implement regular security scanning for base images
5. Intrusion Detection: Consider deploying container-level intrusion detection systems

The Server Manager infrastructure provides the flexibility to implement any of these enhancements while maintaining the simple, template-driven deployment model that makes this system approachable and maintainable.