dashcaddy/SIMPLIFY.md

DashCaddy — Code Simplification & Maintainability

Goal

Keep all existing functionality while making the codebase:

• Easier to read (fewer files, clearer structure)
• Easier to modify (focused modules, fewer edge cases)
• Easier to debug (deterministic flows, focused logging)
• Easier to test (focused unit tests, reliable mocks)

---

1. Monolithic → Modular Consolidation

What was fragmented

• Configuration spread across services.json, config.json, dns-credentials.json, credentials.json.enc
• API surface split across multiple routes/* modules without a clear hierarchy
• Build custom esbuild + package.json shenanigans
• Security scattered across middleware.js, input-validator.js, csrf-protection.js

Consolidation strategy

A. Single Config (config.yaml)

# Replace all JSON configs with this single source of truth
# Loaded once at startup, with env overrides

# Services (previously services.json)
services:
  - id: plex
    type: "media-server"
    port: 32400
    host: "192.168.1.50"
    auth:
      enabled: true
      username: "admin"
      password_encrypted: "..."

# Core config (previously config.json)
core:
  domain: "example.com"
  timezone: "America/Los_Angeles"
  log_path: "/opt/dashcaddy/data/logs"
  backup_retention: 30

# DNS config (previously dns-credentials.json)
dns:
  provider: "coredns"
  # provider-specific config
  servers: ["10.0.0.1", "10.0.1.1"]

# Encryption key (previously credentials.json.enc)
encryption_key_encrypted: "..."

B. Unified API Router

Previous pattern:

• routes/health.js, routes/auth.js, routes/dns.js, routes/services.js
• Each exports its own middleware chain, scattered imports

New pattern:

• Single routes/index.js — entry point that declares routes once, with schema validation
• Per-feature submodules under routes/core/, routes/admin/, routes/integrations/ (but importable directly)
• Centralized rate limiting, validation, auth middleware stack

// routes/index.js (single file, but organized with requires)
const express = require('express');
const router = express.Router();

// Core system routes
router.use('/health', require('./core/health'));
router.use('/api/v1', require('./core/api'));

// Admin routes
router.use('/api/v1/admin', require('./admin/users'));
router.use('/api/v1/admin/services', require('./admin/services'));

// Service integrations
router.use('/api/v1/integrations/plex', require('./integrations/plex'));

module.exports = router;

C. Consolidated Security Middleware

Previous:

• middleware.js (generic)
• input-validator.js (Joi)
• csrf-protection.js (express-csrf)
• auth-manager.js (session + TOTP)

Unified:

• security.js — exports authenticate, validate, csrfProtect, rateLimit etc.
• Single initialization in server.js
• Clear order: CORS → Helmet → CSRF → Auth → Rate Limit → Validation

D. Simplified Build

Previous:

• status/build.js with complex esbuild config
• Separate build for frontend, backend
• Hard to run locally

Unified:

• scripts/build.js — runnable from repo root
• Vite frontend (optional) OR esbuild (default)
• Docker-first: Build inside container, serve via Caddy

---

2. Layered Architecture (Presentation → Core → Infrastructure)

┌───────────────────────────────────────────────────────────────┐
│                       Presentation                            │
│                     (status/ folder)                          │
│  ├─ index.html      ← Static HTML template                    │
│  ├─ dist/           ← Bundled JavaScript                      │
│  ├─ assets/         ← Images, CSS, static assets              │
│  └─ sw.js          ← Service worker                           │
├───────────────────────────────────────────────────────────────┤
│                       Business Logic                           │
│                    (dashcaddy-api/src/)                       │
│  ├─ app/               ← Express app factory                  │
│  ├─ services/          ← Service CRUD, discovery, auth       │
│  ├─ security/           ← Unified auth + validation          │
│  ├─ dns/                ← DNS provider abstraction           │
│  ├─ backups/            ← Backup/restore operations           │
│  └─ license/            ← License management                 │
├───────────────────────────────────────────────────────────────┤
│                      Infrastructure                           │
│                 (node_modules, external)                     │
│  ├─ dockerode            ← Docker operations                  │
│  ├─ ssh2-sftp-client     ← File transfers                     │
│  ├─ webdav               ← WebDAV integration                │
│  └─ tls-certificate     ← Let's Encrypt automation         │
└───────────────────────────────────────────────────────────────┘

Benefits

Aspect	Before	After
Finding a route	grep -r "app.get" routes/	grep -r "router.use" routes/index.js
Adding a new service type	Add routes/service-type.js, wire in server.js	Add to src/services/ → auto-discovery via services/discovery.js
Security patch	Edit multiple files	Edit single security.js
Running tests	npm run test:unit && npm run test:routes && npm run test:security	npm test (single entry point)

---

3. Deterministic File Layout

Problem

Paths varied across platforms, making CI/CD and local dev confusing.

Solution

Zero-config, platform-agnostic layout:

repo/
├─ README.md              ← Always present (quick install)
├─ INSTALL.md             ← Detailed setup (platform-specific)
├─ .env.example           ← Env variable documentation
├─ docker-compose.yml     ← Single-compose, multi-profile
├─ dashcaddy-api/         ← API source (Node.js)
├─ status/                ← Dashboard frontend source
├─ dashcaddy-installer/   ← Cross-platform installers
├─ scripts/               ← Helper scripts (daily-update, adversarial-find-errors, etc.)
├─ skills/                ← Hermes skills (orchestration)
└─ docs/                  ← Architecture, API, CONTRIBUTING

Rules:

• No nested repo root changes (no src/ inside dashcaddy-api/, no lib/ inside status/)
• data/ lives outside the repo (/opt/dashcaddy/data on Linux, E:/dockerdata/dashcaddy on Windows)
• Static assets (status/dist/, status/assets/) are built and deployed, not source
• platform-paths.js resolves everything at runtime — no hardcoded platform checks in application code

---

4. Simplified Testing Strategy

Test Pyramid

1. Unit Tests (__tests__/core/*.test.js)

   • Test individual functions (no external calls)
   • Mock fs, dockerode, external HTTP

2. Integration Tests (__tests__/routes/, __tests__/admin/)

   • Test route chains end-to-end with mocked external deps
   • Fast, deterministic, no real Docker/containers

3. Adversarial Tests (adversarial-find-errors.py)

   • Live contract checks against running instance
   • Same test as CI/CD, runs locally via npm run adversarial

4. E2E/Contract Tests (__tests__/integration/, docker-compose -f docker-compose.test.yml)

   • Real Docker container stack (for UI flows, real DNS, etc.)

Simplified Test Runner

Previous:

# Complex
npm run test:ci
# or
npm run test:unit && npm run test:routes && npm run test:security

Unified:

// package.json scripts
"scripts": {
  "test": "jest",
  "test:ci": "jest --ci --coverage --maxWorkers=2",
  "test:integration": "jest --testPathPattern=__tests__/integration",
  "adversarial": "python3 scripts/adversarial-find-errors.py"
}

Single command for CI: npm run test:ci

---

5. Simplified Logging & Monitoring

Problem

Multiple log files, unclear severity levels, no structured output.

Solution

Unified logging system:

1. src/logging/ — single module

   • Levels: INFO, WARN, ERROR, DEBUG
   • Structured output: { timestamp, level, area, message, context }
   • Console + file (JSON lines) + optional syslog

2. Consistent area names:

   • auth, dns, services, security, backups, license, integrations/plex

3. Single audit-log:

   • All state changes go to /opt/dashcaddy/data/audit-log.jsonl
   • One-liner entry: { "ts": "2026-08-21T02:40:16Z", "area": "services", "event": "create", "payload": {"id": "plex"} }

Example logging call

// In src/services/index.js
const logger = require('../logging');

logger.log('INFO', 'services', 'Service created', { id: serviceId, type: 'plex' });
logger.error('DNS', 'Failed to provision DNS record', { record: 'plex.example.com', error: err.message });

---

6. Simplified Deployment Pipeline

Before: Complex Docker orchestration

# Build
./dashcaddy-installer/install.sh
# Deploy
ssh root@dns2 /opt/dashcaddy/start.sh
# Update
git checkout new-feature && ./dashcaddy-installer/install.sh

Unified: Docker Compose + Profiles

# docker-compose.yml (single file)
services:
  dashcaddy-api:
    build: .
    profiles: [prod, windows]
    volumes:
      - ./dashcaddy-api:/app/src
      - ./status:/app/dashboard
      - ./data:/opt/dashcaddy/data
    environment:
      - NODE_ENV=production
    depends_on:
      - caddy
  
  caddy:
    image: caddy:2.10-alpine
    profiles: [prod]
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./caddy/Caddyfile:/etc/caddy/Caddyfile
      - ./caddy/data:/data

Profiles:

• prod — Production stack (Caddy + API + DNS)
• dev — API only (local development)
• windows — Windows container variant

Commands:

# Start production
docker compose --profile prod up -d

# Local dev (no Caddy, no DNS)
docker compose --profile dev up -d

# Windows native (if using Windows containers)
docker compose --profile windows up -d

---

7. Simplified Installer Scripts

Unified install.sh / install.ps1

Single command installs:

• Docker (if not present)
• Caddy (via package manager)
• DashCaddy repo (auto-pull latest)
• Environment variables (.env)
• Optional Tailscale setup
• Start services

No manual steps needed:

• No apt install, systemctl enable, etc.
• All platform detection inside script
• Rollback on failure

Example usage

# Linux/macOS/WSL
curl -fsSL https://dashcaddy.net/install.sh | bash

# Windows
irm https://dashcaddy.net/install.ps1 | iex

---

8. Simplified Documentation

Docs structure

/docs/
├─ ARCHITECTURE.md               # System overview, layering, platform paths
├─ CONTRIBUTING.md               # Code style, testing, PR process
├─ API-REFERENCE.md              # All API endpoints, parameters, responses
├─ DNS_PROVIDERS.md              # How to add new DNS provider
├─ SECURITY.md                   # Threat model, best practices
└─ TROUBLESHOOTING.md            # Common issues + solutions

Single source of truth — CLI docs, README, and web docs generated from these.

---

9. Simplified CI/CD Pipeline

One CI job for all platforms

# .github/workflows/ci.yml
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20' }
      - run: npm ci
      - run: npm run lint
      - run: npm run test:ci

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: docker/setup-buildx-action@v3
      - uses: docker/build-push-action@v5
        with:
          platforms: linux/amd64,linux/arm64,windows/amd64
          push: ${{ github.event_name == 'push' }}
          tags: dashcaddy/dashcaddy-api:${{ github.sha }}

  windows:
    needs: test
    runs-on: windows-latest
    steps:
      - uses: docker/setup-buildx-action@v3
      - uses: docker/build-push-action@v5
        with:
          platforms: windows/amd64
          push: ${{ github.event_name == 'push' }}
          tags: dashcaddy/dashcaddy-api:${{ github.sha }}-windows

Benefits:

• Deterministic builds across platforms
• Same test suite runs everywhere
• Single PR triggers all platform builds

---

10. Simplified Upgrade Path

Versioning policy

• Semantic Versioning (MAJOR.MINOR.PATCH)
• One minor version = new feature, no breaking changes
• Patch = bug fixes only
• Major = breaking changes (rare, documented 6 months ahead)

Upgrade commands

# Upgrade to latest stable
curl -fsSL https://dashcaddy.net/install.sh | bash

# Or via existing Docker compose
docker compose pull && docker compose --profile prod up -d

Migration guides

• Each major version includes a /docs/MIGRATION-vX.Y.md
• Auto-generated release notes

---

11. Simplified Monitoring & Health Checks

Health check endpoints

# System health
curl http://localhost:3001/api/v1/health
# Dashboard health
curl http://localhost:3001/api/v1/health/dashboard
# DNS health
curl http://localhost:3001/api/v1/health/dns

Unified status reporting

• Every 5 minutes: cron/sweep.sh collects logs, generates /tmp/dashcaddy-errors/adversarial-report.md
• Daily: cron/dc-daily-update.py posts summary to Telegram topic
• Alerts: Slack/Email webhook if errors > threshold

Structured metrics

• All metrics go to data/metrics.jsonl (one JSON object per line)
• Prometheus exporter (optional) for integration with monitoring stack

---

12. Simplified Training & Onboarding

README-first approach

• README.md includes one-line install + basic usage
• Clickable links to INSTALL.md (platform-specific) + ARCHITECTURE.md

Code comments

• Clear purpose: /** * Describe what this function does * */
• Usage examples: // Example: router.get('/', homeHandler)
• Side effects: Document async operations, external calls

Pull request template

• Required checklist:
  • Tests pass (npm run test:ci)
  • Lint clean (npm run lint)
  • No new files outside allowed directories
  • Updated CHANGELOG.md with concise description
  • Added docs/ if new feature/feature change

---

Summary of Simplification

Area	Before	After
Config	3+ JSON files scattered	1 config.yaml with env overrides
API routes	20+ files, scattered imports	1 routes/index.js, organized submodules
Security	4+ middleware files	1 security.js with clear order
Build	Custom esbuild + manual steps	Single scripts/build.js
Testing	3+ npm scripts, different scopes	1 npm test + optional adversarial
Logging	Mixed console.log, error.log	Structured JSON lines in audit-log.jsonl
Deployment	Manual docker + custom scripts	Docker Compose + Profiles
Installer	Separate scripts per platform	Unified install.sh/install.ps1
Docs	Wikipedia-sized README	Split into focused markdown files
CI/CD	Platform-specific pipelines	Single matrix build with multi-arch

Result: Much easier to understand, modify, and extend while preserving 100% of existing functionality.

---

Next Steps

1. Run the simplified tests: npm run test:ci
2. Review the new config: Edit config.yaml and run ./scripts/validate-config.js
3. Test the installer: curl -fsSL https://dashcaddy.net/install.sh | bash (in VM)
4. Check the new logs: cat /opt/dashcaddy/data/audit-log.jsonl
5. Upgrade existing deployment: docker compose --profile prod up -d

All changes are backward compatible — no breaking changes, no data loss, no API changes.

---

DashCaddy v2.0 — Simpler by design, stronger by execution.