diff --git a/.gitignore b/.gitignore index 2be44b4..4ac1716 100644 --- a/.gitignore +++ b/.gitignore @@ -43,12 +43,18 @@ TEST-RESULTS.md TESTING-GUIDE.md DashCA-Plan.md vhdx-cleanup-instructions.md +DESLOPIFICATION-ROADMAP.md +SECURITY-IMPROVEMENTS.md +WHAT-IS-DASHCADDY.md +error-handling-cleanup-summary.md +error-handling-migration-complete.md # Utility scripts (local only) check-e.ps1 disk-scan.ps1 disk-scan2.ps1 fix-wsl-and-mount.ps1 +fix-ctx-routes.sh import-services.js # OS files diff --git a/DESLOPIFICATION-ROADMAP.md b/DESLOPIFICATION-ROADMAP.md deleted file mode 100644 index da60a6b..0000000 --- a/DESLOPIFICATION-ROADMAP.md +++ /dev/null @@ -1,388 +0,0 @@ -# DashCaddy API Deslopification Roadmap - -**Audited:** 2026-03-22 -**Version:** 1.1.0 -**Total Lines:** ~26,000 (API), ~10,000 (dashboard) -**Priority:** API-first (make backend powerful, clean dashboard follows naturally) - ---- - -## Executive Summary - -The DashCaddy API is **feature-complete and security-hardened**, but the codebase shows signs of rapid evolution. While functionally robust, it would significantly benefit from architectural refactoring to improve maintainability, testability, and long-term scalability. - -### Key Strengths -✅ Comprehensive feature set (76+ app templates, Docker/Caddy/DNS management) -✅ Security-conscious (TOTP auth, AES-256-GCM credentials, CSRF protection, audit logging) -✅ Recent test coverage additions (auth, credentials, Docker security) -✅ Modular route organization (routes/ subdirectories) -✅ Shared context pattern for dependency injection - -### Core Issues -❌ **Monolithic `server.js`** (1960 lines) — initialization, middleware, utilities, business logic all in one file -❌ **God object `ctx`** — 50+ properties/methods across multiple domains with hidden dependencies -❌ **Inconsistent patterns** — routes use classes, factory functions, or flat modules with no standard -❌ **No code standards** — ESLint installed but no config, no formatting rules -❌ **Mixed concerns** — HTTP handlers, business logic, validation intertwined in route files - ---- - -## Current Architecture - -``` -dashcaddy-api/ -├── server.js (1960 lines) ← MAIN PROBLEM -│ ├── 89 require() statements -│ ├── 131 top-level declarations -│ ├── Middleware setup -│ ├── Context (`ctx`) assembly (50+ properties) -│ ├── Route mounting -│ ├── Error handlers -│ └── Server startup -├── routes/ -│ ├── auth/ (5 files, modular) ✅ -│ ├── config/ (4 files, modular) ✅ -│ ├── apps/ (6 files, helpers pattern) ⚠️ -│ ├── arr/ (4 files, helpers pattern) ⚠️ -│ ├── recipes/ (3 files) ⚠️ -│ └── *.js (19 flat route files) ❌ -├── Managers (clean, well-separated) -│ ├── auth-manager.js (307 lines) ✅ -│ ├── credential-manager.js (395 lines) ✅ -│ ├── state-manager.js (237 lines) ✅ -│ ├── backup-manager.js (835 lines) ⚠️ -│ ├── health-checker.js (591 lines) ⚠️ -│ └── update-manager.js (911 lines) ⚠️ -├── Utilities -│ ├── input-validator.js (606 lines) ⚠️ -│ ├── crypto-utils.js (340 lines) ✅ -│ ├── middleware.js (430 lines) ⚠️ -│ └── constants.js ✅ -└── Templates - ├── app-templates.js (2496 lines) ⚠️ - └── recipe-templates.js (339 lines) ✅ -``` - -**Legend:** -✅ Good structure -⚠️ Works but could be cleaner -❌ Needs refactoring - ---- - -## Deslopification Phases - -### Phase 1: Foundation & Standards (IMMEDIATE) -**Goal:** Establish code quality baseline before refactoring -**Effort:** 2-4 hours -**Risk:** Low (tooling only, no code changes) - -#### 1.1 Code Standards Setup -- [ ] Create `.eslintrc.js` with recommended rules -- [ ] Add Prettier config (`.prettierrc`) -- [ ] Add npm scripts: `lint`, `lint:fix`, `format` -- [ ] Run `npm run lint:fix` and commit baseline cleanup -- [ ] Add pre-commit hooks (optional) - -**Why first:** Establish formatting/style consistency before making structural changes. Prevents "should I refactor this while I'm here?" scope creep. - -#### 1.2 Dependency Graph Documentation -- [ ] Map `ctx` properties → which routes actually use them -- [ ] Identify circular dependencies (if any) -- [ ] Document shared utilities used across routes - -**Deliverable:** `DEPENDENCIES.md` — reference for refactoring decisions - ---- - -### Phase 2: Extract & Organize (HIGH PRIORITY) -**Goal:** Break `server.js` into logical modules -**Effort:** 1-2 days -**Risk:** Medium (requires testing at each step) - -#### 2.1 Split `server.js` into Layers -**Before:** 1960-line monolith -**After:** Clean initialization flow - -Create new structure: -``` -src/ -├── app.js ← Express app setup (middleware, routes) -├── server.js ← Entry point (load config, start server) -├── config/ -│ ├── index.js ← Load all config (env, files, constants) -│ ├── env.js ← Environment variable validation -│ └── paths.js ← Platform-specific paths -├── context/ -│ ├── index.js ← Assemble context (DI container) -│ ├── docker.js ← Docker-related context properties -│ ├── caddy.js ← Caddy-related context properties -│ ├── dns.js ← DNS context -│ ├── session.js ← Session context -│ └── notification.js ← Notification context -├── middleware/ -│ ├── index.js ← Export all middleware -│ ├── auth.js ← Move from middleware.js -│ ├── error.js ← Error handlers -│ └── security.js ← Helmet, CORS, CSRF -└── routes/ - └── (existing structure) -``` - -**Migration Steps:** -1. Create `src/config/` — extract all config loading from `server.js` -2. Create `src/context/` — split god object into domain modules -3. Create `src/middleware/` — break up `middleware.js` (430 lines) -4. Create `src/app.js` — Express setup + route mounting -5. Slim `server.js` → minimal entry point (~50 lines) - -**Tests:** Ensure existing test suite still passes after each step - ---- - -### Phase 3: Route Standardization (MEDIUM PRIORITY) -**Goal:** Consistent route module pattern across entire API -**Effort:** 2-3 days -**Risk:** Medium (touching business logic) - -#### 3.1 Establish Route Pattern -**Chosen Pattern:** Factory function with explicit dependencies - -```javascript -// routes/services.js (before) -module.exports = (ctx) => { - const router = express.Router(); - // ... uses ctx.docker, ctx.servicesStateManager, ctx.log, etc. - return router; -}; - -// routes/services.js (after) -module.exports = ({ docker, servicesStateManager, log, asyncHandler }) => { - const router = express.Router(); - // ... explicitly passed dependencies - return router; -}; -``` - -**Benefits:** -- Self-documenting (you see what each route needs) -- Easier testing (mock only what's used) -- No hidden dependencies via god object - -#### 3.2 Refactor Routes by Priority -**Order:** Most-used routes first - -1. **High-traffic routes:** - - `routes/services.js` (467 lines) — core service management - - `routes/containers.js` (246 lines) — Docker operations - - `routes/health.js` (297 lines) — health checks - - `routes/dns.js` (632 lines) — DNS management - -2. **Auth routes** (already modular, just align pattern): - - `routes/auth/*` - -3. **Feature routes:** - - `routes/apps/*` - - `routes/arr/*` - - `routes/recipes/*` - -4. **Utility routes:** - - `routes/logs.js` - - `routes/backups.js` - - `routes/ca.js` - - etc. - -**Per-route checklist:** -- [ ] Extract dependencies from `ctx` → explicit parameters -- [ ] Move business logic to service layer (if complex) -- [ ] Validate inputs at route boundary -- [ ] Return consistent error format -- [ ] Add route-level tests - ---- - -### Phase 4: Service Layer Introduction (LOWER PRIORITY) -**Goal:** Separate business logic from HTTP handlers -**Effort:** 3-5 days -**Risk:** Medium-High (significant refactor) - -**Problem:** Routes currently mix HTTP concerns with business logic: -```javascript -// Current: Everything in route handler -router.post('/deploy', async (req, res) => { - // 1. Parse request - // 2. Validate inputs - // 3. Business logic (complex Docker operations) - // 4. Error handling - // 5. Format response -}); -``` - -**Solution:** Service layer pattern -```javascript -// routes/apps/deploy.js -router.post('/deploy', async (req, res) => { - const result = await appDeployService.deploy(req.body); - res.json({ success: true, data: result }); -}); - -// services/app-deploy-service.js -class AppDeployService { - async deploy({ templateId, config }) { - // Pure business logic, no HTTP awareness - } -} -``` - -**Candidates for service extraction:** -- `services/docker-service.js` — container lifecycle, networking -- `services/caddy-service.js` — Caddyfile manipulation, reload -- `services/dns-service.js` — record management, zone operations -- `services/app-deploy-service.js` — template-based deployment -- `services/backup-service.js` — backup/restore workflows - -**Benefits:** -- Routes become thin HTTP adapters (easy to test) -- Business logic testable without HTTP mocking -- Reusable across routes (e.g., CLI tools, cron jobs) - ---- - -### Phase 5: Manager Cleanup (ONGOING) -**Goal:** Refine existing manager modules -**Effort:** 1-2 days (parallel to other phases) - -#### Issues to Address -1. **`backup-manager.js` (835 lines)** — too large, split backup vs restore logic -2. **`update-manager.js` (911 lines)** — complex state machine, extract version comparison utilities -3. **`health-checker.js` (591 lines)** — separate health check logic from notification daemon -4. **`input-validator.js` (606 lines)** — split by domain (docker, caddy, dns validators) - -**Approach:** Incremental splitting, preserve existing API - ---- - -### Phase 6: Template Organization (LOW PRIORITY) -**Goal:** Make templates maintainable and extensible -**Effort:** 1 day - -**Problem:** `app-templates.js` is 2496 lines (76 templates in one file) - -**Solution:** -``` -templates/ -├── index.js ← Export TEMPLATE_CATEGORIES, DIFFICULTY_LEVELS -├── apps/ -│ ├── media/ -│ │ ├── plex.js -│ │ ├── jellyfin.js -│ │ └── ... -│ ├── automation/ -│ └── ... -└── recipes/ - ├── arr-stack.js - └── ... -``` - -**Benefits:** -- Easier to find/edit specific templates -- Contributors can add templates without merge conflicts -- Templates can import shared snippets (e.g., common env vars) - ---- - -## Metrics & Success Criteria - -### Code Quality Metrics (Before → After) - -| Metric | Before | Target | How to Measure | -|--------|--------|--------|----------------| -| `server.js` lines | 1960 | <200 | `wc -l server.js` | -| Avg route file size | ~300 | <150 | `find routes -name '*.js' -exec wc -l {} + \| awk '{sum+=$1; n++} END {print sum/n}'` | -| `ctx` properties | 50+ | 0 (removed) | Manual count | -| ESLint errors | Unknown | 0 | `npm run lint` | -| Test coverage | ~30% | >60% | `npm run test:coverage` | -| Files >500 lines | 8 | <3 | `find . -name '*.js' -exec wc -l {} + \| awk '$1 > 500'` | - -### Developer Experience Improvements -- **Onboarding:** New contributor should understand route structure in <10 minutes -- **Testing:** Mock only what you use (no god object sprawl) -- **Changes:** Touching one domain shouldn't require understanding entire codebase -- **Deployment:** Confidence that refactor didn't break anything (test suite) - ---- - -## Risk Mitigation - -### How to Refactor Safely - -1. **Test suite first** — before touching code: - - Run existing tests: `npm test` - - Identify untested critical paths → add tests - - Establish coverage baseline - -2. **Incremental changes**: - - Each phase = separate branch - - Each phase passes full test suite - - Deploy to test environment (Contabo) before merging - -3. **Preserve API contract**: - - Frontend expects same endpoints/responses - - Dashboard shouldn't need changes during API refactor - - Version routes if breaking changes needed - -4. **Rollback plan**: - - Git tags before each phase merge - - Keep old code in `legacy/` until confidence is high - - Document what changed in each PR - ---- - -## Recommended Order of Execution - -**Week 1: Foundation** -- Day 1-2: Phase 1 (ESLint, Prettier, dependency mapping) -- Day 3-5: Phase 2.1 (split `server.js`) - -**Week 2: Routes** -- Day 1-3: Phase 3.1 (standardize top 5 routes) -- Day 4-5: Phase 3.2 (remaining routes) - -**Week 3: Refinement** -- Day 1-3: Phase 4 (service layer for complex routes) -- Day 4-5: Phase 5 (manager cleanup) - -**Week 4: Polish** -- Day 1-2: Phase 6 (template organization) -- Day 3-5: Documentation, final testing, deployment - -**Total:** ~4 weeks part-time or ~2 weeks full-time - ---- - -## Questions for Sami - -Before starting, clarify: - -1. **Testing strategy:** Current test coverage is partial. Should we: - - Write tests BEFORE refactoring (safer, slower)? - - Refactor with existing tests, add coverage later (faster, riskier)? - -2. **Breaking changes:** Can we introduce backwards-incompatible API changes if we version routes (`/api/v2/...`)? - -3. **Deployment cadence:** Should each phase deploy to production, or batch into one big release? - -4. **Priority tweaks:** Does this roadmap align with "deslopify → market → sell" timeline, or should we focus only on the most visible pain points first? - ---- - -## Next Steps - -**If approved:** -1. Create feature branch: `refactor/deslopification-phase-1` -2. Add ESLint + Prettier configs -3. Run `npm run lint:fix` and commit baseline -4. Create `DEPENDENCIES.md` (ctx usage map) -5. Review with Sami before proceeding to Phase 2 - -**Estimated time to first visible improvement:** 1 week (server.js split + linting) diff --git a/SECURITY-IMPROVEMENTS.md b/SECURITY-IMPROVEMENTS.md deleted file mode 100644 index 4cd5b1e..0000000 --- a/SECURITY-IMPROVEMENTS.md +++ /dev/null @@ -1,289 +0,0 @@ -# DashCaddy Security Improvements -**Date:** 2026-03-21 -**Desloppify Score:** 15.4/100 → Target: 95.0/100 - -## Summary of Changes - -This commit implements critical security improvements identified by Desloppify code analysis, addressing 20 security issues and establishing a foundation for comprehensive test coverage. - ---- - -## 🚨 Phase 1: Critical Security Fixes - -### 1.1 New Sanitization Infrastructure - -**File:** `dashcaddy-api/logger-utils.js` (NEW) - -Created a comprehensive logging sanitization utility to prevent credential leakage in logs: - -- **`sanitizeForLog(data, additionalSensitiveKeys)`**: Recursively redacts sensitive fields from objects/arrays -- **`redactCredential(value)`**: Partially redacts credentials (shows first/last 4 chars only) -- **`safeLog(message, data)`**: Creates safe log objects with automatic sanitization -- **`SENSITIVE_FIELDS`**: 30+ sensitive field name patterns (password, token, apiKey, secret, etc.) - -**Security Impact:** -- Prevents accidental logging of passwords, tokens, API keys, certificates -- Case-insensitive field matching -- Handles nested objects and arrays -- Supports custom sensitive field lists - ---- - -### 1.2 Auth Manager Security Enhancements - -**File:** `dashcaddy-api/auth-manager.js` - -**Changes:** -1. Added `logger-utils` import for future sanitization -2. Added security comments on lines 16-18 (JWT_SECRET handling) -3. Line 48: Added comment clarifying tokens are never logged -4. Line 73: Removed error.message from JWT invalid logs (could leak token data) -5. Line 109: Added comment confirming API keys are never logged - -**Fixed Issues:** -- Lines 16, 17, 96: Hardcoded secret name warnings (clarified these are variable names, not actual secrets) -- Lines 71, 73: Logging sensitive authentication data (confirmed safe - only logs event names, not values) - ---- - -### 1.3 Environment Variable Template - -**File:** `dashcaddy-api/.env.example` (NEW) - -Created comprehensive environment variable template with: -- JWT_SECRET configuration -- Docker/Caddy/DNS settings -- Notification provider configuration (Discord, Telegram, Ntfy) -- Tailscale OAuth settings -- Clear documentation and warnings - -**Security Impact:** -- Provides secure configuration template -- Documents all required/optional environment variables -- Prevents accidental credential commits - ---- - -### 1.4 .gitignore Updates - -**File:** `.gitignore` - -**Added:** -``` -dashcaddy-api/.env -.env -``` - -**Existing (preserved):** -``` -dashcaddy-api/credentials.json -``` - -**Security Impact:** -- Prevents accidental commit of environment variables -- Prevents accidental commit of encrypted credential storage - ---- - -## 📊 Phase 2: Test Coverage Foundation - -### 2.1 Logger Utils Test Suite - -**File:** `dashcaddy-api/__tests__/logger-utils.test.js` (NEW) - -Created comprehensive test suite for logger-utils.js: - -**Test Coverage:** -- ✅ `sanitizeForLog`: 6 test cases - - Sensitive field redaction - - Nested object handling - - Array handling - - Null/undefined handling - - Additional sensitive keys - - Case-insensitive matching -- ✅ `redactCredential`: 5 test cases - - Long string partial redaction - - Short string full redaction - - Null/undefined handling - - Non-string input handling - - Asterisk limiting -- ✅ `safeLog`: 3 test cases - - Safe log object creation - - Timestamp validation - - Empty data handling -- ✅ `SENSITIVE_FIELDS`: 2 test cases - - Common field name presence - - Array length validation - -**Total:** 16 test cases covering all public API functions - -**Test Infrastructure:** -- Uses existing Jest configuration -- Follows standard `__tests__/` directory convention -- Can be run with `npm test` - ---- - -## 📋 Files Modified - -| File | Status | Changes | -|------|--------|---------| -| `dashcaddy-api/logger-utils.js` | ✨ NEW | Logging sanitization utility (133 lines) | -| `dashcaddy-api/__tests__/logger-utils.test.js` | ✨ NEW | Comprehensive test suite (173 lines) | -| `dashcaddy-api/.env.example` | ✨ NEW | Environment variable template | -| `dashcaddy-api/auth-manager.js` | 🔧 MODIFIED | Security comments + import added | -| `.gitignore` | 🔧 MODIFIED | Added .env exclusions | -| `SECURITY-IMPROVEMENTS.md` | ✨ NEW | This document | - ---- - -## 🎯 Desloppify Score Impact - -### Current Remediation (Phase 1-2 Partial) -| Metric | Before | After | Change | -|--------|---------|-------|--------| -| **Overall Score** | 15.4 | ~25-30* | +10-15 pts | -| **Security** | 62.5% | ~80%* | +17.5% | -| **Test Coverage** | 0% | ~5%* | +5% | - -*Estimated - requires rescan to confirm - -### Remaining Work (Phase 3-4) -To reach target score of 95.0/100, the following work remains: - -**High Priority (Phase 3):** -- [ ] Add tests for auth-manager.js (CRITICAL - handles authentication) -- [ ] Add tests for credential-manager.js (CRITICAL - handles secrets) -- [ ] Add tests for docker-security.js (HIGH - security module) -- [ ] Add tests for input-validator.js (HIGH - injection prevention) -- [ ] Refactor server.js (2,100 LOC → split into routes/ + services/) -- [ ] Extract hardcoded constants to named constants - -**Medium Priority (Phase 4):** -- [ ] Subjective code review (naming, API coherence, error consistency) -- [ ] Type safety improvements (JSDoc or TypeScript migration) -- [ ] Documentation improvements (CONTRIBUTING.md, API docs) - ---- - -## 🛠️ How to Deploy These Changes - -### 1. Review Changes -```bash -git diff -``` - -### 2. Run Tests -```bash -cd dashcaddy-api -npm test -``` - -Expected output: 16 tests passing (all in logger-utils.test.js) - -### 3. Copy to Production -On Windows machine (dns1-sami): -```powershell -# Backup current production -Copy-Item C:/caddy/sites/dashcaddy-api C:/caddy/sites/dashcaddy-api.backup -Recurse - -# Deploy new files -Copy-Item dashcaddy-api/logger-utils.js C:/caddy/sites/dashcaddy-api/ -Copy-Item dashcaddy-api/auth-manager.js C:/caddy/sites/dashcaddy-api/ -Copy-Item dashcaddy-api/__tests__ C:/caddy/sites/dashcaddy-api/ -Recurse -Copy-Item dashcaddy-api/.env.example C:/caddy/sites/dashcaddy-api/ - -# Restart container -docker restart dashcaddy-api -``` - -### 4. Verify Deployment -```bash -# Check container logs -docker logs dashcaddy-api --tail 50 - -# Test health endpoint -curl http://localhost:3001/health -``` - ---- - -## 🔒 Security Considerations - -### What Was Fixed -1. ✅ Created centralized logging sanitization -2. ✅ Added security comments to clarify safe logging practices -3. ✅ Created .env template for secure configuration -4. ✅ Updated .gitignore to prevent credential commits -5. ✅ Established test coverage foundation - -### What Still Needs Attention -1. ⚠️ **Rotate any secrets previously committed to git** (if any exist in git history) -2. ⚠️ **Create actual .env file** from .env.example with real values (do NOT commit) -3. ⚠️ **Audit existing logs** for any historical credential leakage -4. ⚠️ **Implement auth-manager tests** to validate security boundaries -5. ⚠️ **Implement credential-manager tests** to validate encryption - ---- - -## 📚 Next Steps - -### Immediate (This Week) -1. Run Desloppify rescan to confirm score improvement -2. Create .env file from template (production servers only) -3. Deploy changes to production -4. Write auth-manager.js tests - -### Short-term (Next 2 Weeks) -1. Complete Phase 2 test coverage (credential-manager, docker-security, input-validator) -2. Begin Phase 3 refactoring (split server.js) -3. Extract magic numbers to named constants - -### Long-term (Next Month) -1. Achieve 80%+ test coverage -2. Complete Phase 4 subjective improvements -3. Reach Desloppify target score of 95.0/100 - ---- - -## 🙏 Acknowledgments - -This security improvement initiative was driven by Desloppify static analysis tool, which identified: -- 20 security issues (4 hardcoded secrets, 16 logging concerns) -- 0% test coverage -- Structural improvements needed across 8 files - -**Tools Used:** -- [Desloppify](https://github.com/peteromallet/desloppify) - Code quality analysis -- Jest - JavaScript testing framework -- ESLint - JavaScript linting (already configured) - ---- - -## 📝 Commit Message Template - -``` -security: implement Phase 1-2 fixes (logger sanitization + tests) - -- Add logger-utils.js for credential sanitization in logs -- Add security comments to auth-manager.js -- Create .env.example template -- Add .env to .gitignore -- Implement comprehensive logger-utils tests (16 cases) - -Desloppify score: 15.4 → ~25-30 (estimated) -Security: 62.5% → ~80% -Test coverage: 0% → ~5% - -Fixes: 20 security issues -Adds: 16 test cases -Created: 3 new files, modified 2 existing files - -See SECURITY-IMPROVEMENTS.md for full details. -``` - ---- - -**Generated:** 2026-03-21 03:45 CET -**Author:** Krystie (OpenClaw AI Assistant) -**Reviewed:** Pending human review diff --git a/WHAT-IS-DASHCADDY.md b/WHAT-IS-DASHCADDY.md deleted file mode 100644 index c8b220c..0000000 --- a/WHAT-IS-DASHCADDY.md +++ /dev/null @@ -1,242 +0,0 @@ -# What is DashCaddy? - -DashCaddy is a self-hosted web dashboard that unifies Docker container management, Caddy reverse proxy configuration, DNS automation, and SSL certificate provisioning into a single interface. It is designed for homelabbers and self-hosters who want to deploy and manage services without manually editing config files, writing Docker Compose YAML, or configuring DNS records by hand. - -You open one page, click "Deploy", pick an app, and DashCaddy handles everything: pulls the Docker image, starts the container, creates a DNS record, adds a reverse proxy block with automatic HTTPS, and registers the service on your dashboard — all in about 30 seconds. - -## The Stack - -| Layer | Technology | Role | -|-------|-----------|------| -| Frontend | Vanilla JS SPA (~12,000 lines across 33 modules) | Dashboard UI, modals, wizards | -| Backend | Node.js / Express (~20,000 lines across 22 modules + 20 route files) | API server with 125+ endpoints | -| Reverse Proxy | Caddy | HTTPS termination, internal CA, automatic certificates | -| DNS | Technitium DNS Server | Automatic A-record creation for `*.sami` domains | -| Containers | Docker (via dockerode) | Application lifecycle management | -| Auth | TOTP (RFC 6238) + JWT | Two-factor authentication for dashboard access | -| Encryption | AES-256-GCM | Credential storage with OS keychain fallback | - -The API server runs inside a Docker container (`caddy-api`) on port 3001. Caddy sits in front of everything on port 443, terminating TLS with certificates signed by its own root CA. - -## What It Does - -### One-Click App Deployment - -55 pre-configured templates across 16 categories (Media, Downloads, Productivity, Development, Monitoring, DNS, Security, and more). Each template defines the Docker image, default port, environment variables, volume mounts, health check endpoint, and setup instructions. Deploying an app: - -1. Pulls the Docker image -2. Creates the container with the right env vars, ports, and volumes -3. Creates a DNS A-record on Technitium (e.g., `plex.sami`) -4. Adds a reverse proxy block to the Caddyfile with TLS -5. Reloads Caddy -6. Registers the service on the dashboard with health monitoring - -### Dashboard - -Real-time service cards showing status (up/slow/down), response time, uptime percentage, and container ID. Each card has controls to open the service, restart the container, view logs, edit settings, manage auto-login credentials, or delete the service. - -Special top-row cards for DNS servers, internet connectivity, TOTP auth status, and the certificate authority. - -### Smart Arr Connect - -A four-phase wizard that auto-detects Plex, Radarr, Sonarr, Overseerr/Jellyseerr, and Prowlarr, fetches their API keys, and wires them together automatically — connecting Overseerr to Plex, configuring Prowlarr with indexers for Radarr/Sonarr, etc. - -### Auto-Login SSO - -Per-service credential storage that authenticates users into services transparently via Caddy's `forward_auth` directive. Supports cookie-based auth, JWT-based auth (Open WebUI, Plex), IP-based auth (router), and Emby/Jellyfin token auth with separate device IDs to avoid token invalidation. - -### DashCA (Certificate Authority Distribution) - -A static site at `ca.sami` that auto-detects the visitor's OS and provides one-click installation of the root CA certificate. Supports Windows (PowerShell), macOS (.mobileconfig), Linux (shell script), iOS (profile), and Android (direct .crt download). - -### Monitoring and Operations - -- **Health Checker**: Periodic HTTP probes with configurable endpoints per service -- **Resource Monitor**: Per-container CPU, memory, disk I/O, and network stats -- **Update Manager**: Checks Docker Hub for newer image versions, one-click updates -- **Backup/Restore**: Export/import full dashboard configuration as JSON -- **Audit Logger**: Tracks all administrative actions -- **Error Log Viewer**: Aggregated error logs with severity filtering -- **Metrics**: Request counts, response times, error rates, business events (deploys, deletions, DNS records created) -- **Notifications**: Configurable alerts for health check failures and resource thresholds - -### Security - -- TOTP two-factor authentication with QR code setup -- CSRF token protection on all mutating endpoints -- Helmet security headers -- Rate limiting (general, strict, TOTP tiers) -- Input validation and sanitization (via `validator` library) -- AES-256-GCM credential encryption with OS keychain integration -- Docker security scanning -- API key management -- Non-root container execution with health checks - -### Other Features - -- Three themes (dark, light, blue) -- Keyboard shortcuts -- Customizable logo with position control -- Weather widget -- Setup wizard with three modes (Simple, Homelab, Public Server) -- Guided onboarding tour (Driver.js) -- Tailscale integration for access control -- Media folder browser for configuring volume mounts -- Interactive API documentation (OpenAPI/Swagger) - ---- - -## Architecture Diagram - -``` - Browser (index.html) - │ - ▼ - Caddy :443 ─── TLS (internal CA) ───┐ - │ │ - ├── /api/* → caddy-api :3001 │ - ├── *.sami → reverse proxy │ - │ to Docker containers │ - └── ca.sami → static DashCA site │ - │ - caddy-api container │ - ├── Express (server.js) │ - │ ├── 20 route modules │ - │ ├── State Manager (lock) │ - │ ├── Credential Manager │ - │ ├── Health Checker │ - │ ├── Resource Monitor │ - │ └── Metrics Collector │ - │ │ - ├──→ Docker Engine (dockerode) │ - ├──→ Caddy Admin API :2019 │ - ├──→ Technitium DNS :5380 │ - └──→ services.json (file-locked) │ -``` - -## Current State - -**Version**: 0.95 (1.0 = public release) - -The project is fully functional and in daily use. All core features work. The codebase has a test suite (17 test files under `__tests__/`) covering validators, crypto, health checks, state management, API endpoints, and integration scenarios. - ---- - -## Obstacles to v1.0 Release - -### 1. Windows-Only — No Cross-Platform Support - -DashCaddy was built on and for Windows. The entire deployment model assumes: -- `C:/caddy/` as the production path -- Windows-style path handling throughout (`C:\caddy\Caddyfile`, `host.docker.internal`) -- Docker Desktop for Windows -- Windows Task Scheduler for backups -- PowerShell for CA certificate installation - -A Linux or macOS user cannot run this without significant path rewiring. For a public release, either the documentation must clearly state "Windows only" or the path handling needs to be abstracted with platform-aware defaults. - -### 2. Hardcoded Infrastructure Assumptions - -The codebase has assumptions baked in that only apply to the author's setup: - -- **`.sami` TLD**: The local domain suffix is referenced throughout (Caddyfile templates, DNS record creation, documentation). A public user would need their own TLD — this needs to be a first-run configuration option, not a find-and-replace exercise. -- **Technitium DNS**: DNS automation assumes Technitium's REST API. Users running Pi-hole, CoreDNS, or no local DNS server have no path. The DNS layer needs to be pluggable or clearly documented as a hard requirement. -- **Docker Desktop**: Container operations assume Docker Desktop's `host.docker.internal` hostname. Native Docker on Linux uses `localhost` differently. -- **Caddy internal CA**: The TLS model assumes Caddy's built-in PKI. Users wanting Let's Encrypt or other CAs need a different onboarding flow (partially addressed by the "Public Server" setup wizard mode). - -### 3. Single-Page HTML Monolith - -The frontend is a ~12,000-line single HTML + 33 JS files architecture with no build step, no bundler, no framework, and no component system. While this means zero build tooling to configure, it creates obstacles: - -- No minification or tree-shaking — the full payload is served on every load -- No code splitting — all 33 modules load upfront -- IIFEs communicate through `window` globals — fragile, hard to test -- No TypeScript — no compile-time safety on a 12k-line frontend -- CSS is embedded in the HTML — no style extraction or scoping - -This works fine for a personal tool but makes contribution and maintenance harder at scale. - -### 4. No Automated Test Coverage for the Frontend - -The backend has 17 test files with unit and integration tests. The frontend has zero tests. The dashboard UI is the primary interface users interact with, and it has no test safety net — no unit tests, no E2E tests, no screenshot regression tests. - -### 5. No CI/CD Pipeline - -There is no GitHub Actions workflow, no pre-commit hooks, no automated linting, and no automated test runs. The deployment process is manual: - -1. Edit files in `e:/CaddyCerts/sites/dashcaddy-api/` -2. Copy JS files to `C:/caddy/sites/dashcaddy-api/` -3. Run `docker restart caddy-api` - -A public project needs at minimum: automated tests on push, a linter, and a documented release process. - -### 6. No Installation or Setup Documentation - -There is no README explaining how to install DashCaddy from scratch. The `CLAUDE.md` is an internal reference for AI assistants. A new user would need: - -- Prerequisites (Docker Desktop, Caddy, Technitium, Node.js) -- Step-by-step installation guide -- First-run configuration walkthrough -- Troubleshooting guide -- Architecture overview - -### 7. Single-User Only - -There is no concept of user accounts, roles, or permissions. TOTP protects access but there's one global session. For a household with multiple users, there's no way to give someone read-only access or restrict who can deploy/delete containers. - -### 8. No Container Orchestration Beyond Single-Host - -DashCaddy manages containers on one Docker host. There's no support for: -- Docker Compose stacks (multi-container apps like Nextcloud + MariaDB + Redis) -- Docker Swarm or Kubernetes -- Remote Docker hosts -- Container networking (custom networks, inter-container communication) - -Apps that need multiple containers (databases, caches, sidecars) must be set up manually. - -### 9. Credential and Secret Management Gaps - -While credentials are encrypted with AES-256-GCM, the encryption key management has limitations: -- The master key derivation and storage strategy isn't documented for end users -- Key rotation exists but there's no scheduled rotation or policy -- Backup exports include encrypted credentials but the key management for restoring on a different machine isn't clear -- No integration with external secret managers (Vault, 1Password, etc.) - -### 10. Incomplete Template Coverage - -55 templates is a strong start, but several popular self-hosted apps are missing, and the template system has constraints: -- No user-contributed templates or template marketplace -- No template versioning — if an image tag changes, templates need manual updates -- No Docker Compose support — templates are single-container only -- Environment variable templating is basic (`{{PORT}}`, `{{SUBDOMAIN}}`) with no conditional logic - -### 11. No Persistent Logging or Metrics Storage - -Metrics (request counts, response times, business events) are in-memory only — they reset on container restart. There's no time-series database, no Prometheus endpoint, no Grafana integration. For a monitoring-focused dashboard, losing all metrics on restart is a significant gap. - -### 12. The Development/Production File Split - -The two-directory development model (`e:/CaddyCerts/sites/` for editing, `C:/caddy/` for production) works for the author but would confuse contributors and can't work as-is for other users. A public release needs a single canonical source of truth with a proper build/deploy pipeline. - ---- - -## What's Strong - -Despite these obstacles, DashCaddy has substantial strengths that position it well for release: - -- **Feature-complete for its core use case**: Deploy apps, manage reverse proxy, automate DNS — it all works -- **Security-first design**: TOTP, CSRF, rate limiting, encryption, input validation, non-root containers -- **Polished UI**: Themes, keyboard shortcuts, onboarding tour, skeleton loaders, responsive design -- **Smart Arr Connect**: A genuinely useful automation that saves significant manual configuration -- **Auto-Login SSO**: Handles the messy reality of diverse auth mechanisms (cookies, JWT, IP-based, localStorage) -- **55 app templates**: Broad coverage of the self-hosting ecosystem -- **Thread-safe state management**: Proper file locking prevents corruption under concurrent access -- **In-memory metrics and monitoring**: Even without persistence, the real-time view is useful -- **Test suite exists**: 17 backend test files covering critical paths -- **Modular route architecture**: 20 route files keep the 125+ endpoints organized and maintainable - -## Summary - -DashCaddy is a mature, feature-rich self-hosting dashboard that solves a real problem — the tedium of manually configuring Docker + reverse proxy + DNS for every new service. It's daily-driver stable for a single Windows user with Caddy and Technitium DNS. - -The gap between "works great for me" and "anyone can install this" is the remaining 0.05 to v1.0. The biggest obstacles are cross-platform support, installation documentation, and removing the hardcoded infrastructure assumptions. The frontend architecture and CI/CD are secondary concerns that matter more for long-term maintainability than for a functional v1.0 release. diff --git a/dashcaddy-api/.gitignore b/dashcaddy-api/.gitignore index dbfd579..45b6b52 100644 --- a/dashcaddy-api/.gitignore +++ b/dashcaddy-api/.gitignore @@ -1,6 +1,7 @@ # Backups .backup/ +server-old.js # Test artifacts __tests__/jest.setup.js diff --git a/dashcaddy-api/routes/auth/session-handlers.js b/dashcaddy-api/routes/auth/session-handlers.js index 0452c7c..7c2822d 100644 --- a/dashcaddy-api/routes/auth/session-handlers.js +++ b/dashcaddy-api/routes/auth/session-handlers.js @@ -1,7 +1,7 @@ const { SESSION_TTL, APP, PLEX, TIMEOUTS, buildMediaAuth } = require('../../constants'); const { createCache, CACHE_CONFIGS } = require('../../cache-config'); -module.exports = function({ authManager, credentialManager, asyncHandler, errorResponse, log }) { +module.exports = function({ authManager, credentialManager, fetchT, asyncHandler, errorResponse, log }) { // App session cache for auto-login /** * Auth session handlers routes factory @@ -71,7 +71,7 @@ module.exports = function({ authManager, credentialManager, asyncHandler, errorR case 'emby': { const mediaAuth = buildMediaAuth(APP.DEVICE_IDS.SSO); try { - const authResp = await ctx.fetchT(`${baseUrl}/Users/AuthenticateByName`, { + const authResp = await fetchT(`${baseUrl}/Users/AuthenticateByName`, { method: 'POST', headers: { 'Content-Type': 'application/json', 'X-Emby-Authorization': mediaAuth }, body: JSON.stringify({ Username: username, Pw: password }), @@ -95,7 +95,7 @@ module.exports = function({ authManager, credentialManager, asyncHandler, errorR } case 'plex': { try { - const plexResp = await ctx.fetchT(PLEX.AUTH_URL, { + const plexResp = await fetchT(PLEX.AUTH_URL, { method: 'POST', headers: { 'Accept': 'application/json', 'Content-Type': 'application/json', @@ -127,7 +127,7 @@ module.exports = function({ authManager, credentialManager, asyncHandler, errorR } try { - const resp = await ctx.fetchT(loginUrl, { + const resp = await fetchT(loginUrl, { method: 'POST', headers: { 'Content-Type': contentType, ...extraHeaders }, body: loginBody, redirect: 'manual', diff --git a/dashcaddy-api/server-old.js b/dashcaddy-api/server-old.js deleted file mode 100644 index 0a82307..0000000 --- a/dashcaddy-api/server-old.js +++ /dev/null @@ -1,1997 +0,0 @@ -const express = require('express'); -const crypto = require('crypto'); -const fs = require('fs'); -const fsp = require('fs').promises; -const { exists, readJsonFile, writeJsonFile, readTextFile, isAccessible } = require('./fs-helpers'); -const os = require('os'); -const http = require('http'); -const https = require('https'); -const { execSync } = require('child_process'); -const path = require('path'); -const { - ValidationError, validateFilePath, validateURL, validateToken, - validateServiceConfig, sanitizeString, isValidPort, validateSecurePath -} = require('./input-validator'); -const validatorLib = require('validator'); -const credentialManager = require('./credential-manager'); -const { CACHE_CONFIGS, createCache } = require('./cache-config'); -const { AppError } = require('./errors'); -const { validateConfig } = require('./config-schema'); -const { resolveServiceUrl } = require('./url-resolver'); -const { - APP, APP_PORTS, ARR_SERVICES, TIMEOUTS, RETRIES, - SESSION_TTL, RATE_LIMITS, CADDY, PLEX, LIMITS, - REGEX, DNS_RECORD_TYPES, DOCKER, TAILSCALE, buildMediaAuth, -} = require('./constants'); -const platformPaths = require('./platform-paths'); - -// Image processing for favicon conversion -let sharp, pngToIco; -try { - sharp = require('sharp'); - pngToIco = require('png-to-ico'); -} catch (e) { - log.warn('server', 'Image processing libraries not available - favicon conversion disabled'); -} - -// Docker integration -const Docker = require('dockerode'); -const docker = new Docker(); - -// App templates -const { APP_TEMPLATES, TEMPLATE_CATEGORIES, DIFFICULTY_LEVELS } = require('./app-templates'); -const { RECIPE_TEMPLATES, RECIPE_CATEGORIES } = require('./recipe-templates'); - -// Crypto utilities for credential encryption -const cryptoUtils = require('./crypto-utils'); - -// New feature modules -const resourceMonitor = require('./resource-monitor'); -const backupManager = require('./backup-manager'); -const healthChecker = require('./health-checker'); -const updateManager = require('./update-manager'); -const selfUpdater = require('./self-updater'); -let dockerMaintenance; -try { dockerMaintenance = require('./docker-maintenance'); } catch (_) { logger.warn('[WARN] docker-maintenance module not found, skipped'); } -let logDigest; -try { logDigest = require('./log-digest'); } catch (_) { logger.warn('[WARN] log-digest module not found, skipped'); } -const StateManager = require('./state-manager'); -const auditLogger = require('./audit-logger'); -const portLockManager = require('./port-lock-manager'); -const dockerSecurity = require('./docker-security'); -const authManager = require('./auth-manager'); -const responseHelpers = require('./response-helpers'); -const configureMiddleware = require('./middleware'); -const { validateStartupConfig, syncHealthCheckerServices } = require('./startup-validator'); -const { CSRF_HEADER_NAME } = require('./csrf-protection'); - -// Route modules -const ctx = require('./routes/context'); -const healthRoutes = require('./routes/health'); -const monitoringRoutes = require('./routes/monitoring'); -const updatesRoutes = require('./routes/updates'); -const authRoutes = require('./routes/auth'); -const configRoutes = require('./routes/config'); -const dnsRoutes = require('./routes/dns'); -const notificationRoutes = require('./routes/notifications'); -const containerRoutes = require('./routes/containers'); -const serviceRoutes = require('./routes/services'); -const tailscaleRoutes = require('./routes/tailscale'); -const sitesRoutes = require('./routes/sites'); -const credentialsRoutes = require('./routes/credentials'); -const arrRoutes = require('./routes/arr'); -const appsRoutes = require('./routes/apps'); -const logsRoutes = require('./routes/logs'); -const backupsRoutes = require('./routes/backups'); -const caRoutes = require('./routes/ca'); -const browseRoutes = require('./routes/browse'); -const errorLogsRoutes = require('./routes/errorlogs'); -const licenseRoutes = require('./routes/license'); -const recipesRoutes = require('./routes/recipes'); -const themesRoutes = require('./routes/themes'); -const { LicenseManager } = require('./license-manager'); -const metrics = require('./metrics'); - -const app = express(); -const PORT = APP.PORT; - -// Configuration from environment variables -const CADDYFILE_PATH = process.env.CADDYFILE_PATH || platformPaths.caddyfile; -const CADDY_ADMIN_URL = process.env.CADDY_ADMIN_URL || platformPaths.caddyAdminUrl; -const SERVICES_FILE = process.env.SERVICES_FILE || platformPaths.servicesFile; -const SERVICES_DIR = path.dirname(SERVICES_FILE); -const CONFIG_FILE = process.env.CONFIG_FILE || path.join(SERVICES_DIR, 'config.json'); -const DNS_CREDENTIALS_FILE = process.env.DNS_CREDENTIALS_FILE || path.join(SERVICES_DIR, 'dns-credentials.json'); -const TAILSCALE_CONFIG_FILE = process.env.TAILSCALE_CONFIG_FILE || path.join(SERVICES_DIR, 'tailscale-config.json'); -const NOTIFICATIONS_FILE = process.env.NOTIFICATIONS_FILE || path.join(SERVICES_DIR, 'notifications.json'); -const TOTP_CONFIG_FILE = process.env.TOTP_CONFIG_FILE || path.join(SERVICES_DIR, 'totp-config.json'); -const ERROR_LOG_FILE = process.env.ERROR_LOG_FILE || path.join(__dirname, 'dashcaddy-errors.log'); -const MAX_ERROR_LOG_SIZE = LIMITS.ERROR_LOG_SIZE; -const BROWSE_ROOTS = (process.env.MEDIA_BROWSE_ROOTS || '') - .split(',') - .filter(r => r.includes('=')) - .map(r => { - const eqIndex = r.indexOf('='); - const containerPath = r.slice(0, eqIndex).trim(); - const hostPath = r.slice(eqIndex + 1).trim(); - return { containerPath, hostPath }; - }); - -// State management with file locking (prevents data corruption) -const servicesStateManager = new StateManager(SERVICES_FILE); -const configStateManager = new StateManager(CONFIG_FILE); - -// License manager for premium feature gating -const licenseManager = new LicenseManager(credentialManager, CONFIG_FILE, console); -const LICENSE_SECRET_FILE = process.env.LICENSE_SECRET_FILE || path.join(__dirname, '.license-secret'); -licenseManager.loadSecret(LICENSE_SECRET_FILE); - -// ===== Site configuration loaded from config.json (#5) ===== -// These are read at startup and refreshed on config save. -// All code should use these instead of hardcoded values. -let siteConfig = { tld: '.home', caName: '', dnsServerIp: '', dnsServerPort: CADDY.DEFAULT_DNS_PORT, dashboardHost: '', timezone: 'UTC', dnsServers: {}, configurationType: 'homelab', domain: '', routingMode: 'subdomain' }; - -function loadSiteConfig() { - try { - if (fs.existsSync(CONFIG_FILE)) { - const raw = JSON.parse(fs.readFileSync(CONFIG_FILE, 'utf8')); - - // Validate config and log any issues (log.warn may not be assigned during initial load) - const { valid, errors: configErrors, warnings: configWarnings } = validateConfig(raw); - if (log.warn) { - if (!valid) { - log.warn('config', 'Config validation errors', { errors: configErrors }); - } - for (const w of configWarnings) { - log.warn('config', w); - } - } - - siteConfig.tld = raw.tld || '.home'; - if (!siteConfig.tld.startsWith('.')) siteConfig.tld = '.' + siteConfig.tld; - siteConfig.caName = raw.caName || ''; - siteConfig.dnsServerIp = (raw.dns && raw.dns.ip) || ''; - siteConfig.dnsServerPort = (raw.dns && raw.dns.port) || CADDY.DEFAULT_DNS_PORT; - siteConfig.dashboardHost = raw.dashboardHost || `status${siteConfig.tld}`; - siteConfig.timezone = raw.timezone || 'UTC'; - siteConfig.dnsServers = raw.dnsServers || {}; - siteConfig.configurationType = raw.configurationType || 'homelab'; - siteConfig.domain = raw.domain || ''; - siteConfig.routingMode = raw.routingMode || 'subdomain'; - siteConfig.pylon = raw.pylon || null; // { url, key? } — health check relay - } - } catch (e) { - // log.error may not be assigned yet during initial module load - if (log.error) { - log.error('config', 'Failed to load site config', { error: e.message }); - } - } -} -loadSiteConfig(); - -/** Build a domain from subdomain + configured TLD or public domain */ -function buildDomain(subdomain) { - if (siteConfig.configurationType === 'public' && siteConfig.domain) { - return `${subdomain}.${siteConfig.domain}`; - } - return `${subdomain}${siteConfig.tld}`; -} - -/** Build full service URL (protocol + host + path) for a given subdomain. - * Subdirectory mode: https://example.com/sonarr - * Subdomain mode: https://sonarr.example.com */ -function buildServiceUrl(subdomain) { - if (siteConfig.routingMode === 'subdirectory' && siteConfig.domain) { - return `https://${siteConfig.domain}/${subdomain}`; - } - return `https://${buildDomain(subdomain)}`; -} - -/** Build full Technitium DNS API URL for any server (handles IP vs hostname, port) */ -function buildDnsUrl(server, apiPath, params) { - const protocol = server.match(/^\d+\.\d+\.\d+\.\d+$/) ? 'http' : 'https'; - const port = protocol === 'http' ? `:${CADDY.DEFAULT_DNS_PORT}` : ''; - const qs = params instanceof URLSearchParams ? params.toString() : new URLSearchParams(params).toString(); - return `${protocol}://${server}${port}${apiPath}?${qs}`; -} - -/** Call a Technitium DNS API endpoint and return parsed JSON */ -async function callDns(server, apiPath, params) { - const url = buildDnsUrl(server, apiPath, params); - const response = await fetchT(url, { - method: 'GET', - headers: { 'Accept': 'application/json' }, - agent: httpsAgent - }, TIMEOUTS.HTTP_LONG); - return response.json(); -} - -// ===== Shared Helpers ===== - -/** Fetch with automatic timeout — adds AbortSignal if no signal is present. - * Drop-in replacement for fetch(); safely ignores calls that already have a signal. */ -function fetchT(url, opts = {}, timeoutMs = TIMEOUTS.HTTP_DEFAULT) { - // Caddy admin API rejects Node.js undici fetch due to Sec-Fetch-* headers triggering - // origin checking. Use raw http.request for Caddy admin calls to avoid this. - if (url.includes(':2019')) { - return _httpFetch(url, opts, timeoutMs); - } - if (!opts.signal) { - opts = { ...opts, signal: AbortSignal.timeout(timeoutMs) }; - } - delete opts.timeout; - return fetch(url, opts); -} - -/** Raw http.request wrapper that returns a fetch-like Response for Caddy admin API */ -function _httpFetch(url, opts = {}, timeoutMs = TIMEOUTS.HTTP_DEFAULT) { - return new Promise((resolve, reject) => { - const parsed = new URL(url); - const options = { - hostname: parsed.hostname, - port: parsed.port || 2019, - path: parsed.pathname + parsed.search, - method: (opts.method || 'GET').toUpperCase(), - headers: { ...opts.headers }, - timeout: timeoutMs, - }; - if (opts.body) { - options.headers['Content-Length'] = Buffer.byteLength(opts.body); - } - const MAX_RESPONSE_SIZE = 10 * 1024 * 1024; // 10MB - const req = http.request(options, (res) => { - let data = ''; - let size = 0; - res.on('data', chunk => { - size += chunk.length; - if (size > MAX_RESPONSE_SIZE) { - res.destroy(); - reject(new Error(`Response from ${url} exceeded ${MAX_RESPONSE_SIZE} bytes`)); - return; - } - data += chunk; - }); - res.on('end', () => { - resolve({ - ok: res.statusCode >= 200 && res.statusCode < 300, - status: res.statusCode, - statusText: res.statusMessage, - json: () => Promise.resolve(JSON.parse(data)), - text: () => Promise.resolve(data), - headers: { get: (k) => res.headers[k.toLowerCase()] }, - }); - }); - }); - req.on('timeout', () => { req.destroy(); reject(new Error(`Request to ${url} timed out after ${timeoutMs}ms`)); }); - req.on('error', reject); - if (opts.body) req.write(opts.body); - req.end(); - }); -} - -/** Pull a Docker image with timeout protection */ -function dockerPull(imageName, timeoutMs = DOCKER.TIMEOUT) { - return new Promise((resolve, reject) => { - const timer = setTimeout(() => reject(new Error(`Docker pull timed out after ${timeoutMs / 1000}s: ${imageName}`)), timeoutMs); - docker.pull(imageName, (err, stream) => { - if (err) { clearTimeout(timer); return reject(err); } - docker.modem.followProgress(stream, (err, output) => { - clearTimeout(timer); - if (err) return reject(err); - resolve(output); - }); - }); - }); -} - -// ===== Structured Logging ===== -const LOG_LEVELS = { debug: 0, info: 1, warn: 2, error: 3 }; -const LOG_LEVEL = LOG_LEVELS[process.env.LOG_LEVEL || 'info'] || 1; - -function log(level, context, message, data = {}) { - if (LOG_LEVELS[level] < LOG_LEVEL) return; - const entry = { - t: new Date().toISOString(), - level, - ctx: context, - msg: message, - }; - if (Object.keys(data).length) entry.data = data; - const fn = level === 'error' ? logger.error : level === 'warn' ? logger.warn : logger.info; - fn(JSON.stringify(entry)); -} -log.info = (ctx, msg, data) => log('info', ctx, msg, data); -log.warn = (ctx, msg, data) => log('warn', ctx, msg, data); -log.error = (ctx, msg, data) => log('error', ctx, msg, data); -log.debug = (ctx, msg, data) => log('debug', ctx, msg, data); - -/** Standard error response — always returns { success: false, error, ...extras } */ -function errorResponse(res, statusCode, message, extras = {}) { - return res.status(statusCode).json({ success: false, error: message, ...extras }); -} - -/** Standard success response — always returns { success: true, ...data } */ -function ok(res, data = {}) { - return res.json({ success: true, ...data }); -} - -/** Look up a single service by ID from services.json */ -async function getServiceById(serviceId) { - const services = await servicesStateManager.read(); - return services.find(s => s.id === serviceId) || null; -} - -/** Find a running Docker container by name substring */ -async function findContainerByName(name, opts = { all: false }) { - const containers = await docker.listContainers(opts); - const match = containers.find(c => - c.Names.some(n => n.toLowerCase().includes(name.toLowerCase())) - ); - return match || null; -} - -/** Read config.json with fallback to empty object */ -async function readConfig() { - return readJsonFile(CONFIG_FILE, {}); -} - -/** Save config.json (merges with existing, atomic with locking) */ -async function saveConfig(updates) { - return await configStateManager.update(config => { - return Object.assign(config, updates); - }); -} - -/** - * Resolve a DNS token: use the provided one or auto-refresh. - * @returns {{ token: string }} or throws with 401-appropriate message - */ -async function requireDnsToken(providedToken) { - if (providedToken) return providedToken; - const result = await ensureValidDnsToken(); - if (result.success) return result.token; - const err = new Error('No valid DNS token available. ' + result.error); - err.statusCode = 401; - throw err; -} - -/** Get all host ports currently in use by Docker containers */ -async function getUsedPorts() { - const containers = await docker.listContainers({ all: false }); - const ports = new Set(); - for (const c of containers) { - for (const p of (c.Ports || [])) { - if (p.PublicPort) ports.add(p.PublicPort); - } - } - return ports; -} - -/** - * Atomically read-modify-write the Caddyfile and reload Caddy. - * Uses a mutex to prevent concurrent modifications from clobbering each other. - * Rolls back on reload failure. - * @param {function} modifyFn - receives current content, returns modified content (or null to skip) - * @returns {{ success: boolean, error?: string }} - */ -let _caddyfileLock = Promise.resolve(); -async function modifyCaddyfile(modifyFn) { - let resolve; - const prev = _caddyfileLock; - _caddyfileLock = new Promise(r => { resolve = r; }); - await prev; // wait for any in-flight modification to finish - try { - const original = await readCaddyfile(); - const modified = await modifyFn(original); - if (modified === null || modified === original) { - return { success: false, error: 'No changes to apply' }; - } - await fsp.writeFile(CADDYFILE_PATH, modified, 'utf8'); - try { - await reloadCaddy(modified); - return { success: true }; - } catch (err) { - // Rollback - await fsp.writeFile(CADDYFILE_PATH, original, 'utf8'); - return { success: false, error: safeErrorMessage(err), rolledBack: true }; - } - } finally { - resolve(); - } -} - -/** Read the current Caddyfile content */ -async function readCaddyfile() { - return fsp.readFile(CADDYFILE_PATH, 'utf8'); -} - -// Error logging function with enhanced context tracking -async function logError(context, error, additionalInfo = {}) { - const timestamp = new Date().toISOString(); - - // Extract request context if a request object is provided - const requestContext = {}; - if (additionalInfo.req) { - const req = additionalInfo.req; - const clientIP = req.ip || req.socket?.remoteAddress || ''; - requestContext.requestId = req.id; - requestContext.ip = clientIP; - requestContext.userAgent = req.get('user-agent'); - requestContext.method = req.method; - requestContext.path = req.path; - // Check session validity using ipSessions cache - const session = ipSessions.get(clientIP); - requestContext.sessionValid = session && session.exp > Date.now(); - delete additionalInfo.req; // Remove req from additionalInfo to avoid circular refs - } - - const logEntry = { - timestamp, - context, - ...requestContext, - error: { - message: error.message || error, - stack: error.stack, - code: error.code - }, - ...additionalInfo - }; - - // Format log line with request context - const contextInfo = Object.keys(requestContext).length > 0 - ? `\nRequest Context: ${JSON.stringify(requestContext, null, 2)}` - : ''; - const logLine = `[${timestamp}] ${context}: ${error.message || error}\n${error.stack || ''}${contextInfo}\nAdditional Info: ${JSON.stringify(additionalInfo, null, 2)}\n${'='.repeat(80)}\n`; - - try { - // #7: Rotate log if it exceeds max size - try { - const stats = await fsp.stat(ERROR_LOG_FILE); - if (stats.size > MAX_ERROR_LOG_SIZE) { - const rotated = ERROR_LOG_FILE + '.1'; - if (await exists(rotated)) await fsp.unlink(rotated); - await fsp.rename(ERROR_LOG_FILE, rotated); - } - } catch (_) { /* file may not exist yet */ } - await fsp.appendFile(ERROR_LOG_FILE, logLine); - } catch (e) { - log.error('errorlog', 'Failed to write to error log', { error: e.message }); - } -} - -/** #6: Return a safe error message to the client without leaking internals */ -function safeErrorMessage(error) { - const msg = error.message || String(error); - - // Detect port conflict errors from Docker - const portMatch = msg.match(/exposing port TCP [^:]*:(\d+)/); - if (portMatch || msg.includes('port is already allocated') || msg.includes('ports are not available')) { - const port = portMatch ? portMatch[1] : 'requested'; - return `[DC-200] Port ${port} is already in use. Try a different port in Advanced Options, or stop the service using that port first.`; - } - - // Only expose messages that are clearly user-facing (short, no paths/stack frames) - if (msg.length < 200 && !msg.includes('/') && !msg.includes('\\') && !msg.includes(' at ')) { - return msg; - } - return 'An internal error occurred'; -} - -/** Wrap async route handlers — catches unhandled errors, logs, and returns 500. - * Eliminates try/catch boilerplate from route definitions. - * @param {Function} fn - async (req, res, next) handler - * @param {string} [context] - label for logError (defaults to req.path) - */ -function asyncHandler(fn, context) { - return async (req, res, next) => { - try { - await fn(req, res, next); - } catch (error) { - // Let typed errors (AppError subclasses) propagate to the global error handler - if (error instanceof AppError) { - return next(error); - } - await logError(context || req.path, error); - if (!res.headersSent) { - errorResponse(res, 500, safeErrorMessage(error)); - } - } - }; -} - -/** #4: Validate Docker container IDs (hex SHA256 prefix or name) */ -const CONTAINER_ID_RE = /^[a-zA-Z0-9][a-zA-Z0-9_.\-]{0,127}$/; -function isValidContainerId(id) { - return typeof id === 'string' && CONTAINER_ID_RE.test(id); -} - -// DNS token management - auto-refresh when expired -let dnsToken = process.env.DNS_ADMIN_TOKEN || ''; -let dnsTokenExpiry = null; - -// Per-server token cache for authenticating against specific DNS servers (e.g., for updates) -const dnsServerTokens = createCache(CACHE_CONFIGS.dnsTokens); // LRU cache: serverIp -> { token, expiry } - -// Tailscale configuration cache -let tailscaleConfig = { - enabled: false, - requireAuth: false, // Require Tailscale for dashboard access - allowedTailnet: null, // Restrict to specific tailnet - devices: [], // Cache of known devices - oauthConfigured: false, // true when OAuth credentials are stored - tailnet: null, // tailnet name for API calls (e.g., "example.com" or "-") - syncInterval: 300, // seconds between API syncs (default 5 min) - lastSync: null // ISO timestamp of last successful sync -}; - -// Load Tailscale config from file -async function loadTailscaleConfig() { - try { - if (await exists(TAILSCALE_CONFIG_FILE)) { - const data = await fsp.readFile(TAILSCALE_CONFIG_FILE, 'utf8'); - tailscaleConfig = { ...tailscaleConfig, ...JSON.parse(data) }; - log.info('config', 'Tailscale config loaded', { enabled: tailscaleConfig.enabled }); - } - } catch (e) { - await logError('loadTailscaleConfig', e); - log.warn('config', 'Could not load Tailscale config', { error: e.message }); - } -} -// Save Tailscale config to file -async function saveTailscaleConfig() { - try { - await writeJsonFile(TAILSCALE_CONFIG_FILE, tailscaleConfig); - } catch (e) { - log.error('config', 'Could not save Tailscale config', { error: e.message }); - } -} - -// Check if an IP is a Tailscale IP (100.x.x.x CGNAT range) -function isTailscaleIP(ip) { - if (!ip) return false; - // Tailscale uses 100.64.0.0/10 CGNAT range - const parts = ip.split('.'); - if (parts.length !== 4) return false; - const first = parseInt(parts[0]); - const second = parseInt(parts[1]); - return first === 100 && second >= 64 && second <= 127; -} - -// Get Tailscale status (cached for performance) -const tailscaleStatusCache = createCache(CACHE_CONFIGS.tailscaleStatus); - -async function getTailscaleStatus() { - const cached = tailscaleStatusCache.get('status'); - if (cached) { - return cached; - } - - try { - const output = execSync('tailscale status --json', { encoding: 'utf8', timeout: 5000 }); - const status = JSON.parse(output); - tailscaleStatusCache.set('status', status); - return status; - } catch (e) { - log.warn('config', 'Could not get Tailscale status', { error: e.message }); - return null; - } -} - -// Get the local Tailscale IP -async function getLocalTailscaleIP() { - try { - const status = await getTailscaleStatus(); - if (status && status.Self && status.Self.TailscaleIPs) { - // Return first IPv4 address - return status.Self.TailscaleIPs.find(ip => !ip.includes(':')); - } - } catch (e) { - log.warn('config', 'Could not get local Tailscale IP', { error: e.message }); - } - return null; -} - -// ── Tailscale OAuth 2.0 Client Credentials ── -let _tsTokenCache = { token: null, expiresAt: 0 }; - -async function getTailscaleAccessToken() { - // Return cached token if still valid (with 60s buffer) - if (_tsTokenCache.token && Date.now() < _tsTokenCache.expiresAt - 60000) { - return _tsTokenCache.token; - } - - const clientId = await credentialManager.retrieve('tailscale.oauth.client_id'); - const clientSecret = await credentialManager.retrieve('tailscale.oauth.client_secret'); - if (!clientId || !clientSecret) return null; - - const res = await fetch(TAILSCALE.OAUTH_TOKEN_URL, { - method: 'POST', - headers: { 'Content-Type': 'application/x-www-form-urlencoded' }, - body: `client_id=${encodeURIComponent(clientId)}&client_secret=${encodeURIComponent(clientSecret)}&grant_type=client_credentials` - }); - - if (!res.ok) { - log.error('tailscale', 'OAuth token exchange failed', { status: res.status }); - _tsTokenCache = { token: null, expiresAt: 0 }; - return null; - } - - const data = await res.json(); - _tsTokenCache = { - token: data.access_token, - expiresAt: Date.now() + (data.expires_in || 3600) * 1000 - }; - return data.access_token; -} - -// Sync device list from Tailscale API (richer than local CLI) -async function syncFromTailscaleAPI() { - const token = await getTailscaleAccessToken(); - const tailnet = tailscaleConfig.tailnet; - if (!token || !tailnet) return null; - - const res = await fetch(`${TAILSCALE.API_BASE}/tailnet/${encodeURIComponent(tailnet)}/devices`, { - headers: { Authorization: `Bearer ${token}` } - }); - if (!res.ok) throw new Error(`Tailscale API: HTTP ${res.status}`); - - const data = await res.json(); - - const devices = (data.devices || []).map(d => ({ - id: d.id, - name: d.name, - hostname: d.hostname, - addresses: d.addresses || [], - ip: (d.addresses || []).find(a => !a.includes(':')) || null, - os: d.os, - user: d.user, - authorized: d.authorized, - tags: d.tags || [], - lastSeen: d.lastSeen, - clientVersion: d.clientVersion, - isExternal: d.isExternal || false - })); - - tailscaleConfig.devices = devices; - tailscaleConfig.lastSync = new Date().toISOString(); - await saveTailscaleConfig(); - - return devices; -} - -let _tsSyncInterval = null; - -function startTailscaleSyncTimer() { - if (_tsSyncInterval) clearInterval(_tsSyncInterval); - const interval = (tailscaleConfig.syncInterval || 300) * 1000; - _tsSyncInterval = setInterval(async () => { - try { - await syncFromTailscaleAPI(); - log.debug('tailscale', 'API sync completed', { deviceCount: tailscaleConfig.devices.length }); - } catch (error) { - log.warn('tailscale', 'API sync failed', { error: error.message }); - } - }, interval); - log.info('tailscale', 'API sync enabled', { interval: interval / 1000 + 's' }); -} - -function stopTailscaleSyncTimer() { - if (_tsSyncInterval) { - clearInterval(_tsSyncInterval); - _tsSyncInterval = null; - } -} - -// TOTP authentication configuration -let totpConfig = { - enabled: false, - sessionDuration: 'never', // 'never' = disabled, or '15m','30m','1h','2h','4h','8h','12h','24h' - isSetUp: false // true once a secret has been verified -}; - -async function loadTotpConfig() { - try { - if (await exists(TOTP_CONFIG_FILE)) { - const data = await fsp.readFile(TOTP_CONFIG_FILE, 'utf8'); - const loaded = JSON.parse(data); - // Never load secret from file — it belongs only in credential-manager - delete loaded.secret; - Object.assign(totpConfig, loaded); - log.info('config', 'TOTP config loaded', { enabled: totpConfig.enabled }); - } - } catch (e) { - await logError('loadTotpConfig', e); - log.warn('config', 'Could not load TOTP config', { error: e.message }); - } -} - -async function saveTotpConfig() { - try { - await writeJsonFile(TOTP_CONFIG_FILE, totpConfig); - } catch (e) { - log.error('config', 'Could not save TOTP config', { error: e.message }); - } -} - -// Load config on startup (async — resolved before server starts listening) -const _configsReady = (async () => { - await loadTailscaleConfig(); - await loadTotpConfig(); -})(); - -// ===== NOTIFICATION SERVICE ===== - -// Notification configuration -let notificationConfig = { - enabled: false, - providers: { - discord: { enabled: false, webhookUrl: '' }, - telegram: { enabled: false, botToken: '', chatId: '' }, - ntfy: { enabled: false, serverUrl: 'https://ntfy.sh', topic: '' } - }, - events: { - containerDown: true, - containerUp: true, - deploymentSuccess: true, - deploymentFailed: true, - serviceError: true - }, - healthCheck: { - enabled: false, - intervalMinutes: 5, - lastCheck: null - } -}; - -// Notification history (in-memory, last 100 entries) -let notificationHistory = []; -const MAX_NOTIFICATION_HISTORY = 100; - -// Load notification config from file (with decryption of sensitive fields) -async function loadNotificationConfig() { - try { - if (await exists(NOTIFICATIONS_FILE)) { - const data = await fsp.readFile(NOTIFICATIONS_FILE, 'utf8'); - const loaded = JSON.parse(data); - - // Decrypt sensitive fields if encrypted - if (loaded._encrypted && loaded.providers) { - if (loaded.providers.discord?.webhookUrl && cryptoUtils.isEncrypted(loaded.providers.discord.webhookUrl)) { - loaded.providers.discord.webhookUrl = cryptoUtils.decrypt(loaded.providers.discord.webhookUrl); - } - if (loaded.providers.telegram?.botToken && cryptoUtils.isEncrypted(loaded.providers.telegram.botToken)) { - loaded.providers.telegram.botToken = cryptoUtils.decrypt(loaded.providers.telegram.botToken); - } - delete loaded._encrypted; - } - - notificationConfig = { ...notificationConfig, ...loaded }; - log.info('config', 'Notification config loaded', { enabled: notificationConfig.enabled }); - } - } catch (e) { - await logError('loadNotificationConfig', e); - log.warn('config', 'Could not load notification config', { error: e.message }); - } -} - -// Save notification config to file (with encryption of sensitive fields) -async function saveNotificationConfig() { - try { - // Create a copy for encryption - const toSave = JSON.parse(JSON.stringify(notificationConfig)); - - // Encrypt sensitive fields - if (toSave.providers) { - if (toSave.providers.discord?.webhookUrl) { - toSave.providers.discord.webhookUrl = cryptoUtils.encrypt(toSave.providers.discord.webhookUrl); - } - if (toSave.providers.telegram?.botToken) { - toSave.providers.telegram.botToken = cryptoUtils.encrypt(toSave.providers.telegram.botToken); - } - } - toSave._encrypted = true; - - await fsp.writeFile(NOTIFICATIONS_FILE, JSON.stringify(toSave, null, 2), 'utf8'); - log.info('config', 'Notification config saved (encrypted)'); - } catch (e) { - await logError('saveNotificationConfig', e); - log.error('config', 'Could not save notification config', { error: e.message }); - } -} - -// Add to notification history -function addNotificationToHistory(notification) { - notificationHistory.unshift({ - ...notification, - timestamp: new Date().toISOString() - }); - if (notificationHistory.length > MAX_NOTIFICATION_HISTORY) { - notificationHistory = notificationHistory.slice(0, MAX_NOTIFICATION_HISTORY); - } -} - -// Send notification via Discord webhook -async function sendDiscordNotification(title, message, type = 'info') { - const { webhookUrl } = notificationConfig.providers.discord; - if (!webhookUrl) return { success: false, error: 'No webhook URL configured' }; - - const colors = { - success: 0x00ff00, // Green - error: 0xff0000, // Red - warning: 0xffff00, // Yellow - info: 0x0099ff // Blue - }; - - const payload = { - embeds: [{ - title: `DashCaddy: ${title}`, - description: message, - color: colors[type] || colors.info, - timestamp: new Date().toISOString(), - footer: { text: 'DashCaddy Notifications' } - }] - }; - - try { - const response = await fetchT(webhookUrl, { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify(payload) - }); - - if (!response.ok) { - throw new Error(`Discord API returned ${response.status}`); - } - - return { success: true }; - } catch (error) { - await logError('sendDiscordNotification', error); - return { success: false, error: error.message }; - } -} - -// Send notification via Telegram bot -async function sendTelegramNotification(title, message, type = 'info') { - const { botToken, chatId } = notificationConfig.providers.telegram; - if (!botToken || !chatId) return { success: false, error: 'Bot token or chat ID not configured' }; - - const emoji = { - success: '✅', - error: '❌', - warning: '⚠️', - info: 'ℹ️' - }; - - const text = `${emoji[type] || emoji.info} *DashCaddy: ${title}*\n\n${message}`; - - try { - const response = await fetchT(`https://api.telegram.org/bot${botToken}/sendMessage`, { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body: JSON.stringify({ - chat_id: chatId, - text: text, - parse_mode: 'Markdown' - }) - }); - - const result = await response.json(); - if (!result.ok) { - throw new Error(result.description || 'Telegram API error'); - } - - return { success: true }; - } catch (error) { - await logError('sendTelegramNotification', error); - return { success: false, error: error.message }; - } -} - -// Send notification via ntfy.sh -async function sendNtfyNotification(title, message, type = 'info') { - const { serverUrl, topic } = notificationConfig.providers.ntfy; - if (!topic) return { success: false, error: 'No topic configured' }; - - const priority = { - success: 3, // default - error: 5, // max - warning: 4, // high - info: 3 // default - }; - - const tags = { - success: 'white_check_mark', - error: 'x', - warning: 'warning', - info: 'information_source' - }; - - try { - const response = await fetchT(`${serverUrl}/${topic}`, { - method: 'POST', - headers: { - 'Title': `DashCaddy: ${title}`, - 'Priority': String(priority[type] || 3), - 'Tags': tags[type] || 'information_source' - }, - body: message - }); - - if (!response.ok) { - throw new Error(`ntfy returned ${response.status}`); - } - - return { success: true }; - } catch (error) { - await logError('sendNtfyNotification', error); - return { success: false, error: error.message }; - } -} - -// Send notification to all enabled providers -async function sendNotification(event, title, message, type = 'info') { - if (!notificationConfig.enabled) { - return { sent: false, reason: 'Notifications disabled' }; - } - - // Check if this event type is enabled - if (notificationConfig.events[event] === false) { - return { sent: false, reason: `Event type '${event}' is disabled` }; - } - - const results = {}; - const providers = notificationConfig.providers; - - if (providers.discord?.enabled) { - results.discord = await sendDiscordNotification(title, message, type); - } - - if (providers.telegram?.enabled) { - results.telegram = await sendTelegramNotification(title, message, type); - } - - if (providers.ntfy?.enabled) { - results.ntfy = await sendNtfyNotification(title, message, type); - } - - // Log to history - addNotificationToHistory({ - event, - title, - message, - type, - results - }); - - return { sent: true, results }; -} - -// Container health monitoring state -let containerHealthState = {}; -let healthCheckInterval = null; - -// Check container health and send notifications -async function checkContainerHealth() { - if (!notificationConfig.enabled || !notificationConfig.healthCheck?.enabled) { - return; - } - - try { - const containers = await docker.listContainers({ all: true }); - const services = (await exists(SERVICES_FILE)) - ? await servicesStateManager.read() - : []; - - // Create a map of container IDs to service names - const serviceMap = {}; - for (const service of services) { - if (service.containerId) { - serviceMap[service.containerId] = service.name || service.id; - } - } - - for (const container of containers) { - const containerId = container.Id; - const containerName = container.Names?.[0]?.replace(/^\//, '') || containerId.slice(0, 12); - const serviceName = serviceMap[containerId] || containerName; - const isRunning = container.State === 'running'; - const previousState = containerHealthState[containerId]; - - // Detect state changes - if (previousState !== undefined && previousState !== isRunning) { - if (isRunning) { - // Container came back up - await sendNotification( - 'containerUp', - 'Container Recovered', - `**${serviceName}** is now running again.`, - 'success' - ); - } else { - // Container went down - await sendNotification( - 'containerDown', - 'Container Down', - `**${serviceName}** has stopped running.\nStatus: ${container.Status}`, - 'error' - ); - } - } - - containerHealthState[containerId] = isRunning; - } - - // Update last check time - notificationConfig.healthCheck.lastCheck = new Date().toISOString(); - } catch (error) { - await logError('checkContainerHealth', error); - } -} - -// Start health check daemon -function startHealthCheckDaemon() { - if (healthCheckInterval) { - clearInterval(healthCheckInterval); - } - - if (!notificationConfig.healthCheck?.enabled) { - log.info('health', 'Health check daemon disabled'); - return; - } - - const intervalMs = (notificationConfig.healthCheck.intervalMinutes || 5) * 60 * 1000; - log.info('health', 'Starting health check daemon', { intervalMinutes: notificationConfig.healthCheck.intervalMinutes }); - - // Initial check - checkContainerHealth(); - - // Periodic checks - healthCheckInterval = setInterval(checkContainerHealth, intervalMs); -} - -// Stop health check daemon -function stopHealthCheckDaemon() { - if (healthCheckInterval) { - clearInterval(healthCheckInterval); - healthCheckInterval = null; - log.info('health', 'Health check daemon stopped'); - } -} - -// Load notification config on startup (async — resolved before server starts listening) -const _notificationsReady = (async () => { - await loadNotificationConfig(); - // Start health check if enabled - if (notificationConfig.healthCheck?.enabled) { - startHealthCheckDaemon(); - } -})(); - -// HTTPS agent for internal Caddy CA — load cert if available, keep system CAs too -const CA_CERT_PATH = process.env.CA_CERT_PATH || '/app/pki/root.crt'; -let httpsAgent; -try { - const caCert = fs.readFileSync(CA_CERT_PATH); - httpsAgent = new https.Agent({ ca: [...require('tls').rootCertificates, caCert] }); - log.info('server', 'HTTPS agent configured with CA certificate + system CAs', { path: CA_CERT_PATH }); -} catch { - httpsAgent = new https.Agent(); - log.error('server', 'CA cert not found — HTTPS calls to internal services may fail', { path: CA_CERT_PATH }); -} - -// ── Configure middleware stack (CORS, auth, rate limiting, etc.) ── -const middlewareResult = configureMiddleware(app, { - siteConfig, totpConfig, tailscaleConfig, - metrics, auditLogger, authManager, log, cryptoUtils, - isValidContainerId, isTailscaleIP, getTailscaleStatus, - RATE_LIMITS, LIMITS, APP, CACHE_CONFIGS, createCache -}); - -const { - strictLimiter, SESSION_DURATIONS, ipSessions, - getClientIP, createIPSession, setSessionCookie, - clearIPSession, clearSessionCookie, isSessionValid -} = middlewareResult; - -// ── Populate route context and mount extracted route modules ── - -// Namespaced groups -Object.assign(ctx.docker, { - client: docker, - pull: dockerPull, - findContainer: findContainerByName, - getUsedPorts, - security: dockerSecurity, -}); -Object.assign(ctx.caddy, { - modify: modifyCaddyfile, - read: readCaddyfile, - reload: reloadCaddy, - generateConfig: generateCaddyConfig, - verifySite: verifySiteAccessible, - adminUrl: CADDY_ADMIN_URL, - filePath: CADDYFILE_PATH, -}); -Object.assign(ctx.dns, { - call: callDns, - buildUrl: buildDnsUrl, - requireToken: requireDnsToken, - ensureToken: ensureValidDnsToken, - createRecord: createDnsRecord, - getToken: () => dnsToken, - setToken: (t) => { dnsToken = t; }, - getTokenExpiry: () => dnsTokenExpiry, - setTokenExpiry: (e) => { dnsTokenExpiry = e; }, - getTokenForServer, - invalidateTokenForServer: (serverIp) => { dnsServerTokens.delete(`${serverIp}:readonly`); dnsServerTokens.delete(`${serverIp}:admin`); }, - refresh: refreshDnsToken, - credentialsFile: DNS_CREDENTIALS_FILE, -}); -Object.assign(ctx.session, { - ipSessions, - durations: SESSION_DURATIONS, - getClientIP, - create: createIPSession, - setCookie: setSessionCookie, - clear: clearIPSession, - clearCookie: clearSessionCookie, - isValid: isSessionValid, -}); -Object.assign(ctx.notification, { - getConfig: () => notificationConfig, - saveConfig: saveNotificationConfig, - send: sendNotification, - sendDiscord: sendDiscordNotification, - sendTelegram: sendTelegramNotification, - sendNtfy: sendNtfyNotification, - getHistory: () => notificationHistory, - clearHistory: () => { notificationHistory = []; }, - startHealthDaemon: startHealthCheckDaemon, - stopHealthDaemon: stopHealthCheckDaemon, - checkHealth: checkContainerHealth, - getHealthState: () => containerHealthState, -}); -Object.assign(ctx.tailscale, { - config: tailscaleConfig, - save: saveTailscaleConfig, - getStatus: getTailscaleStatus, - getLocalIP: getLocalTailscaleIP, - isTailscaleIP, - getAccessToken: getTailscaleAccessToken, - syncAPI: syncFromTailscaleAPI, - startSync: startTailscaleSyncTimer, - stopSync: stopTailscaleSyncTimer, -}); - -// Flat properties (shared across domains) -Object.assign(ctx, { - app, siteConfig, servicesStateManager, configStateManager, - credentialManager, authManager, licenseManager, - healthChecker, updateManager, backupManager, resourceMonitor, - auditLogger, portLockManager, selfUpdater, dockerMaintenance, logDigest, - APP_TEMPLATES, TEMPLATE_CATEGORIES, DIFFICULTY_LEVELS, RECIPE_TEMPLATES, RECIPE_CATEGORIES, - asyncHandler, errorResponse, ok, fetchT, log, logError, safeErrorMessage, - buildDomain, buildServiceUrl, getServiceById, readConfig, saveConfig, addServiceToConfig, - validateURL, strictLimiter, - totpConfig, saveTotpConfig, - loadSiteConfig, loadNotificationConfig, - loadDnsCredentials: () => {}, - SERVICES_FILE, CONFIG_FILE, TOTP_CONFIG_FILE, TAILSCALE_CONFIG_FILE, - NOTIFICATIONS_FILE, ERROR_LOG_FILE, - resyncHealthChecker: () => syncHealthCheckerServices({ log, SERVICES_FILE, servicesStateManager, healthChecker, buildServiceUrl, siteConfig, APP }), -}); - -// Build versioned API router — all route modules attach here -const apiRouter = express.Router(); -apiRouter.use(authRoutes(ctx)); -apiRouter.use(configRoutes(ctx)); -apiRouter.use('/dns', dnsRoutes({ - dns: ctx.dns, - siteConfig: ctx.siteConfig, - asyncHandler: ctx.asyncHandler, - log: ctx.log, - safeErrorMessage: ctx.safeErrorMessage, - fetchT: ctx.fetchT, - credentialManager: ctx.credentialManager -})); -apiRouter.use('/notifications', notificationRoutes(ctx)); -apiRouter.use('/containers', containerRoutes({ - docker: ctx.docker, - log: ctx.log, - asyncHandler: ctx.asyncHandler -})); -apiRouter.use(serviceRoutes({ - servicesStateManager: ctx.servicesStateManager, - credentialManager: ctx.credentialManager, - siteConfig: ctx.siteConfig, - buildServiceUrl: ctx.buildServiceUrl, - buildDomain: ctx.buildDomain, - fetchT: ctx.fetchT, - asyncHandler: ctx.asyncHandler, - SERVICES_FILE: ctx.SERVICES_FILE, - log: ctx.log, - safeErrorMessage: ctx.safeErrorMessage, - resyncHealthChecker: ctx.resyncHealthChecker, - caddy: ctx.caddy, - dns: ctx.dns -})); -apiRouter.use(healthRoutes({ - fetchT: ctx.fetchT, - SERVICES_FILE: ctx.SERVICES_FILE, - servicesStateManager: ctx.servicesStateManager, - siteConfig: ctx.siteConfig, - buildServiceUrl: ctx.buildServiceUrl, - asyncHandler: ctx.asyncHandler, - logError: ctx.logError, - healthChecker: ctx.healthChecker -})); -apiRouter.use(monitoringRoutes({ - resourceMonitor: ctx.resourceMonitor, - docker: ctx.docker, - asyncHandler: ctx.asyncHandler -})); -apiRouter.use(updatesRoutes(ctx)); -apiRouter.use('/tailscale', tailscaleRoutes(ctx)); -apiRouter.use(sitesRoutes(ctx)); -apiRouter.use(credentialsRoutes({ - credentialManager: ctx.credentialManager, - asyncHandler: ctx.asyncHandler -})); -apiRouter.use(arrRoutes(ctx)); -apiRouter.use(appsRoutes(ctx)); -apiRouter.use(logsRoutes(ctx)); -apiRouter.use(backupsRoutes({ - backupManager: ctx.backupManager, - asyncHandler: ctx.asyncHandler -})); -apiRouter.use('/ca', caRoutes(ctx)); -apiRouter.use(browseRoutes(ctx)); -apiRouter.use(errorLogsRoutes({ - ERROR_LOG_FILE: ctx.ERROR_LOG_FILE, - auditLogger: ctx.auditLogger, - asyncHandler: ctx.asyncHandler -})); -apiRouter.use('/license', licenseRoutes({ - licenseManager: ctx.licenseManager, - asyncHandler: ctx.asyncHandler -})); -apiRouter.use('/recipes', recipesRoutes(ctx)); -apiRouter.use(themesRoutes({ asyncHandler })); - -// Inline routes on the API router -apiRouter.get('/health', (req, res) => { - res.json({ status: 'ok', timestamp: new Date().toISOString() }); -}); -apiRouter.get('/csrf-token', (req, res) => { - res.json({ success: true, token: req.csrfToken, headerName: CSRF_HEADER_NAME }); -}); -apiRouter.get('/metrics', (req, res) => { - res.json({ success: true, metrics: metrics.getSummary() }); -}); - -// Mount at /api/v1 (canonical) and /api (legacy compat for tests & external consumers) -app.use('/api/v1', apiRouter); -app.use('/api', apiRouter); - -// Root-level health check (no /api prefix) -app.get('/health', (req, res) => { - res.json({ status: 'ok', timestamp: new Date().toISOString() }); -}); - -// Lightweight probe endpoint - performs real health checks for frontend status dots -app.get('/probe/:id', asyncHandler(async (req, res) => { - const id = req.params.id; - - try { - // Look up service in services.json - let service = null; - if (id !== 'internet' && await exists(SERVICES_FILE)) { - const data = await servicesStateManager.read(); - const services = Array.isArray(data) ? data : data.services || []; - service = services.find(s => s.id === id); - } - - const url = resolveServiceUrl(id, service, siteConfig, buildServiceUrl); - - const parsed = new URL(url); - const isHttps = parsed.protocol === 'https:'; - const lib = isHttps ? https : http; - - const options = { - hostname: parsed.hostname, - port: parsed.port || (isHttps ? 443 : 80), - path: parsed.pathname + parsed.search, - method: 'HEAD', - timeout: 5000, - agent: isHttps ? httpsAgent : undefined, - headers: { 'User-Agent': APP.USER_AGENTS.PROBE }, - }; - - const makeRequest = (method) => new Promise((resolve, reject) => { - const reqOpts = { ...options, method }; - const probeReq = lib.request(reqOpts, (response) => { - response.resume(); - resolve(response.statusCode); - }); - probeReq.on('error', reject); - probeReq.on('timeout', () => { probeReq.destroy(); reject(new Error('Timeout')); }); - probeReq.end(); - }); - - let statusCode; - try { - statusCode = await makeRequest('HEAD'); - // Fall back to GET if HEAD is not supported - if (statusCode === 501 || statusCode === 405) { - statusCode = await makeRequest('GET'); - } - } catch { - // Direct URL failed (e.g. HTTP/1.0 parse errors) — try via Caddy domain - const fallbackUrl = `https://${buildDomain(id)}`; - const fp = new URL(fallbackUrl); - const fLib = require('https'); - statusCode = await new Promise((resolve, reject) => { - const fReq = fLib.request({ - hostname: fp.hostname, port: 443, path: '/', method: 'GET', - timeout: 5000, agent: httpsAgent, - headers: { 'User-Agent': APP.USER_AGENTS.PROBE } - }, (fRes) => { fRes.resume(); resolve(fRes.statusCode); }); - fReq.on('error', reject); - fReq.on('timeout', () => { fReq.destroy(); reject(new Error('Timeout')); }); - fReq.end(); - }); - } - - res.status(statusCode).send(); - } catch { - res.status(502).send(); - } -}, 'probe')); - -// Get network IPs (LAN, Tailscale) for quick selection -app.get('/api/network/ips', (req, res) => { - try { - // Prefer environment variables (set in docker-compose.yml) - const envLan = process.env.HOST_LAN_IP; - const envTailscale = process.env.HOST_TAILSCALE_IP; - - const result = { - localhost: '127.0.0.1', - lan: envLan || null, - tailscale: envTailscale || null, - all: [] - }; - - // If env vars not set, try to detect from network interfaces - if (!envLan || !envTailscale) { - const interfaces = os.networkInterfaces(); - - for (const [name, addrs] of Object.entries(interfaces)) { - for (const addr of addrs) { - // Skip internal and IPv6 - if (addr.internal || addr.family !== 'IPv4') continue; - - const ip = addr.address; - result.all.push({ name, ip }); - - // Detect Tailscale (100.x.x.x range) - if (!result.tailscale && ip.startsWith('100.')) { - result.tailscale = ip; - } - // Detect common LAN ranges - else if (!result.lan && (ip.startsWith('192.168.') || ip.startsWith('10.') || ip.match(/^172\.(1[6-9]|2[0-9]|3[0-1])\./))) { - result.lan = ip; - } - } - } - } - - // Return null if not detected — let the frontend handle it - if (!result.lan) result.lan = null; - if (!result.tailscale) result.tailscale = null; - - res.json(result); - } catch (error) { - errorResponse(res, 500, safeErrorMessage(error)); - } -}); - - -// (TOTP/auth inline routes moved to routes/auth.js) - -// (SSO auth gate + getAppSession moved to routes/auth.js) - -// (Tailscale routes moved to routes/tailscale.js) - -// (Caddy/site routes moved to routes/sites.js) - -// (Assets, config, backup routes moved to routes/config.js) - - -// (Credential management routes moved to routes/credentials.js) - -// ===== DNS TOKEN AUTO-REFRESH FUNCTIONS ===== - -async function refreshDnsToken(username, password, server) { - try { - // Use /api/user/login to get a session token - const params = new URLSearchParams({ - user: username, - pass: password, - includeInfo: 'false' - }); - - const response = await fetchT( - `http://${server}:5380/api/user/login?${params.toString()}`, - { - method: 'POST', - headers: { - 'Accept': 'application/json', - 'Content-Type': 'application/x-www-form-urlencoded' - }, - timeout: 10000 - } - ); - - const result = await response.json(); - - if (result.status === 'ok' && result.token) { - dnsToken = result.token; - // Technitium session tokens expire after inactivity, refresh every 6 hours to keep alive - dnsTokenExpiry = new Date(Date.now() + SESSION_TTL.DNS_TOKEN).toISOString(); - log.info('dns', 'DNS token refreshed', { expires: dnsTokenExpiry }); - return { success: true, token: dnsToken }; - } - - return { success: false, error: result.errorMessage || 'Login failed' }; - } catch (error) { - log.error('dns', 'DNS token refresh error', { error: error.message }); - return { success: false, error: error.message }; - } -} - -async function ensureValidDnsToken() { - // Check if token is valid and not expired - if (dnsToken && dnsTokenExpiry && new Date() < new Date(dnsTokenExpiry)) { - return { success: true, token: dnsToken }; - } - - // Try per-server admin credentials for the primary DNS server - const primaryIp = siteConfig.dnsServerIp; - if (primaryIp) { - const dnsId = dnsIpToDnsId(primaryIp); - if (dnsId) { - // Try admin credentials first (used for DNS record operations) - for (const role of ['admin', 'readonly']) { - try { - const username = await credentialManager.retrieve(`dns.${dnsId}.${role}.username`); - const password = await credentialManager.retrieve(`dns.${dnsId}.${role}.password`); - if (username && password) { - return await refreshDnsToken(username, password, primaryIp); - } - } catch (err) { - log.error('dns', `Per-server ${role} credential error for primary DNS`, { dnsId, error: err.message }); - } - } - } - } - - // Fall back to global credentials - try { - const username = await credentialManager.retrieve('dns.username'); - const password = await credentialManager.retrieve('dns.password'); - const server = await credentialManager.retrieve('dns.server'); - if (username && password) { - return await refreshDnsToken(username, password, server || primaryIp); - } - } catch (err) { - log.error('dns', 'Credential manager error', { error: err.message }); - } - - return { - success: false, - error: 'No DNS credentials configured. Please set up credentials via /api/dns/credentials' - }; -} - -// Map a DNS server IP to its dnsId (dns1, dns2, dns3) using config -function dnsIpToDnsId(serverIp) { - for (const [dnsId, info] of Object.entries(siteConfig.dnsServers || {})) { - if (info.ip === serverIp) return dnsId; - } - return null; -} - -// Get a valid token for a specific DNS server (authenticates directly against that server) -// role: 'readonly' for logs, 'admin' for records/restart/updates (defaults to 'readonly') -async function getTokenForServer(targetServer, role = 'readonly') { - const cacheKey = `${targetServer}:${role}`; - - // Check cached per-server token first - const cached = dnsServerTokens.get(cacheKey); - if (cached && cached.token && cached.expiry && new Date() < new Date(cached.expiry)) { - return { success: true, token: cached.token }; - } - - const serverPort = siteConfig.dnsServerPort || '5380'; - - // Helper to authenticate against a DNS server via login - async function authenticateToServer(username, password) { - const params = new URLSearchParams({ - user: username, - pass: password, - includeInfo: 'false' - }); - - const response = await fetchT( - `http://${targetServer}:${serverPort}/api/user/login?${params.toString()}`, - { - method: 'POST', - headers: { - 'Accept': 'application/json', - 'Content-Type': 'application/x-www-form-urlencoded' - } - } - ); - - const result = await response.json(); - - if (result.status === 'ok' && result.token) { - dnsServerTokens.set(cacheKey, { - token: result.token, - expiry: new Date(Date.now() + SESSION_TTL.DNS_TOKEN).toISOString() - }); - log.info('dns', 'DNS token obtained for server', { server: targetServer, role }); - return { success: true, token: result.token }; - } - - return { success: false, error: result.errorMessage || 'Login failed' }; - } - - const dnsId = dnsIpToDnsId(targetServer); - - // Try per-server credentials with the requested role first - if (dnsId) { - try { - const username = await credentialManager.retrieve(`dns.${dnsId}.${role}.username`); - const password = await credentialManager.retrieve(`dns.${dnsId}.${role}.password`); - if (username && password) { - return await authenticateToServer(username, password); - } - } catch (err) { - log.error('dns', `Per-server ${role} credential error`, { dnsId, server: targetServer, error: err.message }); - } - - // Fall back to the other role (readonly -> admin or admin -> readonly) - const fallbackRole = role === 'readonly' ? 'admin' : 'readonly'; - try { - const username = await credentialManager.retrieve(`dns.${dnsId}.${fallbackRole}.username`); - const password = await credentialManager.retrieve(`dns.${dnsId}.${fallbackRole}.password`); - if (username && password) { - return await authenticateToServer(username, password); - } - } catch (err) { - // ignore fallback errors - } - } - - // Fall back to global credentials - try { - const username = await credentialManager.retrieve('dns.username'); - const password = await credentialManager.retrieve('dns.password'); - if (username && password) { - return await authenticateToServer(username, password); - } - } catch (err) { - log.error('dns', 'Credential manager error', { server: targetServer, error: err.message }); - } - - return { success: false, error: 'No DNS credentials configured' }; -} - -// Load credentials and refresh token on startup -(async function initDnsToken() { - if (dnsToken) { - log.info('dns', 'Using DNS token from environment variable'); - return; - } - - // Get token using credential manager - const result = await ensureValidDnsToken(); - if (result.success) { - log.info('dns', 'DNS token obtained from stored credentials'); - } else if (await credentialManager.retrieve('dns.username')) { - log.warn('dns', 'Failed to get DNS token', { error: result.error }); - } else { - log.info('dns', 'No DNS credentials configured - DNS record management unavailable'); - } -})(); - -// (Arr stack routes moved to routes/arr.js) -// (App deployment routes moved to routes/apps.js) -// (Container management routes moved to routes/containers.js) -// (Docker helper functions moved to routes/apps.js) - -function generateCaddyConfig(subdomain, ip, port, options = {}) { - const { tailscaleOnly = false, allowedIPs = [], subpathSupport = 'strip' } = options; - - // Subdirectory mode: generate handle/handle_path block (injected into main domain block) - if (siteConfig.routingMode === 'subdirectory' && siteConfig.domain) { - let config = ''; - - // Native-support apps: use handle (preserve path prefix) - // Strip-mode apps: use handle_path (remove path prefix before proxying) - if (subpathSupport === 'native') { - config += `\tredir /${subdomain} /${subdomain}/ permanent\n`; - config += `\thandle /${subdomain}/* {\n`; - } else { - config += `\thandle_path /${subdomain}/* {\n`; - } - - if (tailscaleOnly) { - config += `\t\t@blocked not remote_ip 100.64.0.0/10`; - if (allowedIPs.length > 0) config += ` ${allowedIPs.join(' ')}`; - config += `\n\t\trespond @blocked "Access denied. Tailscale connection required." 403\n`; - } - - config += `\t\treverse_proxy ${ip}:${port}\n`; - config += `\t}`; - return config; - } - - // Subdomain mode (default): standalone domain block - let config = `${buildDomain(subdomain)} {\n`; - - if (tailscaleOnly) { - config += ` @blocked not remote_ip 100.64.0.0/10`; - if (allowedIPs.length > 0) { - config += ` ${allowedIPs.join(' ')}`; - } - config += `\n respond @blocked "Access denied. Tailscale connection required." 403\n`; - } - - config += ` reverse_proxy ${ip}:${port}\n`; - config += ` tls internal\n`; - config += `}`; - - return config; -} - -// (generateStaticSiteConfig, addCaddyConfig, pullImage moved to routes/apps.js) - -async function reloadCaddy(content) { - const maxRetries = RETRIES.CADDY_RELOAD; - let lastError = null; - - for (let i = 0; i < maxRetries; i++) { - try { - const response = await fetchT(`${CADDY_ADMIN_URL}/load`, { - method: 'POST', - headers: { 'Content-Type': CADDY.CONTENT_TYPE }, - body: content - }); - - if (response.ok) { - log.info('caddy', 'Caddy configuration reloaded successfully'); - // Wait a moment for Caddy to fully apply the config - await new Promise(resolve => setTimeout(resolve, 1000)); - return; - } - - lastError = await response.text(); - log.warn('caddy', 'Caddy reload attempt failed', { attempt: i + 1, error: lastError }); - } catch (error) { - lastError = error.message; - log.warn('caddy', 'Caddy reload attempt error', { attempt: i + 1, error: lastError }); - } - - if (i < maxRetries - 1) { - await new Promise(resolve => setTimeout(resolve, 2000)); - } - } - - throw new Error(`[DC-303] Caddy reload failed after ${maxRetries} attempts: ${lastError}`); -} - -async function verifySiteAccessible(domain, maxAttempts = 5) { - const delay = 2000; - - for (let i = 0; i < maxAttempts; i++) { - try { - // Try HTTPS first (internal CA) - const response = await fetchT(`https://${domain}/`, { - method: 'HEAD', - agent: httpsAgent, // Ignore cert errors for internal CA - timeout: 5000 - }); - - // Any response (even 4xx) means Caddy is serving the site - log.info('caddy', 'Site is accessible', { domain, status: response.status }); - return true; - } catch (error) { - log.debug('caddy', 'Site verification attempt', { domain, attempt: i + 1, maxAttempts, error: error.message }); - } - - if (i < maxAttempts - 1) { - await new Promise(resolve => setTimeout(resolve, delay)); - } - } - - log.warn('caddy', 'Could not verify site accessibility, but deployment may still have succeeded', { domain }); - return false; -} - -async function createDnsRecord(subdomain, ip) { - // Ensure we have a valid token (auto-refresh if needed) - const tokenResult = await ensureValidDnsToken(); - if (!tokenResult.success) { - throw new Error(`DNS token not available: ${tokenResult.error}. Configure credentials via POST /api/dns/credentials`); - } - - const domain = buildDomain(subdomain); - const zone = siteConfig.tld.replace(/^\./, ''); - - const dnsParams = { token: dnsToken, domain, zone, type: 'A', ipAddress: ip, ttl: '300', overwrite: 'true' }; - const callDnsApi = () => callDns(siteConfig.dnsServerIp, '/api/zones/records/add', dnsParams); - - try { - log.info('dns', 'Creating DNS record', { domain, ip }); - const result = await callDnsApi(); - - if (result.status === 'ok') { - log.info('dns', 'DNS record created', { domain, ip }); - return { success: true }; - } - - // Check for token expired error - try to refresh once - if (result.errorMessage && result.errorMessage.toLowerCase().includes('token')) { - log.info('dns', 'Token appears expired, attempting auto-refresh'); - const refreshResult = await ensureValidDnsToken(); - if (!refreshResult.success) throw new Error(`Token refresh failed: ${refreshResult.error}`); - - const retryResult = await callDnsApi(); - if (retryResult.status === 'ok') { - log.info('dns', 'DNS record created after token refresh', { domain, ip }); - return { success: true }; - } - throw new Error(retryResult.errorMessage || 'Unknown error after token refresh'); - } - - throw new Error(result.errorMessage || 'Unknown error'); - } catch (error) { - throw new Error(`Failed to create DNS record for ${domain}: ${error.message}`); - } -} - -async function addServiceToConfig(service) { - try { - await servicesStateManager.update(services => { - // Check if service already exists - const existingIndex = services.findIndex(s => s.id === service.id); - if (existingIndex >= 0) { - // Update existing service - services[existingIndex] = { ...services[existingIndex], ...service }; - } else { - // Add new service - services.push(service); - } - return services; - }); - log.info('deploy', 'Service added to config', { serviceId: service.id }); - // Sync health checker with updated services list - ctx.resyncHealthChecker?.().catch(() => {}); - } catch (error) { - log.error('deploy', 'Failed to add service to config', { serviceId: service.id, error: error.message }); - throw error; - } -} - -// (Notification routes moved to routes/notifications.js) -// (Stats routes moved to routes/monitoring.js) -// (Container logs routes moved to routes/logs.js) -// (Health service routes moved to routes/health.js) -// (Resource monitoring routes moved to routes/monitoring.js) -// (Backup routes moved to routes/backups.js) -// (CA routes moved to routes/ca.js) -// (Error log, audit log, browse/media routes moved to route modules) - -// API Documentation endpoint -app.get('/api/docs', (req, res) => { - res.setHeader('Content-Security-Policy', "default-src 'self'; script-src 'self' 'unsafe-inline' https://unpkg.com; style-src 'self' 'unsafe-inline' https://unpkg.com; img-src 'self' data: https:; connect-src 'self'; font-src 'self' data: https://unpkg.com;"); - res.send(` - - - - DashCaddy API Documentation - - - - -
- - - -`); -}); - -app.get('/api/docs/spec', asyncHandler(async (req, res) => { - const specPath = path.join(__dirname, 'openapi.yaml'); - if (await exists(specPath)) { - const yaml = await fsp.readFile(specPath, 'utf8'); - res.type('text/yaml').send(yaml); - } else { - errorResponse(res, 404, 'OpenAPI spec not found'); - } -}, 'api-docs-spec')); - -// Unified error handlers (order matters!) -const { notFoundHandler, errorMiddleware } = require('./error-handler'); - -// 404 handler for unmatched API routes -app.use('/api', notFoundHandler); - -// Global error handler (MUST be last middleware) -app.use(errorMiddleware); - -// Export app for testing -module.exports = app; - -if (require.main === module) { -// Validate configuration and wait for async config loads before starting server -(async () => { -await Promise.all([_configsReady, _notificationsReady]); -await licenseManager.load(); -await validateStartupConfig({ log, CADDYFILE_PATH, SERVICES_FILE, CONFIG_FILE, CADDY_ADMIN_URL, PORT }); - -const server = app.listen(PORT, '0.0.0.0', () => { - log.info('server', 'DashCaddy API server started', { port: PORT, caddyfile: CADDYFILE_PATH, caddyAdmin: CADDY_ADMIN_URL, services: SERVICES_FILE }); - if (BROWSE_ROOTS.length > 0) { - log.info('server', 'Media browse roots configured', { roots: BROWSE_ROOTS.map(r => r.hostPath) }); - } - - // Start new feature modules - log.info('server', 'Starting DashCaddy feature modules'); - - // Clean up stale port locks - (async () => { - try { - await portLockManager.cleanupStaleLocks(); - log.info('server', 'Port lock cleanup completed'); - } catch (error) { - log.error('server', 'Port lock cleanup failed', { error: error.message }); - } - })(); - - try { - resourceMonitor.start(); - log.info('server', 'Resource monitoring started'); - } catch (error) { - log.error('server', 'Resource monitoring failed to start', { error: error.message }); - } - - try { - backupManager.start(); - log.info('server', 'Backup manager started'); - } catch (error) { - log.error('server', 'Backup manager failed to start', { error: error.message }); - } - - (async () => { - try { - // Auto-configure health checker from services.json - await syncHealthCheckerServices({ log, SERVICES_FILE, servicesStateManager, healthChecker, buildServiceUrl, siteConfig, APP }); - healthChecker.start(); - log.info('server', 'Health checker started'); - } catch (error) { - log.error('server', 'Health checker failed to start', { error: error.message }); - } - })(); - - try { - updateManager.start(); - log.info('server', 'Update manager started'); - } catch (error) { - log.error('server', 'Update manager failed to start', { error: error.message }); - } - - try { - selfUpdater.start(); - log.info('server', 'Self-updater started', { interval: selfUpdater.config.checkInterval, url: selfUpdater.config.updateUrl }); - // Check for post-update result (did a previous update succeed or roll back?) - selfUpdater.checkPostUpdateResult().then(result => { - if (result) { - log.info('server', 'Post-update result', result); - if (typeof ctx.notification?.send === 'function') { - ctx.notification.send('system.update', - result.success ? 'DashCaddy Updated' : 'DashCaddy Update Failed', - result.success ? `Updated to v${result.version}` : `Update failed: ${result.error || 'Unknown'}. Rolled back.`, - result.success ? 'info' : 'error' - ); - } - } - }).catch(() => {}); - } catch (error) { - log.error('server', 'Self-updater failed to start', { error: error.message }); - } - - if (dockerMaintenance) { - try { - dockerMaintenance.start(); - log.info('server', 'Docker maintenance started'); - dockerMaintenance.on('maintenance-complete', (result) => { - const saved = Math.round(result.spaceReclaimed.total / 1024 / 1024); - if (saved > 0 || result.warnings.length > 0) { - log.info('maintenance', 'Docker maintenance completed', { - spaceReclaimedMB: saved, - pruned: result.pruned, - warnings: result.warnings.length - }); - } - if (result.warnings.length > 0) { - for (const w of result.warnings) log.warn('maintenance', w); - } - }); - } catch (error) { - log.error('server', 'Docker maintenance failed to start', { error: error.message }); - } - } - - if (logDigest) { - try { - logDigest.start(platformPaths.digestDir); - log.info('server', 'Log digest started', { digestDir: platformPaths.digestDir }); - logDigest.on('digest-generated', ({ date }) => { - log.info('digest', `Daily digest generated for ${date}`); - if (typeof ctx.notification?.send === 'function') { - ctx.notification.send('system.digest', 'Daily Log Digest', `Log digest for ${date} is ready. View it in the DashCaddy dashboard.`, 'info'); - } - }); - } catch (error) { - log.error('server', 'Log digest failed to start', { error: error.message }); - } - } - - // Tailscale API sync (if OAuth configured) - if (tailscaleConfig.oauthConfigured) { - startTailscaleSyncTimer(); - // Run initial sync - syncFromTailscaleAPI().catch(e => log.warn('tailscale', 'Initial API sync failed', { error: e.message })); - } - - log.info('server', 'All feature modules initialized'); -}); - -// Graceful shutdown — drain connections before exiting -function shutdown(signal) { - log.info('shutdown', `${signal} received, draining connections...`); - resourceMonitor.stop(); - backupManager.stop(); - if (dockerMaintenance) dockerMaintenance.stop(); - if (logDigest) logDigest.stop(); - healthChecker.stop(); - updateManager.stop(); - selfUpdater.stop(); - stopTailscaleSyncTimer(); - server.close(() => { - log.info('shutdown', 'HTTP server closed'); - process.exit(0); - }); - // Force exit after 5s if connections don't drain - setTimeout(() => process.exit(0), 5000).unref(); -} -process.on('SIGTERM', () => shutdown('SIGTERM')); -process.on('SIGINT', () => shutdown('SIGINT')); -})(); // end async startup -} // end if (require.main === module) - -// #2: Catch unhandled errors so the process doesn't crash silently -process.on('unhandledRejection', (reason) => { - logError('unhandledRejection', reason instanceof Error ? reason : new Error(String(reason))); -}); -process.on('uncaughtException', (error) => { - logError('uncaughtException', error); - // Give the error log time to flush, then exit - setTimeout(() => process.exit(1), 1000).unref(); -}); - - diff --git a/error-handling-cleanup-summary.md b/error-handling-cleanup-summary.md deleted file mode 100644 index 741f575..0000000 --- a/error-handling-cleanup-summary.md +++ /dev/null @@ -1,206 +0,0 @@ -# DashCaddy Error Handling Cleanup - Summary - -## ✅ Completed Changes - -### 1. Unified Error Classes (`dashcaddy-api/errors.js`) -- ✅ Merged all error types into single source of truth -- ✅ Added standard DC-XXX error codes -- ✅ All errors inherit from `AppError` with `isOperational` flag -- ✅ Removed duplicate definitions (NotFoundError, AuthenticationError, etc.) - -**Available Error Classes:** -- `ValidationError` - DC-400 (client validation failures) -- `AuthenticationError` - DC-401 (auth required, with TOTP support) -- `ForbiddenError` - DC-403 (insufficient permissions) -- `NotFoundError` - DC-404 (resource not found) -- `ConflictError` - DC-409 (resource conflicts) -- `RateLimitError` - DC-429 (rate limiting) -- `DockerError` - DC-500-DOCKER (Docker operation failures) -- `CaddyError` - DC-502-CADDY (Caddy proxy errors) -- `DNSError` - DC-502-DNS (DNS service errors) -- `ServiceUnavailableError` - DC-503 (service unavailable) - -### 2. Unified Error Middleware (`dashcaddy-api/error-handler.js`) -- ✅ Single `errorMiddleware` function handles all errors -- ✅ Automatic request context logging -- ✅ Consistent JSON response format -- ✅ Development mode includes stack traces -- ✅ `asyncHandler` wrapper eliminates try/catch boilerplate -- ✅ `notFoundHandler` for 404 routes - -### 3. Server Configuration (`dashcaddy-api/server.js`) -- ✅ Replaced old error handlers with unified system -- ✅ Proper middleware order: routes → notFoundHandler → errorMiddleware -- ✅ Cleaner, more maintainable error handling - -### 4. Route Migrations -- ✅ `routes/themes.js` - Migrated to throw-based errors -- ✅ `routes/services.js` - Updated conflict error to use `ConflictError` -- ✅ `routes/containers.js` - Already using new pattern (no changes needed) - -## 📊 Before vs After - -### Before (Old Pattern) -```javascript -app.get('/api/resource/:id', async (req, res) => { - try { - const resource = await getResource(req.params.id); - if (!resource) { - return res.status(404).json({ - success: false, - error: 'Resource not found' - }); - } - res.json({ success: true, data: resource }); - } catch (error) { - res.status(500).json({ - success: false, - error: error.message - }); - } -}); -``` - -**Problems:** -- 9 lines of error handling boilerplate -- Inconsistent error responses -- No automatic logging -- No error codes -- Manual status code management - -### After (New Pattern) -```javascript -const { asyncHandler } = require('../error-handler'); -const { NotFoundError } = require('../errors'); - -app.get('/api/resource/:id', asyncHandler(async (req, res) => { - const resource = await getResource(req.params.id); - if (!resource) { - throw new NotFoundError(`Resource ${req.params.id}`); - } - res.json({ success: true, data: resource }); -})); -``` - -**Benefits:** -- 4 lines total (55% less code) -- Consistent error format with DC-404 code -- Automatic request context logging -- Type-safe error classes -- Clean, readable route logic - -## 🎯 Standard Error Response Format - -All errors now return consistent JSON: - -```json -{ - "success": false, - "error": "Human-readable error message", - "code": "DC-404", - "resource": "Container abc123" -} -``` - -**Optional fields:** -- `requiresTotp: true` - For authentication errors requiring TOTP -- `retryAfter: 60` - For rate limit errors -- `field: "email"` - For validation errors -- `details: {}` - Additional context for Docker/Caddy/DNS errors -- `stack: "..."` - Stack trace (development mode only) - -## 📝 Migration Guidelines for Remaining Routes - -### Pattern 1: Replace Direct Error Responses -```javascript -// OLD -return res.status(400).json({ success: false, error: 'Invalid input' }); - -// NEW -throw new ValidationError('Invalid input', 'fieldName'); -``` - -### Pattern 2: Wrap Routes with asyncHandler -```javascript -// OLD -router.get('/path', async (req, res) => { - try { - // ... logic - } catch (e) { - res.status(500).json({ success: false, error: e.message }); - } -}); - -// NEW -router.get('/path', asyncHandler(async (req, res) => { - // ... logic (errors automatically caught and handled) -})); -``` - -### Pattern 3: Use Typed Errors -```javascript -// Instead of generic errors: -throw new Error('Something went wrong'); - -// Use specific error classes: -throw new DockerError('Container failed to start', 'start', { containerId }); -throw new NotFoundError('Container abc123'); -throw new ConflictError('Port 8080 already in use', '8080'); -throw new ValidationError('Email is required', 'email'); -``` - -## 🔍 Testing Checklist - -- [ ] All routes return consistent error format -- [ ] Error codes are unique and meaningful -- [ ] Stack traces only appear in development -- [ ] All errors logged with request context -- [ ] 404 routes handled properly -- [ ] Async errors caught automatically -- [ ] TOTP errors include `requiresTotp: true` -- [ ] Rate limit errors include `retryAfter` - -## 📦 Files Modified - -1. `dashcaddy-api/errors.js` - Unified error classes -2. `dashcaddy-api/error-handler.js` - Unified middleware -3. `dashcaddy-api/server.js` - Updated error handler registration -4. `dashcaddy-api/routes/themes.js` - Migrated to new pattern -5. `dashcaddy-api/routes/services.js` - Added ConflictError - -## 🚀 Next Steps - -### High Priority Routes to Migrate -1. `routes/auth/*` - Authentication routes (high traffic) -2. `routes/dns.js` - DNS management -3. `routes/caddy.js` - Caddy proxy operations -4. `routes/recipes/*.js` - Recipe deployment - -### Benefits of Full Migration -- **~40% less code** in route handlers -- **100% consistent** error responses -- **Automatic logging** for all errors -- **Type-safe** error handling -- **Better debugging** with standardized codes - -## 🎉 Impact - -**Code Quality:** -- Eliminated duplicate error handling code -- Standardized error response format -- Type-safe error classes - -**Developer Experience:** -- Routes are shorter and more readable -- No more try/catch boilerplate -- Clear error types for different scenarios - -**Debugging:** -- All errors logged with request context -- Standard error codes for client-side handling -- Stack traces available in development - -**Client Experience:** -- Consistent error format across all endpoints -- Machine-readable error codes -- Clear, descriptive error messages diff --git a/error-handling-migration-complete.md b/error-handling-migration-complete.md deleted file mode 100644 index 1b432cd..0000000 --- a/error-handling-migration-complete.md +++ /dev/null @@ -1,211 +0,0 @@ -# DashCaddy Error Handling Migration - Complete! ✅ - -## Summary - -Successfully migrated DashCaddy from 3 competing error systems to a unified, throw-based error handling architecture. - -## What Was Done - -### Phase 1: Foundation (Commit 64a0018) -- ✅ Created unified error class system (`errors.js`) -- ✅ Built unified error middleware (`error-handler.js`) -- ✅ Updated server configuration -- ✅ Migrated 2 example routes (themes.js, services.js) - -### Phase 2: Mass Migration (Commit b172a21) -- ✅ Migrated 25 route files -- ✅ Converted ~150 error responses -- ✅ Standardized error formats across critical routes - -## Files Migrated (27 total) - -### Authentication Routes (7 files) -- `routes/auth/totp.js` - TOTP login/setup -- `routes/auth/keys.js` - API key management -- `routes/auth/sso-gate.js` - SSO gateway -- `routes/themes.js` - UI themes -- `routes/services.js` - Service management -- `routes/credentials.js` - Credential storage -- `routes/sites.js` - Site configuration - -### Deployment Routes (6 files) -- `routes/apps/deploy.js` - App deployment -- `routes/apps/templates.js` - App templates -- `routes/recipes/deploy.js` - Recipe deployment -- `routes/recipes/manage.js` - Recipe management -- `routes/recipes/index.js` - Recipe listing -- `routes/arr/config.js` - ARR configuration - -### Infrastructure Routes (8 files) -- `routes/dns.js` - DNS management (partial) -- `routes/config/assets.js` - Asset management -- `routes/config/backup.js` - Backup configuration -- `routes/config/settings.js` - Settings -- `routes/logs.js` - Log viewing -- `routes/health.js` - Health checks -- `routes/license.js` - License validation -- `routes/notifications.js` - Notification system - -### Additional Routes (6 files) -- `routes/browse.js` - File browser -- `routes/ca.js` - Certificate authority -- `routes/arr/credentials.js` - ARR credentials -- `routes/tailscale.js` - Tailscale integration -- `routes/updates.js` - Update management - -## Migration Statistics - -### Before -- 3 different error systems competing -- Duplicate error class definitions -- Inconsistent error response formats -- ~250+ manual error responses scattered across codebase -- No standard error codes -- Tons of try/catch boilerplate - -### After -- 1 unified error system -- Single source of truth for error classes -- Standard DC-XXX error codes -- Automatic request context logging -- 40% less error handling code -- Type-safe error classes - -### Code Reduction Example - -**Before (9 lines):** -```javascript -try { - const resource = await getResource(id); - if (!resource) { - return res.status(404).json({ success: false, error: 'Not found' }); - } - res.json({ success: true, data: resource }); -} catch (error) { - res.status(500).json({ success: false, error: error.message }); -} -``` - -**After (4 lines):** -```javascript -const resource = await getResource(id); -if (!resource) throw new NotFoundError(`Resource ${id}`); -res.json({ success: true, data: resource }); -// Middleware handles all errors automatically -``` - -## Error Class Usage - -| Error Class | Count | Use Case | -|------------|-------|----------| -| ValidationError | ~60 | Invalid input, bad format | -| AuthenticationError | ~30 | TOTP, JWT, API key auth | -| ForbiddenError | ~15 | Permission denied | -| NotFoundError | ~40 | Resource not found | -| ConflictError | ~5 | Duplicate resources | -| DockerError | ~10 | Docker operation failures | -| CaddyError | ~5 | Caddy proxy errors | -| DNSError | ~5 | DNS service errors | - -## Standard Error Response Format - -All errors now return: - -```json -{ - "success": false, - "error": "Human-readable error message", - "code": "DC-404", - "resource": "Container abc123" -} -``` - -**Optional fields:** -- `requiresTotp: true` - Authentication requires TOTP -- `retryAfter: 60` - Rate limiting retry delay -- `field: "email"` - Validation error field -- `details: {}` - Additional context -- `stack: "..."` - Stack trace (development only) - -## Remaining Work - -### Files Still Using Old Pattern (~82 instances) -Most remaining are complex patterns with template literals, variable status codes, or dynamic error messages. These are mostly in: - -- `dns.js` - Complex error patterns with API responses -- `services.js` - Some dynamic error handling -- Various other files with edge cases - -### Why These Weren't Auto-Converted -- Template literal error messages (`` `Port ${port} in use` ``) -- Variable status codes (`response.status`) -- Wrapped error responses from APIs (`safeErrorMessage(error)`) -- Conditional error patterns - -### Recommendation -These remaining instances work fine and can be migrated incrementally as those routes are touched. The critical paths are all converted. - -## Testing - -### Manual Testing Checklist -- [x] TOTP login flow -- [x] API key generation -- [x] Recipe deployment -- [x] Theme management -- [x] Service creation -- [ ] DNS record management (partial) -- [ ] Full end-to-end deployment - -### Expected Behavior -- All errors return consistent JSON format -- Error codes follow DC-XXX pattern -- Stack traces only in development -- Request context logged for all errors -- No breaking changes to API contracts - -## Impact - -### Developer Experience -- Routes are shorter and more readable -- No more try/catch boilerplate -- Clear error types for different scenarios -- Easier to add new routes - -### Debugging -- All errors logged with request context -- Standard error codes for client-side handling -- Better stack traces -- Consistent format makes monitoring easier - -### Client Experience -- Consistent error format across all endpoints -- Machine-readable error codes -- Clear, descriptive error messages -- Field-level validation errors - -## Performance - -No performance impact. The middleware adds negligible overhead and eliminates redundant error handling logic. - -## Next Steps (Optional) - -1. **Convert remaining complex patterns** - As routes are touched, convert remaining errorResponse calls -2. **Add error code documentation** - Document all DC-XXX codes for API consumers -3. **Client-side error handling** - Update dashboard to handle new error format -4. **Monitoring integration** - Use error codes for alerting/metrics - -## Success Metrics - -- ✅ 27 files migrated -- ✅ ~170 error responses standardized -- ✅ 40% code reduction in error handling -- ✅ Single source of truth for errors -- ✅ Automatic request logging -- ✅ Type-safe error classes -- ✅ Standard error codes - -## Conclusion - -The error handling migration is **functionally complete**. All critical routes use the new system, providing consistent, professional error responses. The remaining ~80 instances are edge cases that can be migrated incrementally. - -**Result:** DashCaddy now has production-grade error handling that's maintainable, consistent, and developer-friendly. 🎉 diff --git a/fix-ctx-routes.sh b/fix-ctx-routes.sh deleted file mode 100644 index cf46e86..0000000 --- a/fix-ctx-routes.sh +++ /dev/null @@ -1,13 +0,0 @@ -#!/bin/bash -# Systematically fix ctx.* references in all route files - -cd /root/.openclaw/agents/main/workspace/dashcaddy-work/dashcaddy-api - -# Find all route files with ctx errors -echo "Finding routes with ctx errors..." -for file in $(find routes -name "*.js" -type f | grep -v index.js | grep -v helpers.js); do - errors=$(npx eslint "$file" 2>&1 | grep -c "'ctx' is not defined") - if [ "$errors" -gt 0 ]; then - echo "$errors errors in $file" - fi -done | sort -rn