Phase 2 (WIP): Extract config and utils modules

- Created src/config/ (env.js, site-config.js)
- Created src/utils/ (async-handler.js, responses.js, safe-error.js)
- server.js not yet modified (backward-compatible extraction)
This commit is contained in:
Krystie
2026-03-22 11:04:04 +01:00
parent 039d3d07e2
commit d5a6789366
8 changed files with 681 additions and 0 deletions

388
DESLOPIFICATION-ROADMAP.md Normal file
View File

@@ -0,0 +1,388 @@
# DashCaddy API Deslopification Roadmap
**Audited:** 2026-03-22
**Version:** 1.1.0
**Total Lines:** ~26,000 (API), ~10,000 (dashboard)
**Priority:** API-first (make backend powerful, clean dashboard follows naturally)
---
## Executive Summary
The DashCaddy API is **feature-complete and security-hardened**, but the codebase shows signs of rapid evolution. While functionally robust, it would significantly benefit from architectural refactoring to improve maintainability, testability, and long-term scalability.
### Key Strengths
✅ Comprehensive feature set (76+ app templates, Docker/Caddy/DNS management)
✅ Security-conscious (TOTP auth, AES-256-GCM credentials, CSRF protection, audit logging)
✅ Recent test coverage additions (auth, credentials, Docker security)
✅ Modular route organization (routes/ subdirectories)
✅ Shared context pattern for dependency injection
### Core Issues
**Monolithic `server.js`** (1960 lines) — initialization, middleware, utilities, business logic all in one file
**God object `ctx`** — 50+ properties/methods across multiple domains with hidden dependencies
**Inconsistent patterns** — routes use classes, factory functions, or flat modules with no standard
**No code standards** — ESLint installed but no config, no formatting rules
**Mixed concerns** — HTTP handlers, business logic, validation intertwined in route files
---
## Current Architecture
```
dashcaddy-api/
├── server.js (1960 lines) ← MAIN PROBLEM
│ ├── 89 require() statements
│ ├── 131 top-level declarations
│ ├── Middleware setup
│ ├── Context (`ctx`) assembly (50+ properties)
│ ├── Route mounting
│ ├── Error handlers
│ └── Server startup
├── routes/
│ ├── auth/ (5 files, modular) ✅
│ ├── config/ (4 files, modular) ✅
│ ├── apps/ (6 files, helpers pattern) ⚠️
│ ├── arr/ (4 files, helpers pattern) ⚠️
│ ├── recipes/ (3 files) ⚠️
│ └── *.js (19 flat route files) ❌
├── Managers (clean, well-separated)
│ ├── auth-manager.js (307 lines) ✅
│ ├── credential-manager.js (395 lines) ✅
│ ├── state-manager.js (237 lines) ✅
│ ├── backup-manager.js (835 lines) ⚠️
│ ├── health-checker.js (591 lines) ⚠️
│ └── update-manager.js (911 lines) ⚠️
├── Utilities
│ ├── input-validator.js (606 lines) ⚠️
│ ├── crypto-utils.js (340 lines) ✅
│ ├── middleware.js (430 lines) ⚠️
│ └── constants.js ✅
└── Templates
├── app-templates.js (2496 lines) ⚠️
└── recipe-templates.js (339 lines) ✅
```
**Legend:**
✅ Good structure
⚠️ Works but could be cleaner
❌ Needs refactoring
---
## Deslopification Phases
### Phase 1: Foundation & Standards (IMMEDIATE)
**Goal:** Establish code quality baseline before refactoring
**Effort:** 2-4 hours
**Risk:** Low (tooling only, no code changes)
#### 1.1 Code Standards Setup
- [ ] Create `.eslintrc.js` with recommended rules
- [ ] Add Prettier config (`.prettierrc`)
- [ ] Add npm scripts: `lint`, `lint:fix`, `format`
- [ ] Run `npm run lint:fix` and commit baseline cleanup
- [ ] Add pre-commit hooks (optional)
**Why first:** Establish formatting/style consistency before making structural changes. Prevents "should I refactor this while I'm here?" scope creep.
#### 1.2 Dependency Graph Documentation
- [ ] Map `ctx` properties → which routes actually use them
- [ ] Identify circular dependencies (if any)
- [ ] Document shared utilities used across routes
**Deliverable:** `DEPENDENCIES.md` — reference for refactoring decisions
---
### Phase 2: Extract & Organize (HIGH PRIORITY)
**Goal:** Break `server.js` into logical modules
**Effort:** 1-2 days
**Risk:** Medium (requires testing at each step)
#### 2.1 Split `server.js` into Layers
**Before:** 1960-line monolith
**After:** Clean initialization flow
Create new structure:
```
src/
├── app.js ← Express app setup (middleware, routes)
├── server.js ← Entry point (load config, start server)
├── config/
│ ├── index.js ← Load all config (env, files, constants)
│ ├── env.js ← Environment variable validation
│ └── paths.js ← Platform-specific paths
├── context/
│ ├── index.js ← Assemble context (DI container)
│ ├── docker.js ← Docker-related context properties
│ ├── caddy.js ← Caddy-related context properties
│ ├── dns.js ← DNS context
│ ├── session.js ← Session context
│ └── notification.js ← Notification context
├── middleware/
│ ├── index.js ← Export all middleware
│ ├── auth.js ← Move from middleware.js
│ ├── error.js ← Error handlers
│ └── security.js ← Helmet, CORS, CSRF
└── routes/
└── (existing structure)
```
**Migration Steps:**
1. Create `src/config/` — extract all config loading from `server.js`
2. Create `src/context/` — split god object into domain modules
3. Create `src/middleware/` — break up `middleware.js` (430 lines)
4. Create `src/app.js` — Express setup + route mounting
5. Slim `server.js` → minimal entry point (~50 lines)
**Tests:** Ensure existing test suite still passes after each step
---
### Phase 3: Route Standardization (MEDIUM PRIORITY)
**Goal:** Consistent route module pattern across entire API
**Effort:** 2-3 days
**Risk:** Medium (touching business logic)
#### 3.1 Establish Route Pattern
**Chosen Pattern:** Factory function with explicit dependencies
```javascript
// routes/services.js (before)
module.exports = (ctx) => {
const router = express.Router();
// ... uses ctx.docker, ctx.servicesStateManager, ctx.log, etc.
return router;
};
// routes/services.js (after)
module.exports = ({ docker, servicesStateManager, log, asyncHandler }) => {
const router = express.Router();
// ... explicitly passed dependencies
return router;
};
```
**Benefits:**
- Self-documenting (you see what each route needs)
- Easier testing (mock only what's used)
- No hidden dependencies via god object
#### 3.2 Refactor Routes by Priority
**Order:** Most-used routes first
1. **High-traffic routes:**
- `routes/services.js` (467 lines) — core service management
- `routes/containers.js` (246 lines) — Docker operations
- `routes/health.js` (297 lines) — health checks
- `routes/dns.js` (632 lines) — DNS management
2. **Auth routes** (already modular, just align pattern):
- `routes/auth/*`
3. **Feature routes:**
- `routes/apps/*`
- `routes/arr/*`
- `routes/recipes/*`
4. **Utility routes:**
- `routes/logs.js`
- `routes/backups.js`
- `routes/ca.js`
- etc.
**Per-route checklist:**
- [ ] Extract dependencies from `ctx` → explicit parameters
- [ ] Move business logic to service layer (if complex)
- [ ] Validate inputs at route boundary
- [ ] Return consistent error format
- [ ] Add route-level tests
---
### Phase 4: Service Layer Introduction (LOWER PRIORITY)
**Goal:** Separate business logic from HTTP handlers
**Effort:** 3-5 days
**Risk:** Medium-High (significant refactor)
**Problem:** Routes currently mix HTTP concerns with business logic:
```javascript
// Current: Everything in route handler
router.post('/deploy', async (req, res) => {
// 1. Parse request
// 2. Validate inputs
// 3. Business logic (complex Docker operations)
// 4. Error handling
// 5. Format response
});
```
**Solution:** Service layer pattern
```javascript
// routes/apps/deploy.js
router.post('/deploy', async (req, res) => {
const result = await appDeployService.deploy(req.body);
res.json({ success: true, data: result });
});
// services/app-deploy-service.js
class AppDeployService {
async deploy({ templateId, config }) {
// Pure business logic, no HTTP awareness
}
}
```
**Candidates for service extraction:**
- `services/docker-service.js` — container lifecycle, networking
- `services/caddy-service.js` — Caddyfile manipulation, reload
- `services/dns-service.js` — record management, zone operations
- `services/app-deploy-service.js` — template-based deployment
- `services/backup-service.js` — backup/restore workflows
**Benefits:**
- Routes become thin HTTP adapters (easy to test)
- Business logic testable without HTTP mocking
- Reusable across routes (e.g., CLI tools, cron jobs)
---
### Phase 5: Manager Cleanup (ONGOING)
**Goal:** Refine existing manager modules
**Effort:** 1-2 days (parallel to other phases)
#### Issues to Address
1. **`backup-manager.js` (835 lines)** — too large, split backup vs restore logic
2. **`update-manager.js` (911 lines)** — complex state machine, extract version comparison utilities
3. **`health-checker.js` (591 lines)** — separate health check logic from notification daemon
4. **`input-validator.js` (606 lines)** — split by domain (docker, caddy, dns validators)
**Approach:** Incremental splitting, preserve existing API
---
### Phase 6: Template Organization (LOW PRIORITY)
**Goal:** Make templates maintainable and extensible
**Effort:** 1 day
**Problem:** `app-templates.js` is 2496 lines (76 templates in one file)
**Solution:**
```
templates/
├── index.js ← Export TEMPLATE_CATEGORIES, DIFFICULTY_LEVELS
├── apps/
│ ├── media/
│ │ ├── plex.js
│ │ ├── jellyfin.js
│ │ └── ...
│ ├── automation/
│ └── ...
└── recipes/
├── arr-stack.js
└── ...
```
**Benefits:**
- Easier to find/edit specific templates
- Contributors can add templates without merge conflicts
- Templates can import shared snippets (e.g., common env vars)
---
## Metrics & Success Criteria
### Code Quality Metrics (Before → After)
| Metric | Before | Target | How to Measure |
|--------|--------|--------|----------------|
| `server.js` lines | 1960 | <200 | `wc -l server.js` |
| Avg route file size | ~300 | <150 | `find routes -name '*.js' -exec wc -l {} + \| awk '{sum+=$1; n++} END {print sum/n}'` |
| `ctx` properties | 50+ | 0 (removed) | Manual count |
| ESLint errors | Unknown | 0 | `npm run lint` |
| Test coverage | ~30% | >60% | `npm run test:coverage` |
| Files >500 lines | 8 | <3 | `find . -name '*.js' -exec wc -l {} + \| awk '$1 > 500'` |
### Developer Experience Improvements
- **Onboarding:** New contributor should understand route structure in <10 minutes
- **Testing:** Mock only what you use (no god object sprawl)
- **Changes:** Touching one domain shouldn't require understanding entire codebase
- **Deployment:** Confidence that refactor didn't break anything (test suite)
---
## Risk Mitigation
### How to Refactor Safely
1. **Test suite first** — before touching code:
- Run existing tests: `npm test`
- Identify untested critical paths → add tests
- Establish coverage baseline
2. **Incremental changes**:
- Each phase = separate branch
- Each phase passes full test suite
- Deploy to test environment (Contabo) before merging
3. **Preserve API contract**:
- Frontend expects same endpoints/responses
- Dashboard shouldn't need changes during API refactor
- Version routes if breaking changes needed
4. **Rollback plan**:
- Git tags before each phase merge
- Keep old code in `legacy/` until confidence is high
- Document what changed in each PR
---
## Recommended Order of Execution
**Week 1: Foundation**
- Day 1-2: Phase 1 (ESLint, Prettier, dependency mapping)
- Day 3-5: Phase 2.1 (split `server.js`)
**Week 2: Routes**
- Day 1-3: Phase 3.1 (standardize top 5 routes)
- Day 4-5: Phase 3.2 (remaining routes)
**Week 3: Refinement**
- Day 1-3: Phase 4 (service layer for complex routes)
- Day 4-5: Phase 5 (manager cleanup)
**Week 4: Polish**
- Day 1-2: Phase 6 (template organization)
- Day 3-5: Documentation, final testing, deployment
**Total:** ~4 weeks part-time or ~2 weeks full-time
---
## Questions for Sami
Before starting, clarify:
1. **Testing strategy:** Current test coverage is partial. Should we:
- Write tests BEFORE refactoring (safer, slower)?
- Refactor with existing tests, add coverage later (faster, riskier)?
2. **Breaking changes:** Can we introduce backwards-incompatible API changes if we version routes (`/api/v2/...`)?
3. **Deployment cadence:** Should each phase deploy to production, or batch into one big release?
4. **Priority tweaks:** Does this roadmap align with "deslopify → market → sell" timeline, or should we focus only on the most visible pain points first?
---
## Next Steps
**If approved:**
1. Create feature branch: `refactor/deslopification-phase-1`
2. Add ESLint + Prettier configs
3. Run `npm run lint:fix` and commit baseline
4. Create `DEPENDENCIES.md` (ctx usage map)
5. Review with Sami before proceeding to Phase 2
**Estimated time to first visible improvement:** 1 week (server.js split + linting)

View File

@@ -0,0 +1,50 @@
/**
* Environment variable loading and validation
* Central place for all process.env reads
*/
const path = require('path');
const platformPaths = require('../../platform-paths');
const { APP, LIMITS, CADDY } = require('../../constants');
// Resolve services directory from SERVICES_FILE env var
const SERVICES_FILE = process.env.SERVICES_FILE || platformPaths.servicesFile;
const SERVICES_DIR = path.dirname(SERVICES_FILE);
/**
* Application configuration loaded from environment variables
*/
const config = {
// Server
port: APP.PORT,
// Caddy paths
caddyfilePath: process.env.CADDYFILE_PATH || platformPaths.caddyfile,
caddyAdminUrl: process.env.CADDY_ADMIN_URL || platformPaths.caddyAdminUrl,
// State files
servicesFile: SERVICES_FILE,
configFile: process.env.CONFIG_FILE || path.join(SERVICES_DIR, 'config.json'),
dnsCredentialsFile: process.env.DNS_CREDENTIALS_FILE || path.join(SERVICES_DIR, 'dns-credentials.json'),
tailscaleConfigFile: process.env.TAILSCALE_CONFIG_FILE || path.join(SERVICES_DIR, 'tailscale-config.json'),
notificationsFile: process.env.NOTIFICATIONS_FILE || path.join(SERVICES_DIR, 'notifications.json'),
totpConfigFile: process.env.TOTP_CONFIG_FILE || path.join(SERVICES_DIR, 'totp-config.json'),
errorLogFile: process.env.ERROR_LOG_FILE || path.join(__dirname, '../../dashcaddy-errors.log'),
licenseSecretFile: process.env.LICENSE_SECRET_FILE || path.join(__dirname, '../../.license-secret'),
// Limits
maxErrorLogSize: LIMITS.ERROR_LOG_SIZE,
// Media browse roots (optional feature)
browseRoots: (process.env.MEDIA_BROWSE_ROOTS || '')
.split(',')
.filter((r) => r.includes('='))
.map((r) => {
const eqIndex = r.indexOf('=');
const containerPath = r.slice(0, eqIndex).trim();
const hostPath = r.slice(eqIndex + 1).trim();
return { containerPath, hostPath };
}),
};
module.exports = config;

View File

@@ -0,0 +1,23 @@
/**
* Configuration module
* Central exports for all configuration loading
*/
const envConfig = require('./env');
const {
siteConfig,
loadSiteConfig,
buildDomain,
buildServiceUrl,
} = require('./site-config');
module.exports = {
// Environment config
...envConfig,
// Site config
siteConfig,
loadSiteConfig,
buildDomain,
buildServiceUrl,
};

View File

@@ -0,0 +1,103 @@
/**
* Site configuration management
* Loads and validates site-wide settings from config.json
*/
const fs = require('fs');
const { validateConfig } = require('../../config-schema');
const { CADDY } = require('../../constants');
/**
* Site configuration state
* Modified by loadSiteConfig()
*/
const siteConfig = {
tld: '.home',
caName: '',
dnsServerIp: '',
dnsServerPort: CADDY.DEFAULT_DNS_PORT,
dashboardHost: '',
timezone: 'UTC',
dnsServers: {},
configurationType: 'homelab',
domain: '',
routingMode: 'subdomain',
};
/**
* Load site configuration from config.json
* @param {string} configFilePath - Path to config.json
* @param {object} log - Logger instance (optional, may not be available at startup)
*/
function loadSiteConfig(configFilePath, log) {
try {
if (fs.existsSync(configFilePath)) {
const raw = JSON.parse(fs.readFileSync(configFilePath, 'utf8'));
// Validate config and log any issues
const { valid, errors: configErrors, warnings: configWarnings } = validateConfig(raw);
if (log && log.warn) {
if (!valid) {
log.warn('config', 'Config validation errors', { errors: configErrors });
}
for (const w of configWarnings) {
log.warn('config', w);
}
}
// Apply config values
siteConfig.tld = raw.tld || '.home';
if (!siteConfig.tld.startsWith('.')) {
siteConfig.tld = `.${siteConfig.tld}`;
}
siteConfig.caName = raw.caName || '';
siteConfig.dnsServerIp = (raw.dns && raw.dns.ip) || '';
siteConfig.dnsServerPort = (raw.dns && raw.dns.port) || CADDY.DEFAULT_DNS_PORT;
siteConfig.dashboardHost = raw.dashboardHost || `status${siteConfig.tld}`;
siteConfig.timezone = raw.timezone || 'UTC';
siteConfig.dnsServers = raw.dnsServers || {};
siteConfig.configurationType = raw.configurationType || 'homelab';
siteConfig.domain = raw.domain || '';
siteConfig.routingMode = raw.routingMode || 'subdomain';
}
} catch (e) {
if (log && log.error) {
log.error('config', 'Failed to load site config', { error: e.message });
} else {
console.error('[ERROR] Failed to load site config:', e.message);
}
}
}
/**
* Build a domain from subdomain + configured TLD or public domain
* @param {string} subdomain - Service subdomain (e.g., 'sonarr')
* @returns {string} Full domain (e.g., 'sonarr.home' or 'sonarr.example.com')
*/
function buildDomain(subdomain) {
if (siteConfig.configurationType === 'public' && siteConfig.domain) {
return `${subdomain}.${siteConfig.domain}`;
}
return `${subdomain}${siteConfig.tld}`;
}
/**
* Build full service URL (protocol + host + path) for a given subdomain
* Subdirectory mode: https://example.com/sonarr
* Subdomain mode: https://sonarr.example.com
* @param {string} subdomain - Service subdomain
* @returns {string} Full service URL
*/
function buildServiceUrl(subdomain) {
if (siteConfig.routingMode === 'subdirectory' && siteConfig.domain) {
return `https://${siteConfig.domain}/${subdomain}`;
}
return `https://${buildDomain(subdomain)}`;
}
module.exports = {
siteConfig,
loadSiteConfig,
buildDomain,
buildServiceUrl,
};

View File

@@ -0,0 +1,41 @@
/**
* Async route handler wrapper
* Catches async errors and passes them to Express error middleware
*/
const { AppError } = require('../../errors');
const { safeErrorMessage } = require('./safe-error');
/**
* Wrap async Express route handlers to catch errors
* @param {Function} fn - async (req, res, next) handler
* @param {string} [context] - label for logError (defaults to req.path)
* @returns {Function} Express middleware
*/
function asyncHandler(fn, context) {
return async (req, res, next) => {
try {
await fn(req, res, next);
} catch (error) {
// Let typed errors (AppError subclasses) propagate to the global error handler
if (error instanceof AppError) {
return next(error);
}
// Log error (requires logger to be injected)
if (req.app.locals.logError) {
await req.app.locals.logError(context || req.path, error);
}
// Send error response if headers haven't been sent
if (!res.headersSent && req.app.locals.errorResponse) {
req.app.locals.errorResponse(res, 500, safeErrorMessage(error));
} else if (!res.headersSent) {
// Fallback if errorResponse not available
res.status(500).json({ success: false, error: safeErrorMessage(error) });
}
}
};
}
module.exports = asyncHandler;

View File

@@ -0,0 +1,15 @@
/**
* Utility functions
* Common helpers used across the API
*/
const asyncHandler = require('./async-handler');
const { errorResponse, ok } = require('./responses');
const { safeErrorMessage } = require('./safe-error');
module.exports = {
asyncHandler,
errorResponse,
ok,
safeErrorMessage,
};

View File

@@ -0,0 +1,30 @@
/**
* Standard HTTP response helpers
*/
/**
* Standard error response — always returns { success: false, error, ...extras }
* @param {object} res - Express response object
* @param {number} statusCode - HTTP status code
* @param {string} message - Error message
* @param {object} extras - Additional fields to include
* @returns {object} Express response
*/
function errorResponse(res, statusCode, message, extras = {}) {
return res.status(statusCode).json({ success: false, error: message, ...extras });
}
/**
* Standard success response — always returns { success: true, ...data }
* @param {object} res - Express response object
* @param {object} data - Data to include in response
* @returns {object} Express response
*/
function ok(res, data = {}) {
return res.json({ success: true, ...data });
}
module.exports = {
errorResponse,
ok,
};

View File

@@ -0,0 +1,31 @@
/**
* Safe error message sanitization
* Prevents leaking internal paths, stack traces, etc. to clients
*/
/**
* Return a safe error message to the client without leaking internals
* @param {Error|string} error - Error object or string
* @returns {string} Sanitized error message safe for client consumption
*/
function safeErrorMessage(error) {
const msg = error.message || String(error);
// Detect port conflict errors from Docker
const portMatch = msg.match(/exposing port TCP [^:]*:(\d+)/);
if (portMatch || msg.includes('port is already allocated') || msg.includes('ports are not available')) {
const port = portMatch ? portMatch[1] : 'requested';
return `[DC-200] Port ${port} is already in use. Try a different port in Advanced Options, or stop the service using that port first.`;
}
// Only expose messages that are clearly user-facing (short, no paths/stack frames)
if (msg.length < 200 && !msg.includes('/') && !msg.includes('\\') && !msg.includes(' at ')) {
return msg;
}
return 'An internal error occurred';
}
module.exports = {
safeErrorMessage,
};