dashcaddy/DESLOPIFICATION-ROADMAP.md

# DashCaddy API Deslopification Roadmap

**Audited:** 2026-03-22  
**Version:** 1.1.0  
**Total Lines:** ~26,000 (API), ~10,000 (dashboard)  
**Priority:** API-first (make backend powerful, clean dashboard follows naturally)

---

## Executive Summary

The DashCaddy API is **feature-complete and security-hardened**, but the codebase shows signs of rapid evolution. While functionally robust, it would significantly benefit from architectural refactoring to improve maintainability, testability, and long-term scalability.

### Key Strengths
✅ Comprehensive feature set (76+ app templates, Docker/Caddy/DNS management)  
✅ Security-conscious (TOTP auth, AES-256-GCM credentials, CSRF protection, audit logging)  
✅ Recent test coverage additions (auth, credentials, Docker security)  
✅ Modular route organization (routes/ subdirectories)  
✅ Shared context pattern for dependency injection  

### Core Issues
❌ **Monolithic `server.js`** (1960 lines) — initialization, middleware, utilities, business logic all in one file  
❌ **God object `ctx`** — 50+ properties/methods across multiple domains with hidden dependencies  
❌ **Inconsistent patterns** — routes use classes, factory functions, or flat modules with no standard  
❌ **No code standards** — ESLint installed but no config, no formatting rules  
❌ **Mixed concerns** — HTTP handlers, business logic, validation intertwined in route files  

---

## Current Architecture

```
dashcaddy-api/
├── server.js (1960 lines) ← MAIN PROBLEM
│   ├── 89 require() statements
│   ├── 131 top-level declarations
│   ├── Middleware setup
│   ├── Context (`ctx`) assembly (50+ properties)
│   ├── Route mounting
│   ├── Error handlers
│   └── Server startup
├── routes/
│   ├── auth/ (5 files, modular) ✅
│   ├── config/ (4 files, modular) ✅
│   ├── apps/ (6 files, helpers pattern) ⚠️
│   ├── arr/ (4 files, helpers pattern) ⚠️
│   ├── recipes/ (3 files) ⚠️
│   └── *.js (19 flat route files) ❌
├── Managers (clean, well-separated)
│   ├── auth-manager.js (307 lines) ✅
│   ├── credential-manager.js (395 lines) ✅
│   ├── state-manager.js (237 lines) ✅
│   ├── backup-manager.js (835 lines) ⚠️
│   ├── health-checker.js (591 lines) ⚠️
│   └── update-manager.js (911 lines) ⚠️
├── Utilities
│   ├── input-validator.js (606 lines) ⚠️
│   ├── crypto-utils.js (340 lines) ✅
│   ├── middleware.js (430 lines) ⚠️
│   └── constants.js ✅
└── Templates
    ├── app-templates.js (2496 lines) ⚠️
    └── recipe-templates.js (339 lines) ✅
```

**Legend:**  
✅ Good structure  
⚠️ Works but could be cleaner  
❌ Needs refactoring  

---

## Deslopification Phases

### Phase 1: Foundation & Standards (IMMEDIATE)
**Goal:** Establish code quality baseline before refactoring  
**Effort:** 2-4 hours  
**Risk:** Low (tooling only, no code changes)

#### 1.1 Code Standards Setup
- [ ] Create `.eslintrc.js` with recommended rules
- [ ] Add Prettier config (`.prettierrc`)
- [ ] Add npm scripts: `lint`, `lint:fix`, `format`
- [ ] Run `npm run lint:fix` and commit baseline cleanup
- [ ] Add pre-commit hooks (optional)

**Why first:** Establish formatting/style consistency before making structural changes. Prevents "should I refactor this while I'm here?" scope creep.

#### 1.2 Dependency Graph Documentation
- [ ] Map `ctx` properties → which routes actually use them
- [ ] Identify circular dependencies (if any)
- [ ] Document shared utilities used across routes

**Deliverable:** `DEPENDENCIES.md` — reference for refactoring decisions

---

### Phase 2: Extract & Organize (HIGH PRIORITY)
**Goal:** Break `server.js` into logical modules  
**Effort:** 1-2 days  
**Risk:** Medium (requires testing at each step)

#### 2.1 Split `server.js` into Layers
**Before:** 1960-line monolith  
**After:** Clean initialization flow

Create new structure:
```
src/
├── app.js              ← Express app setup (middleware, routes)
├── server.js           ← Entry point (load config, start server)
├── config/
│   ├── index.js        ← Load all config (env, files, constants)
│   ├── env.js          ← Environment variable validation
│   └── paths.js        ← Platform-specific paths
├── context/
│   ├── index.js        ← Assemble context (DI container)
│   ├── docker.js       ← Docker-related context properties
│   ├── caddy.js        ← Caddy-related context properties
│   ├── dns.js          ← DNS context
│   ├── session.js      ← Session context
│   └── notification.js ← Notification context
├── middleware/
│   ├── index.js        ← Export all middleware
│   ├── auth.js         ← Move from middleware.js
│   ├── error.js        ← Error handlers
│   └── security.js     ← Helmet, CORS, CSRF
└── routes/
    └── (existing structure)
```

**Migration Steps:**
1. Create `src/config/` — extract all config loading from `server.js`
2. Create `src/context/` — split god object into domain modules
3. Create `src/middleware/` — break up `middleware.js` (430 lines)
4. Create `src/app.js` — Express setup + route mounting
5. Slim `server.js` → minimal entry point (~50 lines)

**Tests:** Ensure existing test suite still passes after each step

---

### Phase 3: Route Standardization (MEDIUM PRIORITY)
**Goal:** Consistent route module pattern across entire API  
**Effort:** 2-3 days  
**Risk:** Medium (touching business logic)

#### 3.1 Establish Route Pattern
**Chosen Pattern:** Factory function with explicit dependencies

```javascript
// routes/services.js (before)
module.exports = (ctx) => {
  const router = express.Router();
  // ... uses ctx.docker, ctx.servicesStateManager, ctx.log, etc.
  return router;
};

// routes/services.js (after)
module.exports = ({ docker, servicesStateManager, log, asyncHandler }) => {
  const router = express.Router();
  // ... explicitly passed dependencies
  return router;
};
```

**Benefits:**
- Self-documenting (you see what each route needs)
- Easier testing (mock only what's used)
- No hidden dependencies via god object

#### 3.2 Refactor Routes by Priority
**Order:** Most-used routes first

1. **High-traffic routes:**
   - `routes/services.js` (467 lines) — core service management
   - `routes/containers.js` (246 lines) — Docker operations
   - `routes/health.js` (297 lines) — health checks
   - `routes/dns.js` (632 lines) — DNS management

2. **Auth routes** (already modular, just align pattern):
   - `routes/auth/*`

3. **Feature routes:**
   - `routes/apps/*`
   - `routes/arr/*`
   - `routes/recipes/*`

4. **Utility routes:**
   - `routes/logs.js`
   - `routes/backups.js`
   - `routes/ca.js`
   - etc.

**Per-route checklist:**
- [ ] Extract dependencies from `ctx` → explicit parameters
- [ ] Move business logic to service layer (if complex)
- [ ] Validate inputs at route boundary
- [ ] Return consistent error format
- [ ] Add route-level tests

---

### Phase 4: Service Layer Introduction (LOWER PRIORITY)
**Goal:** Separate business logic from HTTP handlers  
**Effort:** 3-5 days  
**Risk:** Medium-High (significant refactor)

**Problem:** Routes currently mix HTTP concerns with business logic:
```javascript
// Current: Everything in route handler
router.post('/deploy', async (req, res) => {
  // 1. Parse request
  // 2. Validate inputs
  // 3. Business logic (complex Docker operations)
  // 4. Error handling
  // 5. Format response
});
```

**Solution:** Service layer pattern
```javascript
// routes/apps/deploy.js
router.post('/deploy', async (req, res) => {
  const result = await appDeployService.deploy(req.body);
  res.json({ success: true, data: result });
});

// services/app-deploy-service.js
class AppDeployService {
  async deploy({ templateId, config }) {
    // Pure business logic, no HTTP awareness
  }
}
```

**Candidates for service extraction:**
- `services/docker-service.js` — container lifecycle, networking
- `services/caddy-service.js` — Caddyfile manipulation, reload
- `services/dns-service.js` — record management, zone operations
- `services/app-deploy-service.js` — template-based deployment
- `services/backup-service.js` — backup/restore workflows

**Benefits:**
- Routes become thin HTTP adapters (easy to test)
- Business logic testable without HTTP mocking
- Reusable across routes (e.g., CLI tools, cron jobs)

---

### Phase 5: Manager Cleanup (ONGOING)
**Goal:** Refine existing manager modules  
**Effort:** 1-2 days (parallel to other phases)

#### Issues to Address
1. **`backup-manager.js` (835 lines)** — too large, split backup vs restore logic
2. **`update-manager.js` (911 lines)** — complex state machine, extract version comparison utilities
3. **`health-checker.js` (591 lines)** — separate health check logic from notification daemon
4. **`input-validator.js` (606 lines)** — split by domain (docker, caddy, dns validators)

**Approach:** Incremental splitting, preserve existing API

---

### Phase 6: Template Organization (LOW PRIORITY)
**Goal:** Make templates maintainable and extensible  
**Effort:** 1 day

**Problem:** `app-templates.js` is 2496 lines (76 templates in one file)

**Solution:**
```
templates/
├── index.js           ← Export TEMPLATE_CATEGORIES, DIFFICULTY_LEVELS
├── apps/
│   ├── media/
│   │   ├── plex.js
│   │   ├── jellyfin.js
│   │   └── ...
│   ├── automation/
│   └── ...
└── recipes/
    ├── arr-stack.js
    └── ...
```

**Benefits:**
- Easier to find/edit specific templates
- Contributors can add templates without merge conflicts
- Templates can import shared snippets (e.g., common env vars)

---

## Metrics & Success Criteria

### Code Quality Metrics (Before → After)

| Metric | Before | Target | How to Measure |
|--------|--------|--------|----------------|
| `server.js` lines | 1960 | <200 | `wc -l server.js` |
| Avg route file size | ~300 | <150 | `find routes -name '*.js' -exec wc -l {} + \| awk '{sum+=$1; n++} END {print sum/n}'` |
| `ctx` properties | 50+ | 0 (removed) | Manual count |
| ESLint errors | Unknown | 0 | `npm run lint` |
| Test coverage | ~30% | >60% | `npm run test:coverage` |
| Files >500 lines | 8 | <3 | `find . -name '*.js' -exec wc -l {} + \| awk '$1 > 500'` |

### Developer Experience Improvements
- **Onboarding:** New contributor should understand route structure in <10 minutes
- **Testing:** Mock only what you use (no god object sprawl)
- **Changes:** Touching one domain shouldn't require understanding entire codebase
- **Deployment:** Confidence that refactor didn't break anything (test suite)

---

## Risk Mitigation

### How to Refactor Safely

1. **Test suite first** — before touching code:
   - Run existing tests: `npm test`
   - Identify untested critical paths → add tests
   - Establish coverage baseline

2. **Incremental changes**:
   - Each phase = separate branch
   - Each phase passes full test suite
   - Deploy to test environment (Contabo) before merging

3. **Preserve API contract**:
   - Frontend expects same endpoints/responses
   - Dashboard shouldn't need changes during API refactor
   - Version routes if breaking changes needed

4. **Rollback plan**:
   - Git tags before each phase merge
   - Keep old code in `legacy/` until confidence is high
   - Document what changed in each PR

---

## Recommended Order of Execution

**Week 1: Foundation**
- Day 1-2: Phase 1 (ESLint, Prettier, dependency mapping)
- Day 3-5: Phase 2.1 (split `server.js`)

**Week 2: Routes**
- Day 1-3: Phase 3.1 (standardize top 5 routes)
- Day 4-5: Phase 3.2 (remaining routes)

**Week 3: Refinement**
- Day 1-3: Phase 4 (service layer for complex routes)
- Day 4-5: Phase 5 (manager cleanup)

**Week 4: Polish**
- Day 1-2: Phase 6 (template organization)
- Day 3-5: Documentation, final testing, deployment

**Total:** ~4 weeks part-time or ~2 weeks full-time

---

## Questions for Sami

Before starting, clarify:

1. **Testing strategy:** Current test coverage is partial. Should we:
   - Write tests BEFORE refactoring (safer, slower)?
   - Refactor with existing tests, add coverage later (faster, riskier)?

2. **Breaking changes:** Can we introduce backwards-incompatible API changes if we version routes (`/api/v2/...`)?

3. **Deployment cadence:** Should each phase deploy to production, or batch into one big release?

4. **Priority tweaks:** Does this roadmap align with "deslopify → market → sell" timeline, or should we focus only on the most visible pain points first?

---

## Next Steps

**If approved:**
1. Create feature branch: `refactor/deslopification-phase-1`
2. Add ESLint + Prettier configs
3. Run `npm run lint:fix` and commit baseline
4. Create `DEPENDENCIES.md` (ctx usage map)
5. Review with Sami before proceeding to Phase 2

**Estimated time to first visible improvement:** 1 week (server.js split + linting)