Phase 2 (WIP): Extract config and utils modules
- Created src/config/ (env.js, site-config.js) - Created src/utils/ (async-handler.js, responses.js, safe-error.js) - server.js not yet modified (backward-compatible extraction)
This commit is contained in:
Krystie
388
DESLOPIFICATION-ROADMAP.md
Normal file
388
DESLOPIFICATION-ROADMAP.md
Normal file
@@ -0,0 +1,388 @@
|
||||
# DashCaddy API Deslopification Roadmap
|
||||
|
||||
**Audited:** 2026-03-22
|
||||
**Version:** 1.1.0
|
||||
**Total Lines:** ~26,000 (API), ~10,000 (dashboard)
|
||||
**Priority:** API-first (make backend powerful, clean dashboard follows naturally)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The DashCaddy API is **feature-complete and security-hardened**, but the codebase shows signs of rapid evolution. While functionally robust, it would significantly benefit from architectural refactoring to improve maintainability, testability, and long-term scalability.
|
||||
|
||||
### Key Strengths
|
||||
✅ Comprehensive feature set (76+ app templates, Docker/Caddy/DNS management)
|
||||
✅ Security-conscious (TOTP auth, AES-256-GCM credentials, CSRF protection, audit logging)
|
||||
✅ Recent test coverage additions (auth, credentials, Docker security)
|
||||
✅ Modular route organization (routes/ subdirectories)
|
||||
✅ Shared context pattern for dependency injection
|
||||
|
||||
### Core Issues
|
||||
❌ **Monolithic `server.js`** (1960 lines) — initialization, middleware, utilities, business logic all in one file
|
||||
❌ **God object `ctx`** — 50+ properties/methods across multiple domains with hidden dependencies
|
||||
❌ **Inconsistent patterns** — routes use classes, factory functions, or flat modules with no standard
|
||||
❌ **No code standards** — ESLint installed but no config, no formatting rules
|
||||
❌ **Mixed concerns** — HTTP handlers, business logic, validation intertwined in route files
|
||||
|
||||
---
|
||||
|
||||
## Current Architecture
|
||||
|
||||
```
|
||||
dashcaddy-api/
|
||||
├── server.js (1960 lines) ← MAIN PROBLEM
|
||||
│ ├── 89 require() statements
|
||||
│ ├── 131 top-level declarations
|
||||
│ ├── Middleware setup
|
||||
│ ├── Context (`ctx`) assembly (50+ properties)
|
||||
│ ├── Route mounting
|
||||
│ ├── Error handlers
|
||||
│ └── Server startup
|
||||
├── routes/
|
||||
│ ├── auth/ (5 files, modular) ✅
|
||||
│ ├── config/ (4 files, modular) ✅
|
||||
│ ├── apps/ (6 files, helpers pattern) ⚠️
|
||||
│ ├── arr/ (4 files, helpers pattern) ⚠️
|
||||
│ ├── recipes/ (3 files) ⚠️
|
||||
│ └── *.js (19 flat route files) ❌
|
||||
├── Managers (clean, well-separated)
|
||||
│ ├── auth-manager.js (307 lines) ✅
|
||||
│ ├── credential-manager.js (395 lines) ✅
|
||||
│ ├── state-manager.js (237 lines) ✅
|
||||
│ ├── backup-manager.js (835 lines) ⚠️
|
||||
│ ├── health-checker.js (591 lines) ⚠️
|
||||
│ └── update-manager.js (911 lines) ⚠️
|
||||
├── Utilities
|
||||
│ ├── input-validator.js (606 lines) ⚠️
|
||||
│ ├── crypto-utils.js (340 lines) ✅
|
||||
│ ├── middleware.js (430 lines) ⚠️
|
||||
│ └── constants.js ✅
|
||||
└── Templates
|
||||
├── app-templates.js (2496 lines) ⚠️
|
||||
└── recipe-templates.js (339 lines) ✅
|
||||
```
|
||||
|
||||
**Legend:**
|
||||
✅ Good structure
|
||||
⚠️ Works but could be cleaner
|
||||
❌ Needs refactoring
|
||||
|
||||
---
|
||||
|
||||
## Deslopification Phases
|
||||
|
||||
### Phase 1: Foundation & Standards (IMMEDIATE)
|
||||
**Goal:** Establish code quality baseline before refactoring
|
||||
**Effort:** 2-4 hours
|
||||
**Risk:** Low (tooling only, no code changes)
|
||||
|
||||
#### 1.1 Code Standards Setup
|
||||
- [ ] Create `.eslintrc.js` with recommended rules
|
||||
- [ ] Add Prettier config (`.prettierrc`)
|
||||
- [ ] Add npm scripts: `lint`, `lint:fix`, `format`
|
||||
- [ ] Run `npm run lint:fix` and commit baseline cleanup
|
||||
- [ ] Add pre-commit hooks (optional)
|
||||
|
||||
**Why first:** Establish formatting/style consistency before making structural changes. Prevents "should I refactor this while I'm here?" scope creep.
|
||||
|
||||
#### 1.2 Dependency Graph Documentation
|
||||
- [ ] Map `ctx` properties → which routes actually use them
|
||||
- [ ] Identify circular dependencies (if any)
|
||||
- [ ] Document shared utilities used across routes
|
||||
|
||||
**Deliverable:** `DEPENDENCIES.md` — reference for refactoring decisions
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Extract & Organize (HIGH PRIORITY)
|
||||
**Goal:** Break `server.js` into logical modules
|
||||
**Effort:** 1-2 days
|
||||
**Risk:** Medium (requires testing at each step)
|
||||
|
||||
#### 2.1 Split `server.js` into Layers
|
||||
**Before:** 1960-line monolith
|
||||
**After:** Clean initialization flow
|
||||
|
||||
Create new structure:
|
||||
```
|
||||
src/
|
||||
├── app.js ← Express app setup (middleware, routes)
|
||||
├── server.js ← Entry point (load config, start server)
|
||||
├── config/
|
||||
│ ├── index.js ← Load all config (env, files, constants)
|
||||
│ ├── env.js ← Environment variable validation
|
||||
│ └── paths.js ← Platform-specific paths
|
||||
├── context/
|
||||
│ ├── index.js ← Assemble context (DI container)
|
||||
│ ├── docker.js ← Docker-related context properties
|
||||
│ ├── caddy.js ← Caddy-related context properties
|
||||
│ ├── dns.js ← DNS context
|
||||
│ ├── session.js ← Session context
|
||||
│ └── notification.js ← Notification context
|
||||
├── middleware/
|
||||
│ ├── index.js ← Export all middleware
|
||||
│ ├── auth.js ← Move from middleware.js
|
||||
│ ├── error.js ← Error handlers
|
||||
│ └── security.js ← Helmet, CORS, CSRF
|
||||
└── routes/
|
||||
└── (existing structure)
|
||||
```
|
||||
|
||||
**Migration Steps:**
|
||||
1. Create `src/config/` — extract all config loading from `server.js`
|
||||
2. Create `src/context/` — split god object into domain modules
|
||||
3. Create `src/middleware/` — break up `middleware.js` (430 lines)
|
||||
4. Create `src/app.js` — Express setup + route mounting
|
||||
5. Slim `server.js` → minimal entry point (~50 lines)
|
||||
|
||||
**Tests:** Ensure existing test suite still passes after each step
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Route Standardization (MEDIUM PRIORITY)
|
||||
**Goal:** Consistent route module pattern across entire API
|
||||
**Effort:** 2-3 days
|
||||
**Risk:** Medium (touching business logic)
|
||||
|
||||
#### 3.1 Establish Route Pattern
|
||||
**Chosen Pattern:** Factory function with explicit dependencies
|
||||
|
||||
```javascript
|
||||
// routes/services.js (before)
|
||||
module.exports = (ctx) => {
|
||||
const router = express.Router();
|
||||
// ... uses ctx.docker, ctx.servicesStateManager, ctx.log, etc.
|
||||
return router;
|
||||
};
|
||||
|
||||
// routes/services.js (after)
|
||||
module.exports = ({ docker, servicesStateManager, log, asyncHandler }) => {
|
||||
const router = express.Router();
|
||||
// ... explicitly passed dependencies
|
||||
return router;
|
||||
};
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Self-documenting (you see what each route needs)
|
||||
- Easier testing (mock only what's used)
|
||||
- No hidden dependencies via god object
|
||||
|
||||
#### 3.2 Refactor Routes by Priority
|
||||
**Order:** Most-used routes first
|
||||
|
||||
1. **High-traffic routes:**
|
||||
- `routes/services.js` (467 lines) — core service management
|
||||
- `routes/containers.js` (246 lines) — Docker operations
|
||||
- `routes/health.js` (297 lines) — health checks
|
||||
- `routes/dns.js` (632 lines) — DNS management
|
||||
|
||||
2. **Auth routes** (already modular, just align pattern):
|
||||
- `routes/auth/*`
|
||||
|
||||
3. **Feature routes:**
|
||||
- `routes/apps/*`
|
||||
- `routes/arr/*`
|
||||
- `routes/recipes/*`
|
||||
|
||||
4. **Utility routes:**
|
||||
- `routes/logs.js`
|
||||
- `routes/backups.js`
|
||||
- `routes/ca.js`
|
||||
- etc.
|
||||
|
||||
**Per-route checklist:**
|
||||
- [ ] Extract dependencies from `ctx` → explicit parameters
|
||||
- [ ] Move business logic to service layer (if complex)
|
||||
- [ ] Validate inputs at route boundary
|
||||
- [ ] Return consistent error format
|
||||
- [ ] Add route-level tests
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Service Layer Introduction (LOWER PRIORITY)
|
||||
**Goal:** Separate business logic from HTTP handlers
|
||||
**Effort:** 3-5 days
|
||||
**Risk:** Medium-High (significant refactor)
|
||||
|
||||
**Problem:** Routes currently mix HTTP concerns with business logic:
|
||||
```javascript
|
||||
// Current: Everything in route handler
|
||||
router.post('/deploy', async (req, res) => {
|
||||
// 1. Parse request
|
||||
// 2. Validate inputs
|
||||
// 3. Business logic (complex Docker operations)
|
||||
// 4. Error handling
|
||||
// 5. Format response
|
||||
});
|
||||
```
|
||||
|
||||
**Solution:** Service layer pattern
|
||||
```javascript
|
||||
// routes/apps/deploy.js
|
||||
router.post('/deploy', async (req, res) => {
|
||||
const result = await appDeployService.deploy(req.body);
|
||||
res.json({ success: true, data: result });
|
||||
});
|
||||
|
||||
// services/app-deploy-service.js
|
||||
class AppDeployService {
|
||||
async deploy({ templateId, config }) {
|
||||
// Pure business logic, no HTTP awareness
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Candidates for service extraction:**
|
||||
- `services/docker-service.js` — container lifecycle, networking
|
||||
- `services/caddy-service.js` — Caddyfile manipulation, reload
|
||||
- `services/dns-service.js` — record management, zone operations
|
||||
- `services/app-deploy-service.js` — template-based deployment
|
||||
- `services/backup-service.js` — backup/restore workflows
|
||||
|
||||
**Benefits:**
|
||||
- Routes become thin HTTP adapters (easy to test)
|
||||
- Business logic testable without HTTP mocking
|
||||
- Reusable across routes (e.g., CLI tools, cron jobs)
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Manager Cleanup (ONGOING)
|
||||
**Goal:** Refine existing manager modules
|
||||
**Effort:** 1-2 days (parallel to other phases)
|
||||
|
||||
#### Issues to Address
|
||||
1. **`backup-manager.js` (835 lines)** — too large, split backup vs restore logic
|
||||
2. **`update-manager.js` (911 lines)** — complex state machine, extract version comparison utilities
|
||||
3. **`health-checker.js` (591 lines)** — separate health check logic from notification daemon
|
||||
4. **`input-validator.js` (606 lines)** — split by domain (docker, caddy, dns validators)
|
||||
|
||||
**Approach:** Incremental splitting, preserve existing API
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Template Organization (LOW PRIORITY)
|
||||
**Goal:** Make templates maintainable and extensible
|
||||
**Effort:** 1 day
|
||||
|
||||
**Problem:** `app-templates.js` is 2496 lines (76 templates in one file)
|
||||
|
||||
**Solution:**
|
||||
```
|
||||
templates/
|
||||
├── index.js ← Export TEMPLATE_CATEGORIES, DIFFICULTY_LEVELS
|
||||
├── apps/
|
||||
│ ├── media/
|
||||
│ │ ├── plex.js
|
||||
│ │ ├── jellyfin.js
|
||||
│ │ └── ...
|
||||
│ ├── automation/
|
||||
│ └── ...
|
||||
└── recipes/
|
||||
├── arr-stack.js
|
||||
└── ...
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Easier to find/edit specific templates
|
||||
- Contributors can add templates without merge conflicts
|
||||
- Templates can import shared snippets (e.g., common env vars)
|
||||
|
||||
---
|
||||
|
||||
## Metrics & Success Criteria
|
||||
|
||||
### Code Quality Metrics (Before → After)
|
||||
|
||||
| Metric | Before | Target | How to Measure |
|
||||
|--------|--------|--------|----------------|
|
||||
| `server.js` lines | 1960 | <200 | `wc -l server.js` |
|
||||
| Avg route file size | ~300 | <150 | `find routes -name '*.js' -exec wc -l {} + \| awk '{sum+=$1; n++} END {print sum/n}'` |
|
||||
| `ctx` properties | 50+ | 0 (removed) | Manual count |
|
||||
| ESLint errors | Unknown | 0 | `npm run lint` |
|
||||
| Test coverage | ~30% | >60% | `npm run test:coverage` |
|
||||
| Files >500 lines | 8 | <3 | `find . -name '*.js' -exec wc -l {} + \| awk '$1 > 500'` |
|
||||
|
||||
### Developer Experience Improvements
|
||||
- **Onboarding:** New contributor should understand route structure in <10 minutes
|
||||
- **Testing:** Mock only what you use (no god object sprawl)
|
||||
- **Changes:** Touching one domain shouldn't require understanding entire codebase
|
||||
- **Deployment:** Confidence that refactor didn't break anything (test suite)
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
### How to Refactor Safely
|
||||
|
||||
1. **Test suite first** — before touching code:
|
||||
- Run existing tests: `npm test`
|
||||
- Identify untested critical paths → add tests
|
||||
- Establish coverage baseline
|
||||
|
||||
2. **Incremental changes**:
|
||||
- Each phase = separate branch
|
||||
- Each phase passes full test suite
|
||||
- Deploy to test environment (Contabo) before merging
|
||||
|
||||
3. **Preserve API contract**:
|
||||
- Frontend expects same endpoints/responses
|
||||
- Dashboard shouldn't need changes during API refactor
|
||||
- Version routes if breaking changes needed
|
||||
|
||||
4. **Rollback plan**:
|
||||
- Git tags before each phase merge
|
||||
- Keep old code in `legacy/` until confidence is high
|
||||
- Document what changed in each PR
|
||||
|
||||
---
|
||||
|
||||
## Recommended Order of Execution
|
||||
|
||||
**Week 1: Foundation**
|
||||
- Day 1-2: Phase 1 (ESLint, Prettier, dependency mapping)
|
||||
- Day 3-5: Phase 2.1 (split `server.js`)
|
||||
|
||||
**Week 2: Routes**
|
||||
- Day 1-3: Phase 3.1 (standardize top 5 routes)
|
||||
- Day 4-5: Phase 3.2 (remaining routes)
|
||||
|
||||
**Week 3: Refinement**
|
||||
- Day 1-3: Phase 4 (service layer for complex routes)
|
||||
- Day 4-5: Phase 5 (manager cleanup)
|
||||
|
||||
**Week 4: Polish**
|
||||
- Day 1-2: Phase 6 (template organization)
|
||||
- Day 3-5: Documentation, final testing, deployment
|
||||
|
||||
**Total:** ~4 weeks part-time or ~2 weeks full-time
|
||||
|
||||
---
|
||||
|
||||
## Questions for Sami
|
||||
|
||||
Before starting, clarify:
|
||||
|
||||
1. **Testing strategy:** Current test coverage is partial. Should we:
|
||||
- Write tests BEFORE refactoring (safer, slower)?
|
||||
- Refactor with existing tests, add coverage later (faster, riskier)?
|
||||
|
||||
2. **Breaking changes:** Can we introduce backwards-incompatible API changes if we version routes (`/api/v2/...`)?
|
||||
|
||||
3. **Deployment cadence:** Should each phase deploy to production, or batch into one big release?
|
||||
|
||||
4. **Priority tweaks:** Does this roadmap align with "deslopify → market → sell" timeline, or should we focus only on the most visible pain points first?
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
**If approved:**
|
||||
1. Create feature branch: `refactor/deslopification-phase-1`
|
||||
2. Add ESLint + Prettier configs
|
||||
3. Run `npm run lint:fix` and commit baseline
|
||||
4. Create `DEPENDENCIES.md` (ctx usage map)
|
||||
5. Review with Sami before proceeding to Phase 2
|
||||
|
||||
**Estimated time to first visible improvement:** 1 week (server.js split + linting)
|
||||
Reference in New Issue
Block a user