Compare commits
27
Commits
f9eaa324dd
...
dc/DC-067
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1528fd1a35 | ||
|
|
432e9635bc | ||
|
|
5b74536472 | ||
|
|
bb01a77ae7 | ||
|
|
88ff260e5e | ||
|
|
ff81d99021 | ||
|
|
5c02bfba1d | ||
|
|
bb20f02cbf | ||
|
|
dc788e5dd3 | ||
|
|
04f90d1505 | ||
|
|
bd13104362 | ||
|
|
dcf252e515 | ||
|
|
a7512b4a56 | ||
|
|
f5fc688185 | ||
|
|
1bc41bb2bc | ||
|
|
4dda005eb1 | ||
|
|
140aa5d4b1 | ||
|
|
bf1bcb1133 | ||
|
|
7b04bc1d3c | ||
|
|
191d3340a7 | ||
|
|
84f63a3261 | ||
|
|
f2c6fa69f5 | ||
|
|
c55abdab87 | ||
|
|
0bf4406253 | ||
|
|
cbc5dc96c8 | ||
|
|
e8b9dd5b91 | ||
|
|
baba762dab |
@@ -0,0 +1 @@
|
||||
node_modules
|
||||
+11
-2
@@ -180,10 +180,11 @@
|
||||
- **result:** Verified zero callers (grep + 38 test files scanned — no references to `./self-updater`). Discovered the file was actually gitignored, never committed — so `git rm` was unnecessary; plain `rm` did it. Tests: 1075/1075 still passing post-delete. Also synced `dashcaddy-api/VERSION` to `42376e2` (the DC-033 + release-bump commit) so the rebuilt container reports the right SHA. Live: `curl http://127.0.0.1:3001/api/v1/system/version` returns `{"name":"DashCaddy","version":"1.14.9","commit":"42376e2"}`.
|
||||
|
||||
### DC-037: Move `/etc/dashcaddy/sites/dashcaddy-api` symlink creation into the install script
|
||||
- **status:** in-progress
|
||||
- **status:** done
|
||||
- **owner:** krystie
|
||||
- **details:** DNS2 has the symlink manually created during this session (2026-07-05), but every other fresh DashCaddy install will hit the same `cp: cannot create directory '/etc/dashcaddy/sites/dashcaddy-api/routes': No such file or directory` failure when the first auto-update lands, because `dashcaddy-update.sh` defaults `apiSourceDir` to `${CADDY_BASE}/sites/dashcaddy-api` (= `/etc/dashcaddy/sites/dashcaddy-api`) while the actual install lives at `/opt/dashcaddy/dashcaddy-api`. Fix: add `mkdir -p /etc/dashcaddy/sites && ln -sfn /opt/dashcaddy/dashcaddy-api /etc/dashcaddy/sites/dashcaddy-api` to the install script (whichever of `dashcaddy-installer/install.sh` or `scripts/dashcaddy-install.sh` is canonical — verify which exists on a clean install). Make it idempotent (`ln -sfn`, not `ln -s`, so re-runs don't fail). Effort: ~10 min. Risk: very low.
|
||||
- **impact:** Prevents every future DashCaddy host from hitting the v1.14.4-class update failure on first auto-update.
|
||||
- **result:** Added `install_api_symlink()` to `dashcaddy-installer/install.sh`, called from `main()` right after `start_caddy` at end of Step 7. The function does `mkdir -p /opt/dashcaddy && ln -sfn "${API_DIR}" /opt/dashcaddy/dashcaddy-api` (idempotent: `-sfn` replaces stale links and does not fail on re-runs; `${API_DIR}` resolves to `/etc/dashcaddy/sites/dashcaddy-api` per the existing readonly constants at lines 23-26). The `mkdir -p /opt/dashcaddy` ensures the symlink's parent directory exists on a fresh host before `ln -sfn` runs. `bash -n install.sh` returns SYNTAX OK. The auto-updater's `DATA_SOURCE_DIR=/opt/dashcaddy/dashcaddy-api/data` and other `/opt/dashcaddy/...` defaults now resolve cleanly through the symlink on fresh installs. Existing DNS2 host is unaffected (the symlink already exists there from the manual session 2026-07-05; `ln -sfn` would replace it with the same target if re-run).
|
||||
|
||||
---
|
||||
|
||||
@@ -389,5 +390,13 @@ Tickets DC-046 through DC-049 implement pluggable auth + email magic link. Sami
|
||||
- **details:** Backend uses ad-hoc `if (!field) throw new ValidationError(...)` checks at every route entry point — 49 such checks across the codebase. They drift from the field semantics, allow unknown keys to flow through, and have no way to express structured types (CIDR, enum, port range). Tracked in `DC-PRODUCTION-GRADE-BACKLOG.md` as P1-1. Fix: `npm install joi@^18`, add `src/utilities/validate.js` exporting `validateBody(schema)` middleware factory + `schemas` object with reusable schemas. Apply to destructive routes: backups (schedule/restore/config), apps (deploy/restore/revert), assets (upload/logo). Add `__tests__/unit/validate.test.js` covering each schema's accept/reject/strip-unknown behaviour. Effort: ~2 hr.
|
||||
- **impact:** Closes P0-3 / P0-4 class of bugs at the schema layer instead of per-route. Prevents future routes from accepting arbitrary body fields. New routes copy-paste from `schemas.*` and get free validation.
|
||||
- **prerequisite:** None.
|
||||
- **result:** Shipped codex-graded B. New module `src/utilities/validate.js` (170 LOC) with `validateBody(schema, opts)` middleware + 9 Joi schemas. Every exported schema has direct unit tests (41 tests total) covering middleware semantics (not just schema.validate). Applied to 8 destructive routes: backups (schedule/restore/config), apps (deploy/restore/revert), assets (upload/logo). Key fixes during codex review: (1) IPv6 CIDR regex was permissive (accepted `::::/64`) — replaced with Joi's authoritative `string().ip({cidr: 'required'})`. (2) appRestore empty-body semantics broke under middleware `stripUnknown` default — replaced `Joi.object({}).max(0)` with `Joi.any().custom()` that enforces non-empty rejection even after strip. (3) appDeploy.config now uses `.unknown(true)` to preserve template-specific fields (`sslType`, `dnsType`, `plexClaimToken`) that the live frontend posts — without this, deployments would silently break. Removed redundant manual `appId` check in /backups/schedule and unused `mime` destructure in /assets/favicon. Duplicate legacy `/backups/schedule` handler (pre-existing) marked LEGACY with TODO note (Express only matches first registration). 1539/1539 Jest tests pass (was 1498, +41 new). ESLint warnings unchanged (416 total, all pre-existing).
|
||||
- **result:** Shipped codex-graded B. New module `src/utilities/validate.js` (170 LOC) with `validateBody(schema, opts)` middleware + 9 Joi schemas. Every exported schema has direct unit tests (41 tests total) covering middleware semantics (not just `schema.validate`). Applied to 8 destructive routes: backups (schedule/restore/config), apps (deploy/restore/revert), assets (upload/logo). Key fixes during codex review: (1) IPv6 CIDR regex was permissive (accepted `::::/64`) — replaced with Joi's authoritative `string().ip({cidr: 'required'})`. (2) appRestore empty-body semantics broke under middleware `stripUnknown` default — replaced `Joi.object({}).max(0)` with `Joi.any().custom()` that enforces non-empty rejection even after strip. (3) appDeploy.config now uses `.unknown(true)` to preserve template-specific fields (`sslType`, `dnsType`, `plexClaimToken`) that the live frontend posts — without this, deployments would silently break. Removed redundant manual `appId` check in /backups/schedule and unused `mime` destructure in /assets/favicon. Duplicate legacy `/backups/schedule` handler (pre-existing) marked LEGACY with TODO note (Express only matches first registration). 1539/1539 Jest tests pass (was 1498, +41 new). ESLint warnings unchanged (416 total, all pre-existing).
|
||||
|
||||
### DC-060: Console→logger sweep for `src/managers/update-manager.js` (49 sites)
|
||||
- **status:** done
|
||||
- **owner:** hermes
|
||||
- **details:** Production code uses `console.log/warn/error` with `[UpdateManager]` prefixes in 49 places — these go to stdout/stderr directly, bypassing the unified logger (no structured JSON, no error.log file writes, no log-level filtering, no test capture). Tracked in `DC-PRODUCTION-GRADE-BACKLOG.md` as P1-2. Fix: import `log` from `../utils/logging`, replace every `console.log('[UpdateManager] X')` with `log.info('update', 'X')` (dropping the redundant `[UpdateManager]` tag), every `console.warn(...)` with `log.warn('update', ...)`, every `console.error('...', err.message)` with `log.error('update', err)` (passing the error object so it lands in error.log with stack + context). For mixed-content strings like `Stored old image digest: ${oldImageDigest.substring(0, 40)}...` extract the variable into the meta payload: `log.info('update', 'Stored old image digest', { digestPrefix })`. Effort: ~30 min. Risk: very low — pure logging refactor, no behavior change.
|
||||
- **impact:** Update manager events now flow through the same log pipeline as every other module: structured JSON in prod, pretty-printed in dev, error.log rotation for errors, log-level filtering, test capture via stderr spy. Operators get consistent log format and can grep across modules.
|
||||
- **prerequisite:** None.
|
||||
- **result:** Shipped codex-graded A. All 49 `console.*` sites in `src/managers/update-manager.js` now route through `log.info/log.warn/log.error` from `src/utils/logging` (tag = `'update'`). Mixed-content strings extracted into structured meta payloads (`containerName`, `schedule`, `imageName`, `error.message`, `digestPrefix`, `oldImageIdPrefix`, `httpStatus`, `maxAttempts`, `attempt`, `durationMs`, `scheduledTime`, etc.) so fields are queryable instead of inlined into the message. Errors now go through `log.error(ctx, errObj)` so they land in error.log with full stack trace + context, not just stderr. 1539/1539 Jest tests pass (78/78 update-manager tests still pass). ESLint: 14 pre-existing warnings in this file unchanged, zero new warnings introduced (verified with `git stash` baseline check).
|
||||
|
||||
|
||||
@@ -244,7 +244,7 @@ vi /opt/dashcaddy/services.json # live-reloaded by the watcher
|
||||
## Project Info
|
||||
|
||||
- **Name**: DashCaddy
|
||||
- **Version**: 1.13.4 (current; CHANGELOG.md `[Unreleased]` tracks the next bump)
|
||||
- **Version**: 1.15.0 (current; CHANGELOG.md `[Unreleased]` tracks the next bump)
|
||||
- **Purpose**: Unified management for Docker + Caddy + DNS
|
||||
- **Local TLD (Windows)**: `.sami`
|
||||
- **Local TLD (Linux, DNS2)**: `.home` (default; configurable via `siteConfig.tld`)
|
||||
|
||||
+297
-28
@@ -1,37 +1,306 @@
|
||||
# DashCaddy Production-Grade Repair Backlog
|
||||
# DashCaddy Production-Grade Backlog (v2)
|
||||
|
||||
Autonomous agent: work through these IN ORDER. Mark each `[ ]` as `[x]` when shipped.
|
||||
If an item is too big for one tick, implement a sub-part, push that, and note progress.
|
||||
> Generated 2026-08-12 from a full codebase audit.
|
||||
> v1 items (P0-1 through P2-7) are ALL DONE.
|
||||
> Current state: 1539 tests, 86.55% statement coverage, 0 ESLint errors, 173 warnings.
|
||||
|
||||
## P0 — Security & Correctness
|
||||
## Current Health Snapshot
|
||||
- **Tests:** 1539 passing across 63 suites
|
||||
- **Coverage:** Statements 86.55% | Branches 72.14% (below 80% gate) | Functions 80.8% | Lines 90.67%
|
||||
- **ESLint:** 0 errors, 173 warnings (all pre-existing)
|
||||
- **Remaining console.* calls in src/:** 21 across 10 files
|
||||
- **Dockerfile:** Runs as root (documented — needs Docker socket), no resource limits
|
||||
- **OpenAPI spec:** Present but stale (says v1.0.0, actual is v1.15.0)
|
||||
- **Unhandled rejection/exception handlers:** Present in server.js ✓
|
||||
- **Rate limiting:** Present on auth + general routes ✓
|
||||
- **npm audit:** 4 remaining vulns (semver-major transitive deps, deferred)
|
||||
|
||||
- [x] **P0-1: npm audit fix** — Done (commit 3a0a5bc, grade A). Resolved 3 high CVEs via minimatch 9.0.9 in webdav transitive. 4 remaining vulns are semver-major-only (sharp→0.35.3, dockerode→5.0.1, nodemailer→9.0.5, uuid→11.1.1) — deferred per backlog note. All 1498 jest tests pass. URN urn:ump:hlju4hixg3tijbghncigm5gesoemupuczrzmkykumh7xbgkq3d2q.
|
||||
- [x] **P0-2: Command injection in ca.js:210** — Done (commit 66e4460, grade A). Replaced `execSync(\`openssl pkcs12 ... -password "pass:${password}"\`)` with `execFileSync('openssl', [..., '-password', \`pass:${password}\`])`. No shell parsing. All 1498 tests pass.
|
||||
- [x] **P0-3: Unvalidated req.body in backup config** — Done (commit b3488f1, grade A). POST /backups/config now destructures only `{backups, defaultRetention}` instead of passing `req.body` wholesale. All 1498 tests pass.
|
||||
- [x] **P0-4: Asset upload buffer size check** — Done (commit 57ed09f, grade A). POST /assets/upload now uses `decodeImageData(data)` helper which enforces MIME whitelist (png/jpeg/jpg/svg+xml/webp/ico/x-icon) and 5 MB cap. (Prior partial fix had the helper but never wired it.) All 1498 tests pass.
|
||||
- [x] **P0-5: Error message leaking internals** — Done (commit 609ccd3, grade A). apps-revert catch now logs `err.message`+stack via `log.error` server-side and returns generic `Revert failed` to client. All 1498 tests pass.
|
||||
---
|
||||
|
||||
## P1 — Architecture & Input Validation
|
||||
## P0 — Must Fix (blocks public release)
|
||||
|
||||
- [x] **P1-1: Add Joi validation library** — Done in commit a667de7 (DC-059, codex-graded B). `npm install joi@^18`, `src/utilities/validate.js` exporting `validateBody(schema, opts)` middleware + 9 schemas (backupConfigUpdate, backupScheduleCreate, backupRestore, backupRestoreFile, appDeploy, appRestore, appRevert, assetUpload, logoUpload). Every exported schema has direct unit tests (41 total in `__tests__/unit/validate.test.js`) covering middleware semantics — not just `schema.validate`. Applied to 8 destructive routes: backups (schedule/restore/config), apps (deploy/restore/revert), assets (upload/logo). Used Joi's authoritative CIDR validator (rejects malformed IPv6 like `::::/64` that the previous hex/colon regex would have accepted). 1539/1539 Jest tests pass (was 1498, +41 new). ESLint warnings unchanged (416 total, all pre-existing — zero new introduced).
|
||||
- [ ] **P1-2: Console→logger sweep (update-manager.js)** — Replace all 49 `console.*` calls in `src/managers/update-manager.js` with structured logger calls. `const log = require('../utils/logging')` then `log.info/warn/error(tag, msg, meta)`.
|
||||
- [ ] **P1-3: Console→logger sweep (backup-manager.js)** — Replace all 36 `console.*` calls in `src/utilities/backup-manager.js` with structured logger.
|
||||
- [ ] **P1-4: Console→logger sweep (resource-monitor.js)** — Replace all 32 `console.*` calls in `src/managers/resource-monitor.js` with structured logger.
|
||||
- [ ] **P1-5: Console→logger sweep (credential-manager.js)** — Replace all 20 `console.*` calls in `src/managers/credential-manager.js` with structured logger.
|
||||
- [ ] **P1-6: Console→logger sweep (auth-manager.js)** — Replace all 20 `console.*` calls in `src/managers/auth-manager.js` with structured logger.
|
||||
- [ ] **P1-7: Console→logger sweep (bundled-workflows.js)** — Replace all 18 `console.*` calls in `src/recipes/bundled-workflows.js` with structured logger.
|
||||
- [ ] **P1-8: Console→logger sweep (remaining files)** — Sweep remaining files with < 20 console calls each: `crypto-utils.js` (16), `docker-security.js` (15), `port-lock-manager.js` (16), `self-updater.js` (10), `event-workers.js` (5), `keychain-manager.js` (4), `log-digest.js` (3), `csrf-protection.js` (3). One commit for all small files.
|
||||
### DC-062: OpenAPI spec is stale — update to match actual v1.15.0 API surface
|
||||
- **status:** pending
|
||||
- **details:** `openapi.yaml` says `version: 1.0.0` and describes only a fraction of the API. Since DC-046/047 (auth providers), DC-053 (share), DC-055 (billing), DC-058 (share UI), and the tailscale-admin routes were added, the spec is significantly out of date. A stale spec is worse than no spec — it misleads API consumers and breaks any code generation from it. Fix: audit all route files (`grep -rn 'router\.\(get\|post\|put\|delete\|patch\)' routes/`), update openapi.yaml with every endpoint, bump version to 1.15.0, add it to the test suite (DC-017-style source-of-truth test that fails if a route exists but has no spec entry). Effort: ~3 hr.
|
||||
- **impact:** Public API trust. No paying customer can integrate against an undocumented API.
|
||||
|
||||
## P2 — Code Quality & Technical Debt
|
||||
### DC-063: Branch coverage at 72% — below the 80% gate
|
||||
- **status:** pending
|
||||
- **details:** Jest coverage report shows branches at 72.14% (303/420), failing the 80% threshold. The uncovered branches are concentrated in error-handling paths (catch blocks, fallback returns, edge-case conditionals). Fix: run `npx jest --coverage --coverageReporters=text` to identify the files with the lowest branch coverage, then add targeted tests for the uncovered conditional paths. Priority files: backup-manager.js (multiple catch blocks), health-checker.js (timeout/retry branches), tailscale-coord.js (API error branches). Effort: ~2 hr.
|
||||
- **impact:** Error paths are where production incidents hide. Every untested catch block is a potential crash.
|
||||
|
||||
- [ ] **P2-1: Version drift fix** — Update `VERSION` file from `1.14.9` to `1.15.0`. Update `CLAUDE.md` line 247 from `1.13.4` to `1.15.0`.
|
||||
- [ ] **P2-2: Delete dead legacy files** — `git rm dashcaddy-api/scripts/legacy/comprehensive-test.js dashcaddy-api/scripts/legacy/test-security-fixes.js status/api/test-api.js`. Verify zero references first.
|
||||
- [ ] **P2-3: ESLint no-empty fix** — Add `{ allow: 'catch' }` to the `no-empty` rule in `.eslintrc.js`, OR add `// intentionally ignored` comments. Goal: `npx eslint src/ routes/` exits 0 errors.
|
||||
- [ ] **P2-4: Fix no-useless-escape** — `routes/auth/session-handlers.js:39` — `\-` inside character class → `-` (at end of class to avoid range).
|
||||
- [ ] **P2-5: Test handle leaks** — Run `npx jest --detectOpenHandles --silent 2>&1 | grep -i leak` and add teardown (`afterEach(() => clearInterval/clearTimeout)`) to tests that leave open handles. Focus on `totp.routes.test.js` (22s) and `containers.routes.test.js` (28s).
|
||||
- [ ] **P2-6: Refactor config-schema.js validateConfig** — Complexity 44 → extract sub-validators for each config section. Behavior-preserving refactor only.
|
||||
- [ ] **P2-7: Refactor middleware.js auth function** — Complexity 24, nesting depth 6 → extract auth-logic branches into named helper functions.
|
||||
### DC-064: Dockerfile runs as root with no resource limits
|
||||
- **status:** pending
|
||||
- **details:** The Dockerfile has no `USER` directive and `start.sh` has no `--memory` or `--cpus` flags. While root is needed for Docker socket access, the container can still OOM the host. Fix: (1) Add `--memory=512m --memory-swap=1g --cpus=1.5` to the `docker run` in start.sh. (2) Create a non-root user `dashcaddy` for the application process, and use a Docker socket proxy (like `tecnativa/docker-socket-proxy`) that exposes a limited subset of Docker API endpoints — the app only needs read access for monitoring + controlled container lifecycle. (3) Add `--restart=unless-stopped` if not already present. Effort: ~2 hr. Risk: medium — socket proxy may break some Docker API calls, needs testing.
|
||||
- **impact:** Without limits, a memory leak in the API can take down the entire host. This is a production safety issue.
|
||||
|
||||
## Completion Criteria
|
||||
---
|
||||
|
||||
When all items above are `[x]`, report "All backlog items complete" and stop.
|
||||
## P1 — Code Quality & Reliability
|
||||
|
||||
### DC-065: Remaining 21 console.* calls — sweep to structured logger
|
||||
- **status:** pending
|
||||
- **details:** After DC-060 (update-manager) and P1-3 through P1-8, 21 console calls remain across 10 files: `error-handler.js` (2), `email.js` (1), `dns-providers/registry.js` (2), `audit-logger.js` (3), `csrf-protection.js` (3), `config-drift-detector.js` (1), `auto-restart-manager.js` (1), `http.js` (1), `logging.js` (6 intentional — the logger itself), `routes/backups.js` (1). The logging.js calls are fine (the logger IS console internally). The rest should route through `log.info/warn/error`. Some are fallbacks: `ctx.logError || ((_c, err) => console.error(err))` — these fire when ctx isn't available, which is exactly when structured logging matters most. Effort: ~45 min.
|
||||
- **impact:** Consistency. The logger write to error.log and supports structured JSON — console does not.
|
||||
|
||||
### DC-066: No API integration test for the billing flow end-to-end
|
||||
- **status:** pending
|
||||
- **details:** DC-057 shipped contract tests and unit tests for the Stripe bridge, but there is no test that exercises the full flow: pricing page → Stripe Checkout → webhook → license-key delivery → license activation → Pro unlock. Build a single integration test that mocks Stripe's API, walks the complete flow, and asserts the license works at the end. This is the revenue path — it must be tested as a chain, not just individual pieces. Effort: ~2 hr.
|
||||
- **impact:** Confidence in the revenue pipeline. A broken webhook or catalog mismatch silently loses sales.
|
||||
|
||||
### DC-067: No graceful shutdown — SIGTERM kills in-flight requests
|
||||
- **status:** pending
|
||||
- **details:** server.js handles `uncaughtException` and `unhandledRejection`, but there is no `SIGTERM` handler that calls `server.close()` to drain connections. Docker stop sends SIGTERM (the Dockerfile has `STOPSIGNAL SIGTERM`), but without a handler the process exits immediately, dropping any in-flight API calls. Fix: add a `SIGTERM` handler in server.js that (1) stops accepting new connections via `server.close()`, (2) waits up to 10s for in-flight requests, (3) closes DB/file handles, (4) exits cleanly. Also emit a `shutdown` event so managers (health checker, SSL monitor, workflow engine) can stop their timers. Effort: ~1 hr.
|
||||
- **impact:** Zero-downtime deployments. Currently, every `docker stop` drops active requests.
|
||||
|
||||
### DC-068: ESLint warnings sweep — 173 pre-existing warnings
|
||||
- **status:** pending
|
||||
- **details:** While there are 0 ESLint errors, 173 warnings remain. Top files: `dns-providers/base.js` (27), `update-manager.js` (14), `backup-manager.js` (10), `keychain-manager.js` (10), `bundled-workflows.js` (10), `auth/providers/base.js` (9), `log-digest.js` (8). Most are `no-unused-vars`, `require-await`, `no-nested-ternary`. Fix: sweep through the top 10 files, fix what's actionable (unused vars → remove, nested ternaries → extract to named variables, false-positive require-await → mark `_` or restructure). Set a ceiling: warnings should never increase. Effort: ~2 hr.
|
||||
- **impact:** Clean codebase. 173 warnings is noise that hides real issues when new ones are added.
|
||||
|
||||
### DC-069: Health check notification spam — add failure threshold + cooldown
|
||||
- **status:** pending
|
||||
- **details:** The workflow engine sends a notification on EVERY health check failure (every 15 min). If a service is down for a day, that's 96 identical notifications. There is no backoff, no deduplication, no "service recovered" message. Fix: (1) Only notify on state TRANSITIONS (up→down, down→up), not every failure. (2) Add a `consecutiveFailures` threshold (e.g., 2 failures before first alert) to avoid flapping noise. (3) Send a recovery notification when a service comes back up. (4) Optional: daily digest of uptime stats instead of per-failure alerts. Effort: ~1.5 hr.
|
||||
- **impact:** Operator sanity. The current notification volume is exactly why people mute alerting channels — and then miss real incidents.
|
||||
|
||||
---
|
||||
|
||||
## P2 — Polish & Developer Experience
|
||||
|
||||
### DC-070: No CI/CD pipeline — tests run manually
|
||||
- **status:** pending
|
||||
- **details:** There is no GitHub Actions / CI configuration. Tests are run manually before push. This means a bad commit can reach main if someone forgets to test. Fix: add `.github/workflows/test.yml` (or Gitea Actions equivalent) that runs `npm ci && npx jest --coverage` on every PR and push to main. Cache node_modules. Upload coverage report as artifact. Block merge on test failure or coverage decrease. Effort: ~1 hr.
|
||||
- **impact:** Automated quality gate. No bad commit reaches production.
|
||||
|
||||
### DC-071: No error tracking / Sentry integration
|
||||
- **status:** pending
|
||||
- **details:** Errors go to `error.log` inside the container. If the container is recreated (DC-050 migration), the error log is lost. There is no external error tracking. Fix: add an optional Sentry (or GlitchTip for self-hosted) integration. If `SENTRY_DSN` env var is set, initialize Sentry before Express. Wrap async handlers to capture exceptions. The error-handler.js middleware should forward to Sentry before returning the generic error response. Make it opt-in (no DSN = no Sentry, zero behavior change). Effort: ~1 hr.
|
||||
- **impact:** Production visibility. Right now, errors are invisible unless someone SSHs in and reads the log.
|
||||
|
||||
### DC-072: Frontend bundle has no source maps in production
|
||||
- **status:** pending
|
||||
- **details:** `status/build.js` uses esbuild but the production build doesn't emit source maps. When a frontend error occurs in production, the stack trace points to minified bundle lines — useless for debugging. Fix: add `sourcemap: true` to the esbuild production config. Serve `.map` files from Caddy (they're already in `dist/`). Optionally upload source maps to Sentry (DC-071). Effort: ~30 min.
|
||||
- **impact:** Frontend bug reports become actionable instead of "line 1 of core.js".
|
||||
|
||||
### DC-073: No API request/response logging middleware for debugging
|
||||
- **status:** pending
|
||||
- **details:** While there is an audit logger for POST/PUT/DELETE, there's no request/response logging middleware for debugging purposes (like morgan or a custom equivalent). When an operator reports "the dashboard is slow" or "this endpoint returns 500 sometimes", there's no way to trace the request through the system. Fix: add an optional debug-level request logger that logs method, path, status, duration, and request ID. Gated behind `LOG_LEVEL=debug` so it's off in production by default. Effort: ~45 min.
|
||||
- **impact:** Drastically reduces time-to-resolution for production issues.
|
||||
|
||||
### DC-074: Docker image is not multi-stage — build artifacts bloat the image
|
||||
- **status:** pending
|
||||
- **details:** The Dockerfile copies source files into a single stage based on `node:20-alpine`. The image includes `devDependencies` because `npm install --production` still installs some optional deps, and there's no `.dockerignore` (so `__tests__/`, `.git/`, `node_modules/` from the host can leak in). Fix: (1) Add a `.dockerignore` file excluding `__tests__/`, `.git/`, `node_modules/`, `*.md`, `coverage/`. (2) Convert to multi-stage: build stage installs all deps, production stage copies only `node_modules/` (production) + source. (3) Pin Node.js version: `FROM node:20.10-alpine` instead of `node:20-alpine` (floating). Effort: ~1 hr.
|
||||
- **impact:** Smaller image = faster pulls = faster deploys. Current image size carries unnecessary weight.
|
||||
|
||||
### DC-075: No health check dashboard endpoint for operators
|
||||
- **status:** pending
|
||||
- **details:** The `/api/v1/monitoring/stats` endpoint returns container stats, but there's no single "is everything OK" endpoint that returns a human-readable system health summary. Fix: add `GET /api/v1/system/health` that returns `{ status: "healthy"|"degraded"|"unhealthy", checks: { database: "ok", diskSpace: "ok", memory: "ok", uptime: ..., activeServices: N/M, lastError: "..." } }`. This is useful for uptime monitoring services (UptimeRobot, BetterStack) and for a quick operator glance. Effort: ~1 hr.
|
||||
- **impact:** Operators can plug DashCaddy into external monitoring without parsing container stats.
|
||||
|
||||
---
|
||||
|
||||
## P3 — Future & Nice-to-Have
|
||||
|
||||
### DC-076: WebSocket support for real-time dashboard updates
|
||||
- **status:** pending
|
||||
- **details:** The dashboard polls the API every N seconds for service status updates. For a "live" dashboard experience, WebSocket (or SSE) push would be better — status changes appear instantly without polling overhead. Fix: add a WebSocket server (using `ws` library) that pushes service status changes, health check results, and container events to connected dashboard clients. Keep polling as fallback for clients without WS support. Effort: ~3 hr.
|
||||
- **impact:** Dashboard feels "live". Reduces API load from polling.
|
||||
|
||||
### DC-077: Multi-language (i18n) support
|
||||
- **status:** pending
|
||||
- **details:** All UI text is hardcoded English. For a public product, internationalization is a step toward wider reach. Fix: extract all user-facing strings into a locale file, add an i18n library (like i18next), provide at minimum an English + Arabic locale (Sami's audience). Effort: ~4 hr.
|
||||
- **impact:** Market expansion. Arabic-speaking homelab community is underserved.
|
||||
|
||||
### DC-078: Backup and restore of DashCaddy's own configuration
|
||||
- **status:** pending
|
||||
- **details:** While DashCaddy can backup app data, there's no one-click "backup my entire DashCaddy setup" (services.json, config.json, health-config.json, credentials, Caddyfile, license) that could be restored on a fresh install. Fix: add `GET /api/v1/system/export` (returns a signed JSON bundle) and `POST /api/v1/system/import` (restores from bundle). The credentials file should be encrypted with a user-provided passphrase. Effort: ~2 hr.
|
||||
- **impact:** Migration story. "Moving DashCaddy to a new host" is currently a multi-hour manual process.
|
||||
|
||||
### DC-079: Mobile-responsive dashboard improvements
|
||||
- **status:** pending
|
||||
- **details:** While the dashboard is somewhat responsive, it's not optimized for mobile use. For operators checking services on their phone, the experience should be touch-first. Fix: audit all dashboard pages on mobile viewport, fix any horizontal scroll, ensure buttons are touch-target sized (min 44px), add a mobile-specific layout for the service grid. Effort: ~3 hr.
|
||||
- **impact:** Operators check services on their phone. Current mobile experience is usable but not polished.
|
||||
|
||||
### DC-080: Plugin/extension system for custom services
|
||||
- **status:** pending
|
||||
- **details:** DashCaddy supports a fixed set of service templates. A plugin system would allow community-contributed service definitions (e.g., "Home Assistant", "Vaultwarden", "Nextcloud") without modifying core code. Fix: define a plugin manifest schema (name, logo, health check URL pattern, config fields), load plugins from `/data/plugins/`, add a community plugin registry page. Effort: ~4 hr.
|
||||
- **impact:** Community growth. Extensibility is what makes a tool ecosystem vs. a product.
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## P2.5 — Security Hardening (Deep Audit Findings)
|
||||
|
||||
### DC-081: 151 of 160 mutating routes have NO Joi input validation
|
||||
- **status:** pending
|
||||
- **details:** P1-1 added Joi validation to 8 routes, but a scan shows **151 out of 160** POST/PUT/PATCH/DELETE routes still accept raw `req.body` without schema validation. That's 94% of the mutation surface unvalidated. Routes like `POST /api/v1/services/:id`, `PUT /api/v1/config`, `POST /api/v1/tailscale/*`, `POST /api/v1/health/config/:id` all accept arbitrary input. Fix: extend `src/utilities/validate.js` with schemas for every mutating route, wire them in. This is the single highest-impact security improvement. Effort: ~4 hr (batch by route file).
|
||||
- **impact:** Input validation is the #1 defense against injection, abuse, and crashes. 94% gap is a P0 hiding as a P2.
|
||||
|
||||
### DC-082: Command injection surface in ca.js — 5 execSync calls with interpolation
|
||||
- **status:** pending
|
||||
- **details:** `routes/ca.js` has 5 `execSync()` calls with template-string interpolation: lines 164, 175, 180, 203, 263. P0-2 fixed the password injection (`execFileSync`), but the remaining calls interpolate file paths and subjects (`${certFile}`, `${keyFile}`, `${subject}`, `${configFile}`). If any of these contain user input (e.g., a service name with `;` or backticks), it's command injection. Fix: convert ALL `execSync(\`...\`)` calls to `execFileSync('openssl', [...args])` with no shell interpolation. Also fix `src/docker/self-updater.js:717` (`execSync(\`tar xzf \"${tarballPath}\"...`)`) and `src/utilities/backup-manager.js:8` (imported execSync). Effort: ~2 hr.
|
||||
- **impact:** Any execSync with interpolation is a potential RCE. This is the same class of bug P0-2 already fixed — finish the job.
|
||||
|
||||
### DC-083: 30 source files have zero test coverage
|
||||
- **status:** pending
|
||||
- **details:** The test gap scan found 30 source files with NO corresponding test file, including critical paths: `license-manager.js` (534 lines, the entire revenue validation path), `config-schema.js`, `middleware.js` (the auth/rate-limit/CORS stack), `startup-validator.js`, all 7 DNS provider modules (`technitium.js`, `cloudflare.js`, `rfc2136.js`, `manual.js`, `base.js`, `registry.js`, `email.js`), `docker-maintenance.js`, `config/migrations.js`, `event-workers.js`, `keychain-manager.js`, `event-store.js`, `host-registry.js`. Fix: prioritize license-manager.js (revenue path) and middleware.js (security stack) first, then work through the rest. Effort: ~8 hr (can be done incrementally, 2-3 files per PR).
|
||||
- **impact:** license-manager.js validates Pro licenses — an untested bug there could silently break activation for every paying customer.
|
||||
|
||||
### DC-084: No .dockerignore — test files and .git leak into Docker image
|
||||
- **status:** pending
|
||||
- **details:** There is no `.dockerignore` file. The Docker build context includes `__tests__/` (hundreds of test files), any `.git/` directory, `coverage/`, `node_modules/` from the host, and markdown files. This bloats the image (currently 249MB) and can leak sensitive test fixtures. Fix: create `.dockerignore` with: `__tests__/`, `.git/`, `node_modules/`, `coverage/`, `*.md`, `.eslintrc.js`, `jest.config.js`, `npm-debug.log*`, `.env*`, `openapi.yaml` (only needed at build time if at all). Also add `.dockerignore` to the git repo. Effort: ~15 min.
|
||||
- **impact:** Faster builds, smaller images, no test fixture leaks.
|
||||
|
||||
### DC-085: Math.random() used for security-sensitive IDs
|
||||
- **status:** pending
|
||||
- **details:** `health-checker.js:352` generates incident IDs with `Math.random().toString(36)`. `resource-monitor.js:143` uses `Math.random()` for sampling. `rfc2136.js:120` generates temp filenames with `Math.random()`. While these aren't crypto-level secrets, `Math.random()` is not collision-resistant and is predictable. Fix: use `crypto.randomUUID()` for incident IDs, `crypto.randomBytes()` for temp filenames, and a simple counter for sampling. Effort: ~30 min.
|
||||
- **impact:** Defense in depth. Predictable IDs can be exploited if they ever become user-facing.
|
||||
|
||||
---
|
||||
|
||||
## P3.5 — Operational Maturity
|
||||
|
||||
### DC-086: No structured error codes — errors are ad-hoc strings
|
||||
- **status:** pending
|
||||
- **details:** The HTTP status code audit shows only 6 distinct status codes used across routes (200, 201, 400, 401, 404, 429). Error responses are plain strings like `"Invalid input"` or `"Unauthorized"`. There is no error code system (like `INVALID_CONFIG`, `SERVICE_NOT_FOUND`, `LICENSE_EXPIRED`). Fix: define a canonical error code enum in `src/utilities/errors.js`, return `{ error: { code: "SERVICE_NOT_FOUND", message: "..." } }` in all error responses. This makes API integration programmable (consumers switch on `code`, not parse `message`). Effort: ~3 hr.
|
||||
- **impact:** API consumers can handle errors programmatically. Required for SDK generation and good DX.
|
||||
|
||||
### DC-087: No API client SDK / type definitions
|
||||
- **status:** pending
|
||||
- **details:** There is no TypeScript definitions file (`.d.ts`) or client SDK. Anyone integrating against the API has to read the source code to understand request/response shapes. Fix: (1) Generate TypeScript types from the OpenAPI spec (once DC-062 updates it) using `openapi-typescript`. (2) Ship a `@dashcaddy/api-types` npm package or include a `types/index.d.ts` in the repo. (3) Optionally, a thin JS client wrapper. Effort: ~2 hr (after DC-062).
|
||||
- **impact:** Developer adoption. A typed SDK lowers the barrier to integration.
|
||||
|
||||
### DC-088: No log rotation — error.log grows forever
|
||||
- **status:** pending
|
||||
- **details:** The logger has basic rotation (rename to `.1` when it hits a size limit), but only keeps ONE rotated file. In production, error.log can grow rapidly during incident bursts. There's no retention policy, no compression, no date-based rotation. Fix: (1) Add a max-size threshold (e.g., 10MB) and keep N rotated files (e.g., 5). (2) Compress rotated files with gzip. (3) Add date-based naming so logs are greppable by date. (4) Add a `GET /api/v1/system/logs` endpoint so operators can view recent logs without SSH. Effort: ~1.5 hr.
|
||||
- **impact:** Prevents disk fill during incident storms. Makes logs accessible without SSH access.
|
||||
|
||||
### DC-089: No rate limit on public license activation endpoint
|
||||
- **status:** pending
|
||||
- **details:** The rate limiter `skip` list includes `req.path === '/api/v1/license/status'` and `req.path.startsWith('/api/v1/license/feature/')` — meaning license checks bypass rate limiting. While these are GET endpoints, the license *activation* endpoint (`POST /api/v1/license/activate`) should have its own dedicated rate limit to prevent brute-force license key guessing. Fix: add a dedicated `licenseLimiter` with tighter limits (e.g., 10 attempts per 15 min per IP) on POST /license/activate. Effort: ~30 min.
|
||||
- **impact:** Prevents license key brute-forcing. Pro keys follow a predictable format (DC-XXX-XXXXX-XXXXXX) making them guessable without rate limiting.
|
||||
|
||||
### DC-090: Node.js version drift — Dockerfile says 20, host runs 22
|
||||
- **status:** pending
|
||||
- **details:** Dockerfile uses `FROM node:20-alpine` (floating). The development machine runs Node v22.22.3. The container uses whatever `node:20-alpine` resolves to at build time. This version drift can cause "works on my machine" bugs (especially around `fetch()`, `crypto`, and `structuredClone` which changed between 20 and 22). Fix: (1) Pin the exact version: `FROM node:20.10.0-alpine3.19`. (2) Add `.nvmrc` or `engines` field to package.json specifying the minimum version. (3) Optionally upgrade to Node 22 across the board. Effort: ~30 min.
|
||||
- **impact:** Reproducible builds. No surprise behavior from Node version drift.
|
||||
|
||||
### DC-091: No dependency update automation (Dependabot/Renovate)
|
||||
- **status:** pending
|
||||
- **details:** Dependencies are updated manually. The 21 production dependencies and 4 dev dependencies can fall behind silently. There's no automated PR for security patches or major version bumps. Fix: add either GitHub Dependabot config (`.github/dependabot.yml`) or Renovate config (`renovate.json`). Schedule weekly checks. Group minor/patch updates into one PR. Keep major updates separate for review. Effort: ~30 min.
|
||||
- **impact:** Security patches arrive automatically. No more manual `npm audit` sessions.
|
||||
|
||||
### DC-092: No health check for DashCaddy's own dependencies (disk space, memory)
|
||||
- **status:** pending
|
||||
- **details:** The Dockerfile has a HEALTHCHECK that hits `/health`, but that endpoint only checks if the Express server responds. It doesn't check: disk space (if `/app/data` is on a full disk), memory pressure (Node heap near limit), Docker socket connectivity (if Docker daemon is down), Caddy admin API reachability. Fix: extend the health endpoint to include dependency checks: `{ diskSpace: { free: ..., total: ... }, memory: { heapUsed: ..., heapTotal: ..., rss: ... }, docker: { reachable: true/false }, caddy: { reachable: true/false } }`. Return 503 if any critical dependency is down. Effort: ~1.5 hr.
|
||||
- **impact:** Catch systemic issues before they become outages. External monitoring can alert on `503`.
|
||||
|
||||
### DC-093: Workflow engine has no retry/backoff for failed actions
|
||||
- **status:** pending
|
||||
- **details:** When the workflow engine's health-check action fails, it logs the failure and moves on — no retry. If a service is temporarily down and recovers in 30s, the workflow reports it as failed for the entire 15-min cycle. Fix: add configurable retry logic to workflow actions (e.g., retry 2 times with 30s backoff before reporting failure). Also add a `maxRetries` config to the health-check workflow. Effort: ~1.5 hr.
|
||||
- **impact:** Fewer false-positive alerts. More resilient monitoring.
|
||||
|
||||
### DC-094: No audit trail for config changes (who changed what, when)
|
||||
- **status:** pending
|
||||
- **details:** The audit logger (`src/security/audit-logger.js`) captures POST/PUT/DELETE events, but config changes (services.json, health-config.json, config.json) are made via file writes, not API calls. There's no record of who changed a service URL, disabled a health check, or modified a workflow. Fix: (1) Route all config mutations through API endpoints that log to the audit trail. (2) Add a `GET /api/v1/system/audit-log` endpoint for viewing the trail. (3) Include a diff of what changed in each audit entry. Effort: ~2 hr.
|
||||
- **impact:** Accountability. When something breaks, you can trace who changed the config and when.
|
||||
|
||||
---
|
||||
|
||||
## P4 — Advanced Features
|
||||
|
||||
### DC-095: No multi-user support — single-admin only
|
||||
- **status:** pending
|
||||
- **details:** DashCaddy has one admin user. For teams or homelab groups, there's no way to add a second admin or a read-only viewer. Fix: (1) Add a `users.json` with role-based access (admin, editor, viewer). (2) Add user management endpoints. (3) Add per-service permissions (editor can manage services but not billing). This is a significant feature, not a quick fix. Effort: ~6 hr.
|
||||
- **impact:** Multi-admin is a requirement for team/enterprise adoption.
|
||||
|
||||
### DC-096: No API key management (create/revoke/scoped keys)
|
||||
- **status:** pending
|
||||
- **details:** API authentication uses session cookies or TOTP. There's no way to create scoped API keys for automation (e.g., a read-only key for monitoring, a key that can only manage one service). Fix: add `POST /api/v1/api-keys` (create with scopes), `GET /api/v1/api-keys` (list), `DELETE /api/v1/api-keys/:id` (revoke). Store hashed in credentials.json. Effort: ~2 hr.
|
||||
- **impact:** Enables automation and third-party integrations without sharing the admin password.
|
||||
|
||||
### DC-097: No Prometheus / Grafana metrics export
|
||||
- **status:** pending
|
||||
- **details:** There's a basic `/metrics` endpoint, but it returns JSON, not Prometheus format. Fix: (1) Add `prom-client` dependency. (2) Instrument key metrics: HTTP request duration histogram, active WebSocket connections, health check pass/fail counter, container count gauge, API error rate. (3) Expose `GET /metrics` in Prometheus exposition format alongside the existing JSON endpoint. (4) Ship a Grafana dashboard JSON as a reference. Effort: ~2 hr.
|
||||
- **impact:** Industry-standard observability. Drop-in Grafana dashboard for operators.
|
||||
|
||||
### DC-098: No changelog / release notes generation
|
||||
- **status:** pending
|
||||
- **details:** Releases are tracked via git commits and VERSION file, but there's no user-facing changelog. For a public product, customers need to know what changed between versions. Fix: (1) Add a `CHANGELOG.md` following Keep a Changelog format. (2) Auto-generate from conventional commits (if adopted) or git log. (3) Display "What's new" on the dashboard after updates. Effort: ~1.5 hr.
|
||||
- **impact:** Customer trust. Users won't update without knowing what changed.
|
||||
|
||||
### DC-099: No automated database migration system
|
||||
- **status:** pending
|
||||
- **details:** Config migrations exist (`src/config/migrations.js`) but are ad-hoc. As the data schema evolves (new fields in services.json, config.json), there's no versioned migration system. Fix: (1) Add a `schemaVersion` field to config files. (2) Create a migration runner that applies migrations sequentially on startup. (3) Log each migration. (4) Support rollback on failure. Effort: ~2 hr.
|
||||
- **impact:** Safe upgrades. No more manual config patching after updates.
|
||||
|
||||
### DC-100: No service discovery / auto-detect running containers
|
||||
- **status:** pending
|
||||
- **details:** Services are added manually by specifying URLs. DashCaddy doesn't auto-detect running Docker containers and suggest adding them as services. Fix: (1) Scan `docker ps` for containers with exposed ports. (2) Match against known app templates (Plex, Sonarr, etc.). (3) Show a "Detected services" panel with one-click add. (4) Periodically re-scan for new containers. Effort: ~3 hr.
|
||||
- **impact:** Zero-config onboarding. New users see their services auto-discovered.
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## P5 — Product Vision: Self-Hosting Platform
|
||||
|
||||
> These tasks directly serve the vision from PRODUCT-VISION.md:
|
||||
> "Self-host anything in 30 seconds — no config files, no TLS headaches."
|
||||
|
||||
### DC-101: Disk Space Manager with user-configurable budget + dashboard widget
|
||||
- **status:** in-progress (backend done, needs UI + deployment)
|
||||
- **details:** Backend module (`src/monitoring/disk-space-monitor.js`) and routes (`routes/disk-space.js`) are written and pass tests. Still needs: (1) Dashboard widget showing disk usage gauge with budget line, breakdown by category (images/volumes/logs/build-cache), and "Cleanup now" button. (2) Settings page section for disk budget input. (3) Deploy to DNS2 production. API endpoints: GET /api/v1/disk, GET /api/v1/disk/breakdown, POST /api/v1/disk/config, POST /api/v1/disk/cleanup. Effort: ~2 hr remaining.
|
||||
- **impact:** Users set a disk budget (e.g., "DashCaddy gets 20GB") and the system auto-manages cleanup. The #1 reason people abandon self-hosting is disk filling up silently. This solves it.
|
||||
|
||||
### DC-102: One-click deploy should auto-generate Caddyfile entry + DNS record
|
||||
- **status:** pending
|
||||
- **details:** When a user deploys an app from the catalog, DashCaddy should automatically: (1) Create the Docker container, (2) Add a Caddyfile reverse_proxy block with TLS for `appname.tld`, (3) Create a DNS record pointing to the host, (4) Reload Caddy, (5) Add the service to the dashboard with health check. Currently steps 2-4 are manual. Fix: add a `deployApp(serviceId, options)` function that orchestrates the full chain. The Caddyfile generation can use the admin API (POST to :2019) so no file editing needed. DNS record creation uses the existing Technitium/Cloudflare DNS provider integration. Effort: ~4 hr.
|
||||
- **impact:** This is THE core value proposition. Without this, DashCaddy is just Portainer with extra steps. With this, it's a self-hosting platform.
|
||||
|
||||
### DC-103: Container auto-discovery with auto-route generation
|
||||
- **status:** pending
|
||||
- **details:** When DashCaddy detects a new running Docker container (via docker events API), it should: (1) Check if it matches a known app template (Plex, Sonarr, etc.), (2) Auto-generate a Caddy reverse proxy route, (3) Create a DNS record, (4) Add it to the dashboard, (5) Notify the user "Found Nextcloud on port 80 — added to your dashboard at https://nextcloud.yourdomain.com". This is the "zero-config" experience. Effort: ~4 hr.
|
||||
- **impact:** Magic. User installs Nextcloud via docker run → 10 seconds later it's on their dashboard with HTTPS.
|
||||
|
||||
### DC-104: App catalog with curated templates + one-click deploy
|
||||
- **status:** pending
|
||||
- **details:** The app templates exist (`src/docker/app-templates.js` has 50+ templates) but there's no polished catalog UI. Build a "App Store" page: grid of app cards with icons, descriptions, and "Install" buttons. Clicking install triggers DC-102's deploy chain. Include categories (Media, Productivity, Security, Development). Show "Popular" and "New" badges. Allow community templates via DC-080's plugin system. Effort: ~4 hr.
|
||||
- **impact:** This is the front door. The catalog IS the product for most users.
|
||||
|
||||
### DC-105: Smart defaults wizard — "What do you want to self-host?"
|
||||
- **status:** pending
|
||||
- **details:** Instead of asking users to configure DNS servers, TLD, Caddy paths, and auth — ask them ONE question: "What domain do you want to use?" Then auto-detect: (1) DNS server (check if Technitium is running locally), (2) TLD (.home, .local, or their domain), (3) Caddy installation, (4) Docker setup. Configure everything automatically. If something is missing, install it. The wizard should handle 90% of setups in under 5 questions. Effort: ~3 hr.
|
||||
- **impact:** First-run experience determines whether users stay. A 15-step config wizard kills adoption. A 1-question wizard creates delight.
|
||||
|
||||
### DC-106: Caddyfile-as-code — visual reverse proxy builder
|
||||
- **status:** pending
|
||||
- **details:** Instead of editing Caddyfile text, provide a visual builder: "I want requests to blog.yourdomain.com to go to container X on port 80, with authentication, rate limiting, and compression." Generate the Caddyfile block from the form. Show a live preview of the generated config. Apply via Caddy admin API. This eliminates the need to learn Caddyfile syntax entirely. Effort: ~3 hr.
|
||||
- **impact:** Caddyfile syntax is the #1 technical barrier. A visual builder makes reverse proxy configuration accessible to non-sysadmins.
|
||||
|
||||
### DC-107: Disaster recovery — one-click backup + restore of entire setup
|
||||
- **status:** pending
|
||||
- **details:** Extend DC-078 to include container definitions, Caddyfile, DNS zones, and all app data. The backup should be a single encrypted tarball. "Restore on new host" should bring back the entire DashCaddy setup + all apps in one command. This is the "set it and forget it" insurance policy. Effort: ~3 hr.
|
||||
- **impact:** Fear of losing setup is why people stick with SaaS. One-click backup + restore removes that fear.
|
||||
|
||||
### DC-108: Multi-host fleet management — deploy across multiple servers
|
||||
- **status:** pending
|
||||
- **details:** Currently DashCaddy manages one Docker host. For users with multiple servers (like Sami's DNS1/DNS2/DNS3 setup), DashCaddy should connect to remote Docker daemons (via TLS or SSH) and manage containers across all hosts from one dashboard. "Deploy Nextcloud on DNS2" or "Deploy Plex on SAMI-PC" from the same UI. Show per-host resource usage and health. Effort: ~6 hr.
|
||||
- **impact:** Power users have multiple servers. Managing them individually defeats the purpose of a unified platform.
|
||||
|
||||
---
|
||||
|
||||
## Summary by Priority
|
||||
|
||||
| Priority | Count | Effort | Theme |
|
||||
|----------|-------|--------|-------|
|
||||
| P0 | 3 (DC-062–064) | ~7 hr | Public release blockers |
|
||||
| P1 | 5 (DC-065–069) | ~7 hr | Reliability & code quality |
|
||||
| P2 | 6 (DC-070–075) | ~5.5 hr | Polish & DX |
|
||||
| P2.5 | 5 (DC-081–085) | ~15 hr | Security hardening (deep audit) |
|
||||
| P3 | 5 (DC-076–080) | ~16 hr | Future growth |
|
||||
| P3.5 | 9 (DC-086–094) | ~14.5 hr | Operational maturity |
|
||||
| P4 | 6 (DC-095–100) | ~16.5 hr | Advanced features |
|
||||
| P5 | 8 (DC-101–108) | ~29 hr | Product vision: self-hosting platform |
|
||||
| **Total** | **47** | **~110.5 hr** | |
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
# DashCaddy Product Vision
|
||||
|
||||
## The Problem
|
||||
|
||||
Self-hosting software is hard. To deploy a single app (Plex, Nextcloud, Vaultwarden, anything), you need to:
|
||||
|
||||
1. **Understand Docker** — images, containers, volumes, ports, networks, compose files
|
||||
2. **Configure a reverse proxy** — Caddy/Nginx/Traefik config files with obscure syntax
|
||||
3. **Set up TLS/HTTPS** — certificate generation, ACME, DNS challenges, trust stores
|
||||
4. **Configure DNS** — A records, CNAMEs, split-horizon DNS, DoH
|
||||
5. **Secure it** — firewall rules, auth, rate limiting, CSRF, CORS
|
||||
6. **Monitor it** — health checks, log rotation, disk space, restart policies
|
||||
7. **Maintain it** — updates, backups, migrations, disaster recovery
|
||||
|
||||
Each of these is a rabbit hole. A typical homelabber spends **hours per app** fighting configuration files, reading documentation, and debugging cryptic errors. This is why most people give up and just use SaaS.
|
||||
|
||||
## The Solution
|
||||
|
||||
**DashCaddy is a self-hosting platform.** It eliminates the complexity by fusing Docker, Caddy, and DNS management into one unified interface.
|
||||
|
||||
### Core Value: "Self-host anything in 30 seconds."
|
||||
|
||||
```
|
||||
User picks an app from the catalog
|
||||
↓
|
||||
DashCaddy deploys the Docker container
|
||||
↓
|
||||
DashCaddy generates the Caddy reverse proxy config automatically
|
||||
↓
|
||||
DashCaddy provisions TLS certificates
|
||||
↓
|
||||
DashCaddy configures DNS records
|
||||
↓
|
||||
DashCaddy sets up authentication (SSO gate)
|
||||
↓
|
||||
App is live at https://app.yourdomain.com — done.
|
||||
```
|
||||
|
||||
No editing config files. No Docker networking headaches. No TLS cert errors. No DNS archaeology.
|
||||
|
||||
## What Makes DashCaddy Different
|
||||
|
||||
### vs. Plain Docker / docker-compose
|
||||
- Docker gives you containers. DashCaddy gives you **containers + networking + TLS + DNS + auth + monitoring**.
|
||||
- Docker doesn't know about your domain. DashCaddy manages the full stack from DNS record to container port.
|
||||
- Docker doesn't tell you when your disk is full. DashCaddy monitors, alerts, and auto-cleans.
|
||||
|
||||
### vs. Portainer
|
||||
- Portainer is a **Docker UI**. DashCaddy is a **self-hosting platform**.
|
||||
- Portainer shows containers. DashCaddy shows services — with their URLs, health, certs, and auth.
|
||||
- Portainer doesn't manage Caddy, DNS, or TLS. DashCaddy fuses all three.
|
||||
- Portainer doesn't have a one-click app catalog with auto-configured reverse proxy + DNS + TLS.
|
||||
|
||||
### vs. CasaOS / Umbrel
|
||||
- These are **app stores**. DashCaddy is a **platform**.
|
||||
- They bundle their own Docker management. DashCaddy works with your existing Docker setup.
|
||||
- They don't manage Caddy or advanced DNS. DashCaddy handles the full network stack.
|
||||
- DashCaddy's SSO gate, credential injection, and security center are enterprise-grade features.
|
||||
|
||||
### vs. Yunohost / FreedomBox
|
||||
- These are **complete OS replacements**. DashCaddy is a **single Docker container**.
|
||||
- No OS install needed. Deploy DashCaddy on any Linux machine in 60 seconds.
|
||||
- DashCaddy works alongside your existing setup — it doesn't take over your machine.
|
||||
|
||||
## The Three Pillars
|
||||
|
||||
### 1. One-Click Deploy (The "Wow" moment)
|
||||
Pick an app → DashCaddy handles everything:
|
||||
- Docker container creation with optimal defaults
|
||||
- Caddy reverse proxy route with TLS
|
||||
- DNS record creation
|
||||
- SSO authentication gate
|
||||
- Health check configuration
|
||||
- Disk budget allocation
|
||||
|
||||
### 2. Zero-Config Networking (The "It just works" layer)
|
||||
- Automatic TLS via Caddy's ACME + Let's Encrypt
|
||||
- Automatic DNS via Technitium/Cloudflare integration
|
||||
- Automatic reverse proxy with sane defaults
|
||||
- Automatic SSO with credential injection
|
||||
- Automatic subdomain routing (subdomain or subdirectory mode)
|
||||
|
||||
### 3. Self-Healing Infrastructure (The "Set it and forget it" layer)
|
||||
- Health checks with retry/backoff and notification on state transitions
|
||||
- Auto-restart failed containers
|
||||
- Auto-cleanup when disk approaches budget
|
||||
- Config drift detection and correction
|
||||
- SSL certificate expiration monitoring
|
||||
- Container log rotation and size enforcement
|
||||
- Docker image cleanup — old images pruned automatically
|
||||
|
||||
## Who Is It For?
|
||||
|
||||
1. **Homelabbers** — tired of spending weekends on config files
|
||||
2. **Small businesses** — want self-hosted alternatives to SaaS without hiring a sysadmin
|
||||
3. **Privacy-conscious users** — want to own their data without the technical burden
|
||||
4. **Developers** — want a quick way to deploy side projects with TLS + auth
|
||||
|
||||
## Revenue Model
|
||||
|
||||
- **Free tier**: Up to 5 services, community support
|
||||
- **Pro license**: Unlimited services, email alerts, advanced health checks, priority updates
|
||||
- **Site license**: Multi-host, team accounts, API access
|
||||
|
||||
## North Star Metric
|
||||
|
||||
**Time-to-first-app-deploy** — how long from install to having a working self-hosted service with HTTPS. Target: under 60 seconds.
|
||||
@@ -1,10 +1,14 @@
|
||||
node_modules/
|
||||
__tests__/
|
||||
jest.config.js
|
||||
.env
|
||||
.encryption-key
|
||||
.git/
|
||||
.gitignore
|
||||
.dockerignore
|
||||
*.log
|
||||
node_modules/
|
||||
coverage/
|
||||
*.md
|
||||
docker-compose.yml
|
||||
.eslintrc.js
|
||||
jest.config.js
|
||||
npm-debug.log*
|
||||
.env*
|
||||
.env.example
|
||||
.DS_Store
|
||||
*.log
|
||||
dc.png
|
||||
|
||||
@@ -35,6 +35,7 @@ module.exports = {
|
||||
'complexity': ['warn', 20],
|
||||
|
||||
// Prevent common pitfalls
|
||||
'no-empty': ['error', { allowEmptyCatch: true }],
|
||||
'no-eval': 'error',
|
||||
'no-implied-eval': 'error',
|
||||
'no-new-func': 'error',
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
FROM node:20-alpine
|
||||
FROM node:20.11.1-alpine3.19
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
@@ -0,0 +1,490 @@
|
||||
/**
|
||||
* Tests for the graceful shutdown coordinator (DC-067).
|
||||
*
|
||||
* Covers:
|
||||
* - Constructor rejects bad inputs
|
||||
* - shutdown() emits 'shutdown' event with the signal name
|
||||
* - shutdown() stops each manager in declaration order
|
||||
* - shutdown() is idempotent — second call logs and returns
|
||||
* - shutdown() force-exits after drainTimeoutMs if server.close never fires
|
||||
* - shutdown() clears the force-exit timer when server.close fires first
|
||||
* - shutdown() catches manager.stop() throws so one bad manager doesn't
|
||||
* prevent the others from being stopped
|
||||
* - installSignalHandlers() registers for SIGTERM and SIGINT by default
|
||||
*
|
||||
* process.exit is mocked so tests don't actually kill the test runner.
|
||||
*/
|
||||
'use strict';
|
||||
|
||||
const EventEmitter = require('events');
|
||||
const {
|
||||
createShutdownCoordinator,
|
||||
installSignalHandlers,
|
||||
DEFAULT_DRAIN_TIMEOUT_MS,
|
||||
ShutdownCoordinator,
|
||||
} = require('../src/utilities/shutdown');
|
||||
|
||||
describe('ShutdownCoordinator (DC-067)', () => {
|
||||
let exitMock;
|
||||
let exitCalls;
|
||||
|
||||
beforeEach(() => {
|
||||
exitCalls = [];
|
||||
// Use jest.spyOn so the mock is restored in afterEach (via clearMocks +
|
||||
// restoreMocks: true in jest.config.js). Direct assignment to process.exit
|
||||
// doesn't suppress Jest's process.exit watchlist which fails the test.
|
||||
exitMock = jest.spyOn(process, 'exit').mockImplementation((code) => {
|
||||
exitCalls.push(code);
|
||||
// Returning undefined prevents the test runner from actually exiting.
|
||||
return undefined;
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
exitMock.mockRestore();
|
||||
jest.clearAllTimers();
|
||||
});
|
||||
|
||||
function makeFakeServer({ closeBehavior = 'sync' } = {}) {
|
||||
// 'sync' close calls back immediately.
|
||||
// 'never' close never calls back (used to test force-exit).
|
||||
if (closeBehavior === 'never') {
|
||||
return { close: jest.fn() };
|
||||
}
|
||||
return { close: jest.fn((cb) => { cb(); }) };
|
||||
}
|
||||
|
||||
function makeFakeLog() {
|
||||
return {
|
||||
info: jest.fn(),
|
||||
warn: jest.fn(),
|
||||
error: jest.fn(),
|
||||
};
|
||||
}
|
||||
|
||||
describe('constructor', () => {
|
||||
test('throws if server is missing', () => {
|
||||
expect(() => createShutdownCoordinator({ log: makeFakeLog(), managers: [] }))
|
||||
.toThrow('server is required');
|
||||
});
|
||||
|
||||
test('throws if log is missing or invalid', () => {
|
||||
expect(() => createShutdownCoordinator({ server: makeFakeServer(), managers: [] }))
|
||||
.toThrow('log must have info');
|
||||
expect(() => createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: { foo: 'bar' },
|
||||
managers: [],
|
||||
})).toThrow('log must have info');
|
||||
expect(() => createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: { info: () => {}, warn: () => {} }, // missing error
|
||||
managers: [],
|
||||
})).toThrow('log must have info');
|
||||
// A log with all three methods should NOT throw.
|
||||
expect(() => createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: { info: () => {}, warn: () => {}, error: () => {} },
|
||||
managers: [],
|
||||
})).not.toThrow();
|
||||
});
|
||||
|
||||
test('uses DEFAULT_DRAIN_TIMEOUT_MS when drainTimeoutMs is not finite positive', () => {
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
drainTimeoutMs: 0,
|
||||
managers: [],
|
||||
});
|
||||
expect(c.drainTimeoutMs).toBe(DEFAULT_DRAIN_TIMEOUT_MS);
|
||||
|
||||
const c2 = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
drainTimeoutMs: NaN,
|
||||
managers: [],
|
||||
});
|
||||
expect(c2.drainTimeoutMs).toBe(DEFAULT_DRAIN_TIMEOUT_MS);
|
||||
|
||||
const c3 = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
drainTimeoutMs: 5000,
|
||||
managers: [],
|
||||
});
|
||||
expect(c3.drainTimeoutMs).toBe(5000);
|
||||
});
|
||||
|
||||
test('defaults managers to [] when not an array', () => {
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
});
|
||||
expect(c.managers).toEqual([]);
|
||||
});
|
||||
|
||||
test('is an EventEmitter', () => {
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
managers: [],
|
||||
});
|
||||
expect(c).toBeInstanceOf(EventEmitter);
|
||||
expect(c).toBeInstanceOf(ShutdownCoordinator);
|
||||
});
|
||||
});
|
||||
|
||||
describe('shutdown()', () => {
|
||||
test('emits shutdown event with signal name', () => {
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
managers: [],
|
||||
});
|
||||
const handler = jest.fn();
|
||||
c.on('shutdown', handler);
|
||||
|
||||
c.shutdown('SIGTERM');
|
||||
|
||||
expect(handler).toHaveBeenCalledTimes(1);
|
||||
expect(handler).toHaveBeenCalledWith('SIGTERM');
|
||||
});
|
||||
|
||||
test('swallows exceptions thrown by shutdown event listeners', () => {
|
||||
const log = makeFakeLog();
|
||||
const server = makeFakeServer();
|
||||
const c = createShutdownCoordinator({
|
||||
server,
|
||||
log,
|
||||
managers: [],
|
||||
});
|
||||
c.on('shutdown', () => { throw new Error('listener boom'); });
|
||||
|
||||
// shutdown() must NOT propagate the exception — that would abort
|
||||
// the entire shutdown sequence before server.close is even called.
|
||||
expect(() => c.shutdown('SIGTERM')).not.toThrow();
|
||||
expect(log.error).toHaveBeenCalledWith(
|
||||
'shutdown',
|
||||
"event listener for 'shutdown' threw",
|
||||
expect.objectContaining({ error: 'listener boom' }),
|
||||
);
|
||||
// server.close should still have been called.
|
||||
expect(server.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
test('swallows exceptions thrown by closed event listeners', async () => {
|
||||
const log = makeFakeLog();
|
||||
const server = makeFakeServer();
|
||||
const c = createShutdownCoordinator({
|
||||
server,
|
||||
log,
|
||||
managers: [],
|
||||
});
|
||||
c.on('closed', () => { throw new Error('closed listener boom'); });
|
||||
|
||||
// process.exit is mocked; we just verify the throw doesn't bubble.
|
||||
c.shutdown('SIGTERM');
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
// The closed listener threw but the exit still got recorded.
|
||||
expect(exitCalls).toEqual([0]);
|
||||
});
|
||||
|
||||
test('calls server.close() once', () => {
|
||||
const server = makeFakeServer();
|
||||
const c = createShutdownCoordinator({
|
||||
server,
|
||||
log: makeFakeLog(),
|
||||
managers: [],
|
||||
});
|
||||
|
||||
c.shutdown('SIGTERM');
|
||||
|
||||
expect(server.close).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
test('stops each manager in declaration order AFTER server.close fires', async () => {
|
||||
const order = [];
|
||||
const managers = [
|
||||
{ name: 'first', stop: jest.fn(() => { order.push('first'); }) },
|
||||
{ name: 'second', stop: jest.fn(() => { order.push('second'); }) },
|
||||
{ name: 'third', stop: jest.fn(() => { order.push('third'); }) },
|
||||
];
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
managers,
|
||||
});
|
||||
|
||||
c.shutdown('SIGTERM');
|
||||
// Wait for the async chain (server.close → _stopManagersInOrder →
|
||||
// process.exit) to settle. The mock exit is synchronous so this
|
||||
// resolves once all microtasks drain.
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
expect(order).toEqual(['first', 'second', 'third']);
|
||||
});
|
||||
|
||||
test('does NOT stop managers until server.close callback fires', () => {
|
||||
const order = [];
|
||||
// Use a server whose close callback fires only when we manually call it.
|
||||
let deferredCloseCb;
|
||||
const server = {
|
||||
close: jest.fn((cb) => { deferredCloseCb = cb; }),
|
||||
};
|
||||
const managers = [
|
||||
{ name: 'first', stop: jest.fn(() => { order.push('first'); }) },
|
||||
];
|
||||
const c = createShutdownCoordinator({
|
||||
server,
|
||||
log: makeFakeLog(),
|
||||
managers,
|
||||
});
|
||||
|
||||
c.shutdown('SIGTERM');
|
||||
|
||||
// server.close has been called but its callback hasn't fired yet.
|
||||
expect(server.close).toHaveBeenCalledTimes(1);
|
||||
// Manager has NOT been stopped yet — server is still draining.
|
||||
expect(order).toEqual([]);
|
||||
|
||||
// Now fire the deferred callback to simulate drain completion.
|
||||
deferredCloseCb();
|
||||
|
||||
// Manager stopped AFTER server.close fired.
|
||||
expect(order).toEqual(['first']);
|
||||
});
|
||||
|
||||
test('continues stopping remaining managers if one throws', async () => {
|
||||
const order = [];
|
||||
const managers = [
|
||||
{ name: 'first', stop: jest.fn(() => { order.push('first'); }) },
|
||||
{ name: 'broken', stop: jest.fn(() => { throw new Error('boom'); }) },
|
||||
{ name: 'third', stop: jest.fn(() => { order.push('third'); }) },
|
||||
];
|
||||
const log = makeFakeLog();
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log,
|
||||
managers,
|
||||
});
|
||||
|
||||
c.shutdown('SIGTERM');
|
||||
await new Promise((resolve) => setImmediate(resolve));
|
||||
|
||||
expect(order).toEqual(['first', 'third']);
|
||||
expect(log.warn).toHaveBeenCalledWith(
|
||||
'shutdown',
|
||||
'manager stop failed: broken',
|
||||
expect.objectContaining({ error: 'boom' }),
|
||||
);
|
||||
});
|
||||
|
||||
test('is idempotent — second shutdown() returns without re-running', () => {
|
||||
const server = makeFakeServer();
|
||||
const c = createShutdownCoordinator({
|
||||
server,
|
||||
log: makeFakeLog(),
|
||||
managers: [{ name: 'm', stop: jest.fn() }],
|
||||
});
|
||||
|
||||
c.shutdown('SIGTERM');
|
||||
c.shutdown('SIGTERM');
|
||||
c.shutdown('SIGINT');
|
||||
|
||||
expect(server.close).toHaveBeenCalledTimes(1);
|
||||
expect(c.isShuttingDown()).toBe(true);
|
||||
});
|
||||
|
||||
test('isShuttingDown() flips false→true on first shutdown call', () => {
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
managers: [],
|
||||
});
|
||||
expect(c.isShuttingDown()).toBe(false);
|
||||
c.shutdown('SIGTERM');
|
||||
expect(c.isShuttingDown()).toBe(true);
|
||||
});
|
||||
|
||||
test('force-exits after drainTimeoutMs if server.close never fires', () => {
|
||||
jest.useFakeTimers();
|
||||
try {
|
||||
const server = makeFakeServer({ closeBehavior: 'never' });
|
||||
const log = makeFakeLog();
|
||||
const c = createShutdownCoordinator({
|
||||
server,
|
||||
log,
|
||||
drainTimeoutMs: 1000,
|
||||
managers: [],
|
||||
});
|
||||
|
||||
c.shutdown('SIGTERM');
|
||||
expect(exitCalls).toEqual([]);
|
||||
|
||||
jest.advanceTimersByTime(999);
|
||||
expect(exitCalls).toEqual([]);
|
||||
|
||||
jest.advanceTimersByTime(2);
|
||||
expect(exitCalls).toEqual([0]);
|
||||
expect(log.warn).toHaveBeenCalledWith(
|
||||
'shutdown',
|
||||
expect.stringContaining('drain timeout (1000ms) reached before HTTP server closed'),
|
||||
);
|
||||
} finally {
|
||||
jest.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
test('force-exits after drainTimeoutMs if manager.stop() hangs after HTTP close', () => {
|
||||
jest.useFakeTimers();
|
||||
try {
|
||||
const server = makeFakeServer(); // calls back immediately
|
||||
const log = makeFakeLog();
|
||||
// Manager that NEVER resolves — simulates a hung cleanup.
|
||||
const hungManager = {
|
||||
name: 'hung',
|
||||
stop: jest.fn(() => new Promise(() => {})), // never resolves
|
||||
};
|
||||
const c = createShutdownCoordinator({
|
||||
server,
|
||||
log,
|
||||
drainTimeoutMs: 1000,
|
||||
managers: [hungManager],
|
||||
});
|
||||
|
||||
c.shutdown('SIGTERM');
|
||||
// After the synchronous shutdown() call: server.close has fired
|
||||
// (serverClosed=true), but hungManager.stop() has been called and
|
||||
// its promise is pending. managersStopped is still false.
|
||||
// process.exit should NOT have been called yet.
|
||||
expect(exitCalls).toEqual([]);
|
||||
|
||||
jest.advanceTimersByTime(1001);
|
||||
// Now the safety-net timer fires — force-exit because manager hung.
|
||||
expect(exitCalls).toEqual([0]);
|
||||
expect(log.warn).toHaveBeenCalledWith(
|
||||
'shutdown',
|
||||
expect.stringContaining('after HTTP close (manager hung)'),
|
||||
);
|
||||
} finally {
|
||||
jest.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
test('clears force-exit timer when manager drain completes promptly', () => {
|
||||
jest.useFakeTimers();
|
||||
try {
|
||||
const server = makeFakeServer(); // calls back on the same tick
|
||||
const log = makeFakeLog();
|
||||
// Quick-stopping manager. The close callback awaits stop(),
|
||||
// which resolves immediately, so managersStopped flips true
|
||||
// and the safety-net timer is cleared before it can fire.
|
||||
const fastManager = {
|
||||
name: 'fast',
|
||||
stop: jest.fn(() => Promise.resolve()),
|
||||
};
|
||||
const c = createShutdownCoordinator({
|
||||
server,
|
||||
log,
|
||||
drainTimeoutMs: 1000,
|
||||
managers: [fastManager],
|
||||
});
|
||||
|
||||
c.shutdown('SIGTERM');
|
||||
|
||||
// Flush microtasks so the close callback's await stop() resolves,
|
||||
// managersStopped flips true, the timer is cleared, and
|
||||
// process.exit(0) is recorded exactly once.
|
||||
return Promise.resolve().then(() => Promise.resolve()).then(() => {
|
||||
expect(exitCalls).toEqual([0]);
|
||||
|
||||
// Advance well past the drain timeout — no extra exit should fire.
|
||||
jest.advanceTimersByTime(5000);
|
||||
expect(exitCalls).toEqual([0]);
|
||||
});
|
||||
} finally {
|
||||
jest.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('installSignalHandlers()', () => {
|
||||
// Track listeners added during each test so we can remove them in
|
||||
// afterEach. process.on() listeners leak across tests otherwise.
|
||||
let addedListeners;
|
||||
let originalProcessOn;
|
||||
|
||||
beforeEach(() => {
|
||||
addedListeners = [];
|
||||
originalProcessOn = process.on;
|
||||
// Wrap process.on to record every (signal, listener) pair we add.
|
||||
// Must capture originalProcessOn at wrap time so we can call it.
|
||||
const realOn = originalProcessOn;
|
||||
process.on = function patchedOn(signal, listener) {
|
||||
addedListeners.push({ signal, listener });
|
||||
return realOn.call(process, signal, listener);
|
||||
};
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.on = originalProcessOn;
|
||||
for (const { signal, listener } of addedListeners) {
|
||||
originalProcessOn.call(process, signal, listener); // ensure clean slate
|
||||
process.removeListener(signal, listener);
|
||||
}
|
||||
addedListeners = [];
|
||||
});
|
||||
|
||||
test('registers listeners on the given signals', () => {
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
managers: [],
|
||||
});
|
||||
const shutdownSpy = jest.spyOn(c, 'shutdown');
|
||||
|
||||
installSignalHandlers(c);
|
||||
|
||||
// Emit fake signals through process.emit to verify the listener was
|
||||
// registered (process.on listens to the process EventEmitter).
|
||||
process.emit('SIGTERM');
|
||||
process.emit('SIGINT');
|
||||
|
||||
expect(shutdownSpy).toHaveBeenCalledWith('SIGTERM');
|
||||
expect(shutdownSpy).toHaveBeenCalledWith('SIGINT');
|
||||
});
|
||||
|
||||
test('accepts custom signal list', () => {
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
managers: [],
|
||||
});
|
||||
const shutdownSpy = jest.spyOn(c, 'shutdown');
|
||||
|
||||
installSignalHandlers(c, ['SIGHUP']);
|
||||
|
||||
process.emit('SIGHUP');
|
||||
expect(shutdownSpy).toHaveBeenCalledWith('SIGHUP');
|
||||
});
|
||||
|
||||
test('is idempotent — calling installSignalHandlers twice does not double-register', () => {
|
||||
const c = createShutdownCoordinator({
|
||||
server: makeFakeServer(),
|
||||
log: makeFakeLog(),
|
||||
managers: [],
|
||||
});
|
||||
const shutdownSpy = jest.spyOn(c, 'shutdown');
|
||||
|
||||
installSignalHandlers(c);
|
||||
installSignalHandlers(c); // second call
|
||||
installSignalHandlers(c); // third call
|
||||
|
||||
// The installedSignals tracker should have one entry per signal.
|
||||
expect(c._installedSignals).toEqual(['SIGTERM', 'SIGINT']);
|
||||
|
||||
process.emit('SIGTERM');
|
||||
expect(shutdownSpy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -3,6 +3,9 @@
|
||||
"version": "1.15.0",
|
||||
"description": "DashCaddy API server - Dashboard backend for Docker, Caddy & DNS management",
|
||||
"main": "server.js",
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
},
|
||||
"scripts": {
|
||||
"start": "node server.js",
|
||||
"test": "jest",
|
||||
|
||||
@@ -36,7 +36,7 @@ module.exports = function({ authManager: _authManager, credentialManager: _crede
|
||||
break;
|
||||
case 'router': {
|
||||
// Validate baseUrl is a safe hostname before using in shell command
|
||||
if (!baseUrl || typeof baseUrl !== 'string' || !/^https?:\/\/[a-zA-Z0-9]([a-zA-Z0-9.\-]{0,253}[a-zA-Z0-9])?(:\d{1,5})?(\/|$)/.test(baseUrl)) {
|
||||
if (!baseUrl || typeof baseUrl !== 'string' || !/^https?:\/\/[a-zA-Z0-9]([a-zA-Z0-9.-]{0,253}[a-zA-Z0-9])?(:\d{1,5})?(\/|$)/.test(baseUrl)) {
|
||||
log.warn('auth', 'Router auto-login rejected: invalid baseUrl', { serviceId, baseUrl: String(baseUrl).substring(0, 50) });
|
||||
appSessionCache.set(serviceId, { failed: true, exp: Date.now() + SESSION_TTL.FAILED_LOGIN });
|
||||
return null;
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
const express = require('express');
|
||||
const { success, error: errorResponse } = require('../src/utils/responses');
|
||||
|
||||
/**
|
||||
* Disk space management routes
|
||||
*
|
||||
* GET /disk — current usage snapshot (budget, breakdown, status)
|
||||
* GET /disk/breakdown — detailed breakdown incl. per-container log sizes
|
||||
* GET /disk/config — get disk budget settings
|
||||
* POST /disk/config — update disk budget settings
|
||||
* POST /disk/cleanup — trigger manual cleanup (standard|aggressive|logs-only)
|
||||
*/
|
||||
module.exports = function({ diskSpaceMonitor, asyncHandler, log }) {
|
||||
const router = express.Router();
|
||||
|
||||
// Current disk usage snapshot
|
||||
router.get('/', asyncHandler(async (req, res) => {
|
||||
const snapshot = await diskSpaceMonitor.getSnapshot();
|
||||
success(res, snapshot);
|
||||
}, 'disk-get'));
|
||||
|
||||
// Detailed breakdown (includes per-container log sizes)
|
||||
router.get('/breakdown', asyncHandler(async (req, res) => {
|
||||
const breakdown = await diskSpaceMonitor.getDetailedBreakdown();
|
||||
success(res, breakdown);
|
||||
}, 'disk-breakdown'));
|
||||
|
||||
// Get disk budget config
|
||||
router.get('/config', asyncHandler(async (req, res) => {
|
||||
success(res, diskSpaceMonitor.getConfig());
|
||||
}, 'disk-config-get'));
|
||||
|
||||
// Update disk budget config
|
||||
router.post('/config', asyncHandler(async (req, res) => {
|
||||
const { diskBudgetGB, warningThresholdPct, criticalThresholdPct, autoCleanup, enabled, cleanupAggressivePct } = req.body;
|
||||
|
||||
const updates = {};
|
||||
if (typeof diskBudgetGB === 'number' && diskBudgetGB > 0) updates.diskBudgetGB = Math.min(diskBudgetGB, 1000);
|
||||
if (typeof warningThresholdPct === 'number') updates.warningThresholdPct = Math.min(Math.max(warningThresholdPct, 50), 99);
|
||||
if (typeof criticalThresholdPct === 'number') updates.criticalThresholdPct = Math.min(Math.max(criticalThresholdPct, 60), 99);
|
||||
if (typeof cleanupAggressivePct === 'number') updates.cleanupAggressivePct = Math.min(Math.max(cleanupAggressivePct, 70), 99);
|
||||
if (typeof autoCleanup === 'boolean') updates.autoCleanup = autoCleanup;
|
||||
if (typeof enabled === 'boolean') updates.enabled = enabled;
|
||||
|
||||
const config = diskSpaceMonitor.configure(updates);
|
||||
log.info('disk', 'Disk budget updated', updates);
|
||||
|
||||
success(res, { message: 'Disk budget updated', config });
|
||||
}, 'disk-config-set'));
|
||||
|
||||
// Manual cleanup trigger
|
||||
router.post('/cleanup', asyncHandler(async (req, res) => {
|
||||
const level = req.body?.level || 'standard';
|
||||
if (!['standard', 'aggressive', 'logs-only'].includes(level)) {
|
||||
return errorResponse(res, 'Invalid cleanup level. Use: standard, aggressive, or logs-only', 400);
|
||||
}
|
||||
|
||||
log.info('disk', 'Manual cleanup triggered', { level, by: req.auth?.user || 'api' });
|
||||
const result = await diskSpaceMonitor.performCleanup(level);
|
||||
success(res, result);
|
||||
}, 'disk-cleanup'));
|
||||
|
||||
return router;
|
||||
};
|
||||
@@ -1,7 +1,20 @@
|
||||
const express = require('express');
|
||||
const rateLimit = require('express-rate-limit');
|
||||
const { success, error: errorResponse } = require('../src/utils/responses');
|
||||
const { ValidationError } = require('../src/utilities/errors');
|
||||
|
||||
// Dedicated rate limiter for license activation — prevents brute-force key guessing.
|
||||
// Pro keys follow a predictable format (DC-XXX-XXXXX-XXXXXX), so without rate
|
||||
// limiting an attacker could enumerate valid keys.
|
||||
const licenseActivateLimiter = rateLimit({
|
||||
windowMs: 15 * 60 * 1000, // 15 minutes
|
||||
max: 10, // 10 attempts per window per IP
|
||||
standardHeaders: true,
|
||||
legacyHeaders: false,
|
||||
message: { success: false, error: 'Too many license activation attempts. Please try again later.' },
|
||||
skip: () => process.env.NODE_ENV === 'test',
|
||||
});
|
||||
|
||||
/**
|
||||
* License routes factory
|
||||
* @param {Object} deps - Explicit dependencies
|
||||
@@ -13,7 +26,7 @@ module.exports = function({ licenseManager, asyncHandler }) {
|
||||
const router = express.Router();
|
||||
|
||||
// Activate a license code
|
||||
router.post('/activate', asyncHandler(async (req, res) => {
|
||||
router.post('/activate', licenseActivateLimiter, asyncHandler(async (req, res) => {
|
||||
const { code } = req.body;
|
||||
if (!code) {
|
||||
throw new ValidationError('License code is required');
|
||||
|
||||
@@ -1,489 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Comprehensive DashCaddy Security Test Suite
|
||||
* Tests all 11 security fixes with detailed verification
|
||||
*/
|
||||
|
||||
const http = require('http');
|
||||
const crypto = require('crypto');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const API_BASE = process.env.API_BASE || 'http://localhost:3001';
|
||||
const colors = {
|
||||
reset: '\x1b[0m',
|
||||
green: '\x1b[32m',
|
||||
red: '\x1b[31m',
|
||||
yellow: '\x1b[33m',
|
||||
blue: '\x1b[34m',
|
||||
cyan: '\x1b[36m',
|
||||
magenta: '\x1b[35m'
|
||||
};
|
||||
|
||||
const testResults = {
|
||||
passed: 0,
|
||||
failed: 0,
|
||||
warnings: 0,
|
||||
total: 0,
|
||||
details: []
|
||||
};
|
||||
|
||||
function log(message, color = 'reset') {
|
||||
console.log(`${colors[color]}${message}${colors.reset}`);
|
||||
}
|
||||
|
||||
function logSection(title) {
|
||||
console.log(`\n${colors.cyan}${'═'.repeat(60)}${colors.reset}`);
|
||||
console.log(`${colors.cyan} ${title}${colors.reset}`);
|
||||
console.log(`${colors.cyan}${'═'.repeat(60)}${colors.reset}\n`);
|
||||
}
|
||||
|
||||
function recordTest(name, passed, message, warning = false) {
|
||||
testResults.total++;
|
||||
if (warning) {
|
||||
testResults.warnings++;
|
||||
log(` ⚠ ${name}: ${message}`, 'yellow');
|
||||
} else if (passed) {
|
||||
testResults.passed++;
|
||||
log(` ✓ ${name}: ${message}`, 'green');
|
||||
} else {
|
||||
testResults.failed++;
|
||||
log(` ✗ ${name}: ${message}`, 'red');
|
||||
}
|
||||
testResults.details.push({ name, passed, message, warning });
|
||||
}
|
||||
|
||||
async function makeRequest(path, options = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const url = new URL(path, API_BASE);
|
||||
const requestOptions = {
|
||||
hostname: url.hostname,
|
||||
port: url.port || 80,
|
||||
path: url.pathname + url.search,
|
||||
method: options.method || 'GET',
|
||||
headers: options.headers || {},
|
||||
timeout: options.timeout || 10000
|
||||
};
|
||||
|
||||
const req = http.request(requestOptions, (res) => {
|
||||
let data = '';
|
||||
res.on('data', chunk => data += chunk);
|
||||
res.on('end', () => {
|
||||
resolve({
|
||||
statusCode: res.statusCode,
|
||||
headers: res.headers,
|
||||
body: data,
|
||||
data: data && (data.startsWith('{') || data.startsWith('[')) ?
|
||||
(() => { try { return JSON.parse(data); } catch(e) { return null; } })() : data
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
req.on('error', reject);
|
||||
req.on('timeout', () => {
|
||||
req.destroy();
|
||||
reject(new Error('Request timeout'));
|
||||
});
|
||||
|
||||
if (options.body) {
|
||||
req.write(typeof options.body === 'string' ? options.body : JSON.stringify(options.body));
|
||||
}
|
||||
|
||||
req.end();
|
||||
});
|
||||
}
|
||||
|
||||
// Test 1: Startup Validation & Health Checks
|
||||
async function testStartupValidation() {
|
||||
logSection('TEST 1: Startup Validation & Health Checks');
|
||||
|
||||
try {
|
||||
const response = await makeRequest('/health');
|
||||
if (response.statusCode === 200 && response.data?.status === 'ok') {
|
||||
recordTest('Health Endpoint', true, `Server healthy (${response.data.timestamp})`);
|
||||
} else {
|
||||
recordTest('Health Endpoint', false, `Unexpected response: ${response.statusCode}`);
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('Health Endpoint', false, `Error: ${error.message}`);
|
||||
}
|
||||
|
||||
// Check for startup validation in logs (requires Docker access)
|
||||
log('\n Manual check: Run "docker logs dashcaddy-api | grep validation"', 'yellow');
|
||||
log(' Expected: "✓ Startup configuration validation passed"', 'yellow');
|
||||
}
|
||||
|
||||
// Test 2: CSRF Protection
|
||||
async function testCSRFProtection() {
|
||||
logSection('TEST 2: CSRF Protection');
|
||||
|
||||
// Test 2a: CSRF cookie is set
|
||||
try {
|
||||
const response = await makeRequest('/api/services');
|
||||
const csrfCookie = response.headers['set-cookie']?.find(c => c.includes('dashcaddy_csrf'));
|
||||
|
||||
if (csrfCookie) {
|
||||
const hasMaxAge = csrfCookie.includes('Max-Age');
|
||||
const hasSameSite = csrfCookie.includes('SameSite=Strict');
|
||||
|
||||
if (hasMaxAge && hasSameSite) {
|
||||
recordTest('CSRF Cookie', true, 'Cookie set with correct attributes (Max-Age, SameSite=Strict)');
|
||||
} else {
|
||||
recordTest('CSRF Cookie', true, 'Cookie set but missing some attributes', true);
|
||||
}
|
||||
} else {
|
||||
recordTest('CSRF Cookie', false, 'CSRF cookie not set in response');
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('CSRF Cookie', false, `Error: ${error.message}`);
|
||||
}
|
||||
|
||||
// Test 2b: POST without CSRF token is blocked
|
||||
try {
|
||||
const response = await makeRequest('/api/test-endpoint', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: { test: 'data' }
|
||||
});
|
||||
|
||||
if (response.data?.error?.includes('CSRF') || response.data?.message?.includes('CSRF')) {
|
||||
recordTest('CSRF Validation', true, 'POST blocked without CSRF token');
|
||||
} else if (response.statusCode === 401) {
|
||||
recordTest('CSRF Validation', true, 'Request requires authentication (CSRF check bypassed)', true);
|
||||
} else {
|
||||
recordTest('CSRF Validation', false, `Unexpected: ${JSON.stringify(response.data)}`);
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('CSRF Validation', false, `Error: ${error.message}`);
|
||||
}
|
||||
|
||||
// Test 2c: CSRF token endpoint (may require auth)
|
||||
try {
|
||||
const response = await makeRequest('/api/csrf-token');
|
||||
|
||||
if (response.statusCode === 200 && response.data?.token) {
|
||||
recordTest('CSRF Token Endpoint', true, 'Token endpoint returns valid token');
|
||||
} else if (response.statusCode === 401) {
|
||||
recordTest('CSRF Token Endpoint', true, 'Endpoint requires authentication (expected with TOTP)', true);
|
||||
} else {
|
||||
recordTest('CSRF Token Endpoint', false, `Unexpected response: ${response.statusCode}`);
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('CSRF Token Endpoint', false, `Error: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 3: Request Size Limits
|
||||
async function testRequestSizeLimits() {
|
||||
logSection('TEST 3: Request Size Limits');
|
||||
|
||||
// Test 3a: Small payload (should work)
|
||||
try {
|
||||
const smallPayload = { data: 'a'.repeat(100) };
|
||||
const response = await makeRequest('/api/services', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(smallPayload)
|
||||
});
|
||||
|
||||
if (response.statusCode !== 413) {
|
||||
recordTest('Small Payload', true, `Accepted (${response.statusCode})`);
|
||||
} else {
|
||||
recordTest('Small Payload', false, 'Small payload rejected as too large');
|
||||
}
|
||||
} catch (error) {
|
||||
if (!error.message.includes('413')) {
|
||||
recordTest('Small Payload', true, 'Accepted (non-size error)');
|
||||
} else {
|
||||
recordTest('Small Payload', false, `Rejected: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 3b: Check if large payloads are rejected (without actually sending 2MB)
|
||||
log('\n Info: Testing large payload rejection requires actual 2MB POST', 'blue');
|
||||
log(' Expected behavior: Payloads > 1MB rejected with 413', 'blue');
|
||||
recordTest('Large Payload Rejection', true, 'Mechanism in place (verified in logs)', true);
|
||||
}
|
||||
|
||||
// Test 4: Enhanced Error Logging
|
||||
async function testErrorLogging() {
|
||||
logSection('TEST 4: Enhanced Error Logging (Request IDs)');
|
||||
|
||||
try {
|
||||
const response = await makeRequest('/api/services');
|
||||
const requestId = response.headers['x-request-id'];
|
||||
|
||||
if (requestId) {
|
||||
const uuidRegex = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
|
||||
if (uuidRegex.test(requestId)) {
|
||||
recordTest('Request ID Header', true, `Valid UUID: ${requestId.substring(0, 13)}...`);
|
||||
} else {
|
||||
recordTest('Request ID Header', false, `Invalid UUID format: ${requestId}`);
|
||||
}
|
||||
} else {
|
||||
recordTest('Request ID Header', false, 'X-Request-ID header not present');
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('Request ID Header', false, `Error: ${error.message}`);
|
||||
}
|
||||
|
||||
log('\n Manual check: Error logs should include IP, User-Agent, Method, Path', 'yellow');
|
||||
log(' Run: docker logs dashcaddy-api | grep -i "error" | tail -5', 'yellow');
|
||||
}
|
||||
|
||||
// Test 5: Authentication Layer
|
||||
async function testAuthentication() {
|
||||
logSection('TEST 5: Authentication Layer');
|
||||
|
||||
// Test 5a: Auth endpoints exist
|
||||
try {
|
||||
const response = await makeRequest('/api/auth/keys');
|
||||
|
||||
if (response.statusCode === 401) {
|
||||
recordTest('Auth Endpoints', true, 'Auth required (TOTP enabled)');
|
||||
} else if (response.statusCode === 200) {
|
||||
recordTest('Auth Endpoints', true, 'Endpoint accessible (TOTP disabled)', true);
|
||||
} else {
|
||||
recordTest('Auth Endpoints', false, `Unexpected status: ${response.statusCode}`);
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('Auth Endpoints', false, `Error: ${error.message}`);
|
||||
}
|
||||
|
||||
// Test 5b: Check AuthManager in logs
|
||||
log('\n Manual check: Verify AuthManager initialized', 'yellow');
|
||||
log(' Run: docker logs dashcaddy-api | grep AuthManager', 'yellow');
|
||||
log(' Expected: "[AuthManager] Initialized"', 'yellow');
|
||||
}
|
||||
|
||||
// Test 6: Port Locking
|
||||
async function testPortLocking() {
|
||||
logSection('TEST 6: Port Locking Mechanism');
|
||||
|
||||
log(' Manual check: Port lock directory created in container', 'yellow');
|
||||
log(' Run: docker logs dashcaddy-api | grep PortLockManager', 'yellow');
|
||||
log(' Expected: "[PortLockManager] Created lock directory: /app/.port-locks"', 'yellow');
|
||||
log(' Expected: "[PortLockManager] Cleanup complete: X stale locks removed"', 'yellow');
|
||||
|
||||
// Check if module exists locally
|
||||
const modulePath = path.join(__dirname, 'port-lock-manager.js');
|
||||
if (fs.existsSync(modulePath)) {
|
||||
recordTest('Port Lock Module', true, 'port-lock-manager.js exists');
|
||||
} else {
|
||||
recordTest('Port Lock Module', false, 'port-lock-manager.js not found');
|
||||
}
|
||||
}
|
||||
|
||||
// Test 7: Docker Security Module
|
||||
async function testDockerSecurity() {
|
||||
logSection('TEST 7: Docker Image Verification');
|
||||
|
||||
const modulePath = path.join(__dirname, 'docker-security.js');
|
||||
if (fs.existsSync(modulePath)) {
|
||||
recordTest('Docker Security Module', true, 'docker-security.js exists');
|
||||
} else {
|
||||
recordTest('Docker Security Module', false, 'docker-security.js not found');
|
||||
}
|
||||
|
||||
log('\n Manual check: Docker security initialized', 'yellow');
|
||||
log(' Run: docker logs dashcaddy-api | grep DockerSecurity', 'yellow');
|
||||
log(' Expected: "[DockerSecurity] Initialized in verify mode"', 'yellow');
|
||||
}
|
||||
|
||||
// Test 8: Hardcoded Secrets Removal
|
||||
async function testSecretsRemoval() {
|
||||
logSection('TEST 8: Hardcoded Secrets Removal');
|
||||
|
||||
try {
|
||||
const templatesPath = path.join(__dirname, 'app-templates.js');
|
||||
const content = fs.readFileSync(templatesPath, 'utf8');
|
||||
|
||||
const changeMe123 = (content.match(/changeme123/g) || []).length;
|
||||
const secretsConfigs = (content.match(/secrets:\s*\[/g) || []).length;
|
||||
|
||||
if (changeMe123 === 0) {
|
||||
recordTest('Hardcoded Secrets', true, 'No "changeme123" found in templates');
|
||||
} else {
|
||||
recordTest('Hardcoded Secrets', false, `Found ${changeMe123} instances of "changeme123"`);
|
||||
}
|
||||
|
||||
if (secretsConfigs >= 10) {
|
||||
recordTest('Secrets Configurations', true, `Found ${secretsConfigs} secrets configs`);
|
||||
} else {
|
||||
recordTest('Secrets Configurations', false, `Only ${secretsConfigs} configs (expected 14+)`);
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('Hardcoded Secrets', false, `Error reading templates: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 9: LRU Cache Implementation
|
||||
async function testLRUCache() {
|
||||
logSection('TEST 9: Session Management (LRU Cache)');
|
||||
|
||||
// Check if cache-config exists
|
||||
const cacheConfigPath = path.join(__dirname, 'cache-config.js');
|
||||
if (fs.existsSync(cacheConfigPath)) {
|
||||
recordTest('LRU Cache Module', true, 'cache-config.js exists');
|
||||
|
||||
try {
|
||||
const content = fs.readFileSync(cacheConfigPath, 'utf8');
|
||||
if (content.includes('LRUCache')) {
|
||||
recordTest('LRU Implementation', true, 'Uses LRUCache from lru-cache package');
|
||||
} else {
|
||||
recordTest('LRU Implementation', false, 'LRUCache not found in cache-config.js');
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('LRU Implementation', false, `Error: ${error.message}`);
|
||||
}
|
||||
} else {
|
||||
recordTest('LRU Cache Module', false, 'cache-config.js not found');
|
||||
}
|
||||
|
||||
// Check server.js for cache usage
|
||||
try {
|
||||
const serverPath = path.join(__dirname, 'server.js');
|
||||
const content = fs.readFileSync(serverPath, 'utf8');
|
||||
|
||||
const cacheUsage = (content.match(/createCache\(/g) || []).length;
|
||||
if (cacheUsage >= 4) {
|
||||
recordTest('Cache Usage', true, `Found ${cacheUsage} cache instances in server.js`);
|
||||
} else {
|
||||
recordTest('Cache Usage', false, `Only ${cacheUsage} instances (expected 4+)`);
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('Cache Usage', false, `Error: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 10: Frontend CSRF Integration
|
||||
async function testFrontendCSRF() {
|
||||
logSection('TEST 10: Frontend CSRF Integration');
|
||||
|
||||
try {
|
||||
const indexPath = path.join(__dirname, '..', 'status', 'index.html');
|
||||
|
||||
if (!fs.existsSync(indexPath)) {
|
||||
recordTest('Frontend File', false, 'index.html not found');
|
||||
return;
|
||||
}
|
||||
|
||||
const content = fs.readFileSync(indexPath, 'utf8');
|
||||
|
||||
// Check for CSRF helper functions
|
||||
if (content.includes('getCSRFToken') && content.includes('secureFetch')) {
|
||||
recordTest('CSRF Helpers', true, 'getCSRFToken() and secureFetch() found');
|
||||
} else {
|
||||
recordTest('CSRF Helpers', false, 'CSRF helper functions not found');
|
||||
}
|
||||
|
||||
// Check for secureFetch usage
|
||||
const secureFetchUsage = (content.match(/secureFetch\(/g) || []).length;
|
||||
if (secureFetchUsage >= 30) {
|
||||
recordTest('Frontend Integration', true, `${secureFetchUsage} secureFetch calls found`);
|
||||
} else {
|
||||
recordTest('Frontend Integration', false, `Only ${secureFetchUsage} calls (expected 30+)`);
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('Frontend CSRF', false, `Error: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 11: Path Traversal Protection
|
||||
async function testPathTraversal() {
|
||||
logSection('TEST 11: Path Traversal Protection');
|
||||
|
||||
// Check if validateSecurePath exists in input-validator
|
||||
try {
|
||||
const validatorPath = path.join(__dirname, 'input-validator.js');
|
||||
const content = fs.readFileSync(validatorPath, 'utf8');
|
||||
|
||||
if (content.includes('validateSecurePath')) {
|
||||
recordTest('Path Validation Function', true, 'validateSecurePath() found in input-validator.js');
|
||||
|
||||
if (content.includes('fs.promises.realpath') || content.includes('realpath')) {
|
||||
recordTest('Realpath Implementation', true, 'Uses fs.realpath() for symlink resolution');
|
||||
} else {
|
||||
recordTest('Realpath Implementation', false, 'Does not use realpath()');
|
||||
}
|
||||
} else {
|
||||
recordTest('Path Validation Function', false, 'validateSecurePath() not found');
|
||||
}
|
||||
} catch (error) {
|
||||
recordTest('Path Traversal Protection', false, `Error: ${error.message}`);
|
||||
}
|
||||
|
||||
log('\n Note: Path traversal endpoints require authentication to test', 'yellow');
|
||||
}
|
||||
|
||||
// Main test runner
|
||||
async function runAllTests() {
|
||||
log('\n╔════════════════════════════════════════════════════════════╗', 'magenta');
|
||||
log('║ DashCaddy Comprehensive Security Test Suite ║', 'magenta');
|
||||
log('╚════════════════════════════════════════════════════════════╝', 'magenta');
|
||||
|
||||
log(`\nAPI Base: ${API_BASE}`, 'blue');
|
||||
log(`Test Time: ${new Date().toISOString()}`, 'blue');
|
||||
log('\nRunning comprehensive security tests...\n', 'blue');
|
||||
|
||||
await testStartupValidation();
|
||||
await testCSRFProtection();
|
||||
await testRequestSizeLimits();
|
||||
await testErrorLogging();
|
||||
await testAuthentication();
|
||||
await testPortLocking();
|
||||
await testDockerSecurity();
|
||||
await testSecretsRemoval();
|
||||
await testLRUCache();
|
||||
await testFrontendCSRF();
|
||||
await testPathTraversal();
|
||||
|
||||
// Summary
|
||||
logSection('TEST SUMMARY');
|
||||
|
||||
const passRate = testResults.total > 0
|
||||
? ((testResults.passed / testResults.total) * 100).toFixed(1)
|
||||
: 0;
|
||||
|
||||
log(`Total Tests: ${testResults.total}`, 'blue');
|
||||
log(`Passed: ${testResults.passed}`, 'green');
|
||||
log(`Failed: ${testResults.failed}`, testResults.failed > 0 ? 'red' : 'green');
|
||||
log(`Warnings: ${testResults.warnings}`, 'yellow');
|
||||
log(`Success Rate: ${passRate}%`, passRate >= 80 ? 'green' : 'yellow');
|
||||
|
||||
if (testResults.failed > 0) {
|
||||
log('\nFailed Tests:', 'red');
|
||||
testResults.details
|
||||
.filter(t => !t.passed && !t.warning)
|
||||
.forEach(t => log(` ✗ ${t.name}: ${t.message}`, 'red'));
|
||||
}
|
||||
|
||||
if (testResults.warnings > 0) {
|
||||
log('\nWarnings (Manual Verification Needed):', 'yellow');
|
||||
testResults.details
|
||||
.filter(t => t.warning)
|
||||
.forEach(t => log(` ⚠ ${t.name}: ${t.message}`, 'yellow'));
|
||||
}
|
||||
|
||||
log('\n' + '═'.repeat(60), 'cyan');
|
||||
|
||||
if (testResults.failed === 0) {
|
||||
log('\n✅ ALL AUTOMATED TESTS PASSED!', 'green');
|
||||
log('Review warnings above for manual verification steps.\n', 'yellow');
|
||||
} else {
|
||||
log('\n⚠️ Some tests failed. Review details above.\n', 'yellow');
|
||||
}
|
||||
|
||||
process.exit(testResults.failed > 0 ? 1 : 0);
|
||||
}
|
||||
|
||||
// Run tests
|
||||
if (require.main === module) {
|
||||
runAllTests().catch(error => {
|
||||
log(`\nFatal error: ${error.message}`, 'red');
|
||||
console.error(error);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = { runAllTests };
|
||||
@@ -1,386 +0,0 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Automated Testing Script for DashCaddy Security Fixes
|
||||
*
|
||||
* Tests all implemented security improvements:
|
||||
* 1. Path traversal protection
|
||||
* 2. Request size limits
|
||||
* 3. Startup validation
|
||||
* 4. Port locking
|
||||
* 5. Session management (LRU cache)
|
||||
* 6. Enhanced error logging
|
||||
* 7. Hardcoded secrets removal
|
||||
*/
|
||||
|
||||
const http = require('http');
|
||||
const https = require('https');
|
||||
const crypto = require('crypto');
|
||||
|
||||
const API_BASE = process.env.API_BASE || 'http://localhost:3001';
|
||||
const TEST_RESULTS = [];
|
||||
|
||||
// Color codes for terminal output
|
||||
const colors = {
|
||||
reset: '\x1b[0m',
|
||||
green: '\x1b[32m',
|
||||
red: '\x1b[31m',
|
||||
yellow: '\x1b[33m',
|
||||
blue: '\x1b[34m',
|
||||
cyan: '\x1b[36m'
|
||||
};
|
||||
|
||||
function log(message, color = 'reset') {
|
||||
console.log(`${colors[color]}${message}${colors.reset}`);
|
||||
}
|
||||
|
||||
function logTest(name) {
|
||||
console.log(`\n${colors.cyan}━━━ Testing: ${name} ━━━${colors.reset}`);
|
||||
}
|
||||
|
||||
function logResult(passed, message) {
|
||||
const icon = passed ? '✓' : '✗';
|
||||
const color = passed ? 'green' : 'red';
|
||||
log(` ${icon} ${message}`, color);
|
||||
TEST_RESULTS.push({ passed, message });
|
||||
}
|
||||
|
||||
async function makeRequest(path, options = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const url = new URL(path, API_BASE);
|
||||
const isHttps = url.protocol === 'https:';
|
||||
const client = isHttps ? https : http;
|
||||
|
||||
const requestOptions = {
|
||||
hostname: url.hostname,
|
||||
port: url.port || (isHttps ? 443 : 80),
|
||||
path: url.pathname + url.search,
|
||||
method: options.method || 'GET',
|
||||
headers: options.headers || {},
|
||||
...options
|
||||
};
|
||||
|
||||
const req = client.request(requestOptions, (res) => {
|
||||
let data = '';
|
||||
res.on('data', chunk => data += chunk);
|
||||
res.on('end', () => {
|
||||
resolve({
|
||||
statusCode: res.statusCode,
|
||||
headers: res.headers,
|
||||
body: data,
|
||||
data: data ? (data.startsWith('{') || data.startsWith('[') ? JSON.parse(data) : data) : null
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
req.on('error', reject);
|
||||
|
||||
if (options.body) {
|
||||
req.write(typeof options.body === 'string' ? options.body : JSON.stringify(options.body));
|
||||
}
|
||||
|
||||
req.end();
|
||||
});
|
||||
}
|
||||
|
||||
// Test 1: Path Traversal Protection
|
||||
async function testPathTraversal() {
|
||||
logTest('Path Traversal Protection');
|
||||
|
||||
const attacks = [
|
||||
{ path: '/api/browse/directories?path=../../../../../../etc/passwd', desc: 'Unix path traversal' },
|
||||
{ path: '/api/browse/directories?path=..\\..\\..\\Windows\\System32', desc: 'Windows path traversal' },
|
||||
{ path: '/api/browse/directories?path=%2e%2e%2f%2e%2e%2fetc%2fpasswd', desc: 'URL-encoded traversal' },
|
||||
{ path: '/api/browse/directories?path=/allowed/media/../../../secrets', desc: 'Mixed path traversal' }
|
||||
];
|
||||
|
||||
for (const attack of attacks) {
|
||||
try {
|
||||
const response = await makeRequest(attack.path);
|
||||
if (response.statusCode === 403 || response.statusCode === 400) {
|
||||
logResult(true, `Blocked: ${attack.desc}`);
|
||||
} else {
|
||||
logResult(false, `NOT BLOCKED (${response.statusCode}): ${attack.desc}`);
|
||||
}
|
||||
} catch (error) {
|
||||
logResult(false, `Error testing ${attack.desc}: ${error.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Test 2: Request Size Limits
|
||||
async function testRequestSizeLimits() {
|
||||
logTest('Request Size Limits');
|
||||
|
||||
// Test 1: Small payload (should work)
|
||||
try {
|
||||
const smallPayload = { data: 'a'.repeat(100) };
|
||||
const response = await makeRequest('/api/services', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(smallPayload)
|
||||
});
|
||||
logResult(true, 'Small payload accepted (100 bytes)');
|
||||
} catch (error) {
|
||||
logResult(false, `Small payload rejected: ${error.message}`);
|
||||
}
|
||||
|
||||
// Test 2: Large payload on general endpoint (should fail)
|
||||
try {
|
||||
const largePayload = { data: 'a'.repeat(2 * 1024 * 1024) }; // 2MB
|
||||
const response = await makeRequest('/api/services', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(largePayload)
|
||||
});
|
||||
if (response.statusCode === 413 || response.statusCode === 400) {
|
||||
logResult(true, 'Large payload rejected on general endpoint (2MB)');
|
||||
} else {
|
||||
logResult(false, `Large payload NOT rejected (status: ${response.statusCode})`);
|
||||
}
|
||||
} catch (error) {
|
||||
if (error.message.includes('413') || error.message.includes('ECONNRESET')) {
|
||||
logResult(true, 'Large payload rejected (connection reset)');
|
||||
} else {
|
||||
logResult(false, `Unexpected error: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 3: Large payload on logo endpoint (should work)
|
||||
try {
|
||||
const largeImage = 'a'.repeat(5 * 1024 * 1024); // 5MB
|
||||
const response = await makeRequest('/api/logo', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ logo: largeImage })
|
||||
});
|
||||
if (response.statusCode !== 413) {
|
||||
logResult(true, 'Large payload accepted on logo endpoint (5MB)');
|
||||
} else {
|
||||
logResult(false, 'Large payload rejected on logo endpoint');
|
||||
}
|
||||
} catch (error) {
|
||||
// May fail for other reasons (auth, validation), but not size
|
||||
if (!error.message.includes('413')) {
|
||||
logResult(true, 'Logo endpoint accepts large payloads (failed for non-size reason)');
|
||||
} else {
|
||||
logResult(false, `Logo endpoint rejected large payload: ${error.message}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Test 3: Startup Validation
|
||||
async function testStartupValidation() {
|
||||
logTest('Startup Validation');
|
||||
|
||||
// Check if server is running (implies validation passed)
|
||||
try {
|
||||
const response = await makeRequest('/health');
|
||||
if (response.statusCode === 200) {
|
||||
logResult(true, 'Server started successfully (validation passed)');
|
||||
} else {
|
||||
logResult(false, `Server health check failed: ${response.statusCode}`);
|
||||
}
|
||||
} catch (error) {
|
||||
logResult(false, `Cannot reach server: ${error.message}`);
|
||||
}
|
||||
|
||||
// Check for validation logs (requires access to logs)
|
||||
log(' → Check Docker logs for: "✓ Startup configuration validation passed"', 'yellow');
|
||||
}
|
||||
|
||||
// Test 4: Enhanced Error Logging (Request ID)
|
||||
async function testEnhancedLogging() {
|
||||
logTest('Enhanced Error Logging');
|
||||
|
||||
try {
|
||||
// Make a request that will be logged
|
||||
const response = await makeRequest('/api/services');
|
||||
|
||||
// Check if X-Request-ID header is present
|
||||
if (response.headers['x-request-id']) {
|
||||
const requestId = response.headers['x-request-id'];
|
||||
const isValidUUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i.test(requestId);
|
||||
|
||||
if (isValidUUID) {
|
||||
logResult(true, `Request ID header present and valid: ${requestId.substring(0, 8)}...`);
|
||||
} else {
|
||||
logResult(false, `Request ID present but invalid format: ${requestId}`);
|
||||
}
|
||||
} else {
|
||||
logResult(false, 'Request ID header not present');
|
||||
}
|
||||
} catch (error) {
|
||||
logResult(false, `Error testing logging: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 5: Session Management (LRU Cache)
|
||||
async function testSessionManagement() {
|
||||
logTest('Session Management (LRU Cache)');
|
||||
|
||||
log(' → This test requires code inspection (cannot test cache behavior externally)', 'yellow');
|
||||
log(' → Manual verification: Check server.js for LRUCache usage', 'yellow');
|
||||
|
||||
// We can test that sessions still work
|
||||
try {
|
||||
const response = await makeRequest('/api/totp/setup', { method: 'POST' });
|
||||
if (response.statusCode === 200 || response.statusCode === 401) {
|
||||
logResult(true, 'Session-based endpoints still functional');
|
||||
} else {
|
||||
logResult(false, `Unexpected response from session endpoint: ${response.statusCode}`);
|
||||
}
|
||||
} catch (error) {
|
||||
logResult(false, `Error testing session endpoints: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 6: Hardcoded Secrets Removal
|
||||
async function testSecretsRemoval() {
|
||||
logTest('Hardcoded Secrets Removal');
|
||||
|
||||
try {
|
||||
// Read app-templates.js and check for "changeme123"
|
||||
const fs = require('fs');
|
||||
const templatesPath = require('path').join(__dirname, 'app-templates.js');
|
||||
const content = fs.readFileSync(templatesPath, 'utf8');
|
||||
|
||||
const matches = content.match(/changeme123/g);
|
||||
if (!matches || matches.length === 0) {
|
||||
logResult(true, 'No hardcoded "changeme123" passwords found');
|
||||
} else {
|
||||
logResult(false, `Found ${matches.length} instances of "changeme123" still in templates`);
|
||||
}
|
||||
|
||||
// Check for secrets arrays
|
||||
const secretsMatches = content.match(/secrets:\s*\[/g);
|
||||
if (secretsMatches && secretsMatches.length >= 10) {
|
||||
logResult(true, `Found ${secretsMatches.length} secrets configurations`);
|
||||
} else {
|
||||
logResult(false, `Only found ${secretsMatches?.length || 0} secrets configurations (expected 14+)`);
|
||||
}
|
||||
} catch (error) {
|
||||
logResult(false, `Error reading templates: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 7: Port Locking Mechanism
|
||||
async function testPortLocking() {
|
||||
logTest('Port Locking Mechanism');
|
||||
|
||||
try {
|
||||
// Check if .port-locks directory exists
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const locksDir = path.join(__dirname, '.port-locks');
|
||||
|
||||
if (fs.existsSync(locksDir)) {
|
||||
logResult(true, 'Port locks directory exists');
|
||||
|
||||
// Check if it's writable
|
||||
try {
|
||||
const testFile = path.join(locksDir, 'test-write');
|
||||
fs.writeFileSync(testFile, 'test');
|
||||
fs.unlinkSync(testFile);
|
||||
logResult(true, 'Port locks directory is writable');
|
||||
} catch (error) {
|
||||
logResult(false, `Port locks directory not writable: ${error.message}`);
|
||||
}
|
||||
} else {
|
||||
logResult(false, 'Port locks directory does not exist');
|
||||
}
|
||||
|
||||
// Check if PortLockManager module exists
|
||||
const portLockPath = path.join(__dirname, 'port-lock-manager.js');
|
||||
if (fs.existsSync(portLockPath)) {
|
||||
logResult(true, 'PortLockManager module exists');
|
||||
} else {
|
||||
logResult(false, 'PortLockManager module not found');
|
||||
}
|
||||
} catch (error) {
|
||||
logResult(false, `Error testing port locking: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Test 8: Docker Security Module
|
||||
async function testDockerSecurity() {
|
||||
logTest('Docker Image Verification');
|
||||
|
||||
try {
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
// Check if docker-security.js exists
|
||||
const securityPath = path.join(__dirname, 'docker-security.js');
|
||||
if (fs.existsSync(securityPath)) {
|
||||
logResult(true, 'DockerSecurity module exists');
|
||||
} else {
|
||||
logResult(false, 'DockerSecurity module not found');
|
||||
}
|
||||
|
||||
// Check if config file exists
|
||||
const configPath = path.join(__dirname, 'docker-security-config.json');
|
||||
if (fs.existsSync(configPath)) {
|
||||
const config = JSON.parse(fs.readFileSync(configPath, 'utf8'));
|
||||
logResult(true, `Security config exists (mode: ${config.verificationMode || 'not set'})`);
|
||||
} else {
|
||||
log(' → Security config will be created on first use', 'yellow');
|
||||
logResult(true, 'Config will be auto-created');
|
||||
}
|
||||
} catch (error) {
|
||||
logResult(false, `Error testing Docker security: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Main test runner
|
||||
async function runTests() {
|
||||
log('\n╔════════════════════════════════════════════════════╗', 'cyan');
|
||||
log('║ DashCaddy Security Fixes - Test Suite ║', 'cyan');
|
||||
log('╚════════════════════════════════════════════════════╝', 'cyan');
|
||||
|
||||
log(`\nAPI Base URL: ${API_BASE}`, 'blue');
|
||||
log('Starting tests...\n', 'blue');
|
||||
|
||||
// Run all tests
|
||||
await testStartupValidation();
|
||||
await testPathTraversal();
|
||||
await testRequestSizeLimits();
|
||||
await testEnhancedLogging();
|
||||
await testSessionManagement();
|
||||
await testSecretsRemoval();
|
||||
await testPortLocking();
|
||||
await testDockerSecurity();
|
||||
|
||||
// Summary
|
||||
log('\n╔════════════════════════════════════════════════════╗', 'cyan');
|
||||
log('║ Test Summary ║', 'cyan');
|
||||
log('╚════════════════════════════════════════════════════╝', 'cyan');
|
||||
|
||||
const passed = TEST_RESULTS.filter(r => r.passed).length;
|
||||
const failed = TEST_RESULTS.filter(r => !r.passed).length;
|
||||
const total = TEST_RESULTS.length;
|
||||
|
||||
log(`\nTotal Tests: ${total}`, 'blue');
|
||||
log(`Passed: ${passed}`, 'green');
|
||||
log(`Failed: ${failed}`, failed > 0 ? 'red' : 'green');
|
||||
log(`Success Rate: ${((passed / total) * 100).toFixed(1)}%\n`, failed === 0 ? 'green' : 'yellow');
|
||||
|
||||
if (failed > 0) {
|
||||
log('Failed tests:', 'red');
|
||||
TEST_RESULTS.filter(r => !r.passed).forEach(r => {
|
||||
log(` ✗ ${r.message}`, 'red');
|
||||
});
|
||||
}
|
||||
|
||||
process.exit(failed > 0 ? 1 : 0);
|
||||
}
|
||||
|
||||
// Run tests if executed directly
|
||||
if (require.main === module) {
|
||||
runTests().catch(error => {
|
||||
log(`\nFatal error: ${error.message}`, 'red');
|
||||
console.error(error);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
|
||||
module.exports = { runTests };
|
||||
+39
-30
@@ -252,43 +252,52 @@ process.on('uncaughtException', (error) => {
|
||||
log.info('server', 'All feature modules initialized');
|
||||
});
|
||||
|
||||
// Graceful shutdown
|
||||
const shutdown = (signal) => {
|
||||
log.info('shutdown', `${signal} received, draining connections...`);
|
||||
|
||||
const resourceMonitor = require('./src/managers/resource-monitor');
|
||||
const backupManager = require('./src/utilities/backup-manager');
|
||||
const healthChecker = require('./src/monitoring/health-checker');
|
||||
const updateManager = require('./src/managers/update-manager');
|
||||
const selfUpdater = require('./src/docker/self-updater');
|
||||
|
||||
resourceMonitor.stop();
|
||||
backupManager.stop();
|
||||
healthChecker.stop();
|
||||
updateManager.stop();
|
||||
selfUpdater.stop();
|
||||
// Graceful shutdown (DC-067) — drains in-flight HTTP connections, stops
|
||||
// each manager in deterministic order, emits a 'shutdown' event for any
|
||||
// additional listeners, and force-exits after a 10s drain timeout.
|
||||
// Idempotent: a second SIGTERM during shutdown is a no-op.
|
||||
const {
|
||||
createShutdownCoordinator,
|
||||
installSignalHandlers,
|
||||
DEFAULT_DRAIN_TIMEOUT_MS,
|
||||
} = require('./src/utilities/shutdown');
|
||||
|
||||
const optionalManagers = [];
|
||||
try {
|
||||
const dockerMaintenance = require('./src/docker/docker-maintenance');
|
||||
dockerMaintenance.stop();
|
||||
} catch { /* optional */ }
|
||||
|
||||
optionalManagers.push({
|
||||
name: 'docker-maintenance',
|
||||
stop: () => require('./src/docker/docker-maintenance').stop(),
|
||||
});
|
||||
} catch { /* optional module */ }
|
||||
try {
|
||||
const logDigest = require('./src/security/log-digest');
|
||||
logDigest.stop();
|
||||
} catch { /* optional */ }
|
||||
optionalManagers.push({
|
||||
name: 'log-digest',
|
||||
stop: () => require('./src/security/log-digest').stop(),
|
||||
});
|
||||
} catch { /* optional module */ }
|
||||
|
||||
server.close(() => {
|
||||
log.info('shutdown', 'HTTP server closed');
|
||||
process.exit(0);
|
||||
const coordinator = createShutdownCoordinator({
|
||||
server,
|
||||
log,
|
||||
drainTimeoutMs: DEFAULT_DRAIN_TIMEOUT_MS,
|
||||
managers: [
|
||||
{ name: 'resource-monitor', stop: () => require('./src/managers/resource-monitor').stop() },
|
||||
{ name: 'backup-manager', stop: () => require('./src/utilities/backup-manager').stop() },
|
||||
{ name: 'health-checker', stop: () => require('./src/monitoring/health-checker').stop() },
|
||||
{ name: 'update-manager', stop: () => require('./src/managers/update-manager').stop() },
|
||||
{ name: 'self-updater', stop: () => require('./src/docker/self-updater').stop() },
|
||||
...optionalManagers,
|
||||
],
|
||||
});
|
||||
|
||||
// Force exit after 5s if connections don't drain
|
||||
setTimeout(() => process.exit(0), 5000).unref();
|
||||
};
|
||||
// Expose the shutdown signal as an event so additional listeners can
|
||||
// subscribe without touching this file. The coordinator is an
|
||||
// EventEmitter and emits 'shutdown' on SIGTERM/SIGINT.
|
||||
coordinator.on('shutdown', (signal) => {
|
||||
log.info('shutdown', 'shutdown event observed', { signal });
|
||||
});
|
||||
|
||||
process.on('SIGTERM', () => shutdown('SIGTERM'));
|
||||
process.on('SIGINT', () => shutdown('SIGINT'));
|
||||
installSignalHandlers(coordinator);
|
||||
|
||||
} catch (error) {
|
||||
console.error('[FATAL] Server startup failed:', error);
|
||||
|
||||
@@ -90,9 +90,11 @@ const DependencyManager = require('./managers/dependency-manager');
|
||||
const autoRestartRoutes = require('../routes/auto-restart');
|
||||
const configDriftRoutes = require('../routes/config-drift');
|
||||
const sslMonitorRoutes = require('../routes/ssl-monitor');
|
||||
const diskSpaceRoutes = require('../routes/disk-space');
|
||||
const { AutoRestartManager } = require('./managers/auto-restart-manager');
|
||||
const { ConfigDriftDetector } = require('./managers/config-drift-detector');
|
||||
const SSLMonitor = require('./monitoring/ssl-monitor');
|
||||
const { DiskSpaceMonitor } = require('./monitoring/disk-space-monitor');
|
||||
const DNSPropagationChecker = require('./dns/dns-propagation');
|
||||
|
||||
// Constants
|
||||
@@ -455,6 +457,12 @@ async function createApp() {
|
||||
sslMonitor.start(3600000); // 1 hour
|
||||
log.info('app', 'SSL monitor initialized');
|
||||
|
||||
// Initialize disk space monitor (disk budget + auto-cleanup)
|
||||
const diskSpaceMonitor = new DiskSpaceMonitor({ log, config: ctx.siteConfig });
|
||||
ctx.diskSpaceMonitor = diskSpaceMonitor;
|
||||
diskSpaceMonitor.start(600000); // 10 min
|
||||
log.info('app', 'Disk space monitor initialized', { budgetGB: diskSpaceMonitor.getConfig().diskBudgetGB });
|
||||
|
||||
// Initialize DNS propagation checker
|
||||
const dnsPropagationChecker = new DNSPropagationChecker(ctx);
|
||||
ctx.dnsPropagationChecker = dnsPropagationChecker;
|
||||
@@ -709,6 +717,11 @@ async function createApp() {
|
||||
asyncHandler: ctx.asyncHandler,
|
||||
logError: ctx.logError,
|
||||
}));
|
||||
apiRouter.use('/disk', diskSpaceRoutes({
|
||||
diskSpaceMonitor: ctx.diskSpaceMonitor,
|
||||
asyncHandler: ctx.asyncHandler,
|
||||
log: ctx.log,
|
||||
}));
|
||||
|
||||
// Inline API routes (mounted under /api/v1 below)
|
||||
// Note: /health lives at root only — see root-level health check below.
|
||||
|
||||
@@ -10,6 +10,7 @@
|
||||
|
||||
const { execFile } = require('child_process');
|
||||
const { promisify } = require('util');
|
||||
const crypto = require('crypto');
|
||||
const dns = require('dns');
|
||||
const os = require('os');
|
||||
const path = require('path');
|
||||
@@ -117,7 +118,7 @@ class RFC2136Provider extends BaseDNSProvider {
|
||||
*/
|
||||
async _runNsupdate(commands) {
|
||||
const script = commands.join('\n') + '\n';
|
||||
const tmpFile = path.join(os.tmpdir(), `nsupdate-${Date.now()}-${Math.random().toString(36).slice(2, 8)}.cmd`);
|
||||
const tmpFile = path.join(os.tmpdir(), `nsupdate-${crypto.randomBytes(4).toString('hex')}.cmd`);
|
||||
|
||||
try {
|
||||
await fs.promises.writeFile(tmpFile, script, { mode: 0o600 });
|
||||
|
||||
@@ -10,6 +10,7 @@
|
||||
const EventEmitter = require('events');
|
||||
const https = require('https');
|
||||
const http = require('http');
|
||||
const { log } = require('../utils/logging');
|
||||
const fs = require('fs');
|
||||
const fsp = require('fs').promises;
|
||||
const path = require('path');
|
||||
@@ -86,7 +87,7 @@ class SelfUpdater extends EventEmitter {
|
||||
start() {
|
||||
if (!this.config.enabled || this.checkTimer) return;
|
||||
|
||||
console.log('[SelfUpdater] Starting auto-update checks every %ds', this.config.checkInterval / 1000);
|
||||
log.info('updater', 'Starting auto-update checks', { intervalMs: this.config.checkInterval });
|
||||
|
||||
// First check after a short delay (let server finish startup)
|
||||
setTimeout(() => {
|
||||
@@ -124,7 +125,7 @@ class SelfUpdater extends EventEmitter {
|
||||
return { version: pkg.version, commit };
|
||||
} catch { /* try next candidate */ }
|
||||
}
|
||||
console.error('[SelfUpdater] getLocalVersion failed: no candidate package.json found');
|
||||
log.error('updater', 'getLocalVersion failed: no candidate package.json found');
|
||||
return { version: '0.0.0', commit: null };
|
||||
}
|
||||
|
||||
@@ -158,7 +159,7 @@ class SelfUpdater extends EventEmitter {
|
||||
// Fire-and-forget; the response shouldn't block on the container rebuild.
|
||||
setImmediate(() => {
|
||||
this._autoCheckAndApply().catch(err =>
|
||||
console.error('[SelfUpdater] %s-triggered update error: %s', triggeredBy, err.message)
|
||||
log.error('updater', err, { triggeredBy })
|
||||
);
|
||||
});
|
||||
return { accepted: true, triggeredBy };
|
||||
@@ -174,7 +175,7 @@ class SelfUpdater extends EventEmitter {
|
||||
try {
|
||||
remote = await this._fetchJson(`${this.config.updateUrl}/version.json`);
|
||||
} catch (primaryErr) {
|
||||
console.warn('[SelfUpdater] Primary server failed:', primaryErr.message, '— trying mirror');
|
||||
log.warn('updater', 'Primary server failed, trying mirror', { error: primaryErr.message });
|
||||
try {
|
||||
remote = await this._fetchJson(`${this.config.mirrorUrl}/version.json`);
|
||||
sourceUrl = this.config.mirrorUrl;
|
||||
@@ -240,7 +241,7 @@ class SelfUpdater extends EventEmitter {
|
||||
try {
|
||||
await this._downloadFile(primaryUrl, tarballPath);
|
||||
} catch (dlErr) {
|
||||
console.warn('[SelfUpdater] Primary download failed:', dlErr.message, '— trying mirror');
|
||||
log.warn('updater', 'Primary download failed, trying mirror', { error: dlErr.message });
|
||||
// Ensure file is fully cleaned up before mirror attempt
|
||||
try { fs.unlinkSync(tarballPath); } catch { /* ignore */ }
|
||||
await this._downloadFile(mirrorUrl, tarballPath);
|
||||
@@ -468,11 +469,11 @@ class SelfUpdater extends EventEmitter {
|
||||
try {
|
||||
const result = await this.checkForUpdate();
|
||||
if (result.available && result.remote) {
|
||||
console.log('[SelfUpdater] Update available: %s → %s', result.local.version, result.remote.version);
|
||||
log.info('updater', 'Update available', { localVersion: result.local.version, remoteVersion: result.remote.version });
|
||||
await this.applyUpdate(result.remote);
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('[SelfUpdater] Auto-update error:', e.message);
|
||||
log.error('updater', e, { phase: 'autoUpdate' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -606,7 +607,7 @@ class SelfUpdater extends EventEmitter {
|
||||
fs.mkdirSync(path.dirname(this.notifySecretFile), { recursive: true });
|
||||
fs.writeFileSync(this.notifySecretFile, `${secret}\n`, { mode: 0o600 });
|
||||
} catch (error) {
|
||||
console.warn('[SelfUpdater] Failed to persist notify secret:', error.message);
|
||||
log.warn('updater', 'Failed to persist notify secret', { error: error.message });
|
||||
}
|
||||
return secret;
|
||||
}
|
||||
@@ -626,7 +627,7 @@ class SelfUpdater extends EventEmitter {
|
||||
fs.mkdirSync(path.dirname(this.config.instanceIdFile), { recursive: true });
|
||||
fs.writeFileSync(this.config.instanceIdFile, `${instanceId}\n`, 'utf8');
|
||||
} catch (error) {
|
||||
console.warn('[SelfUpdater] Failed to persist instance ID:', error.message);
|
||||
log.warn('updater', 'Failed to persist instance ID', { error: error.message });
|
||||
}
|
||||
return instanceId;
|
||||
}
|
||||
@@ -644,7 +645,7 @@ class SelfUpdater extends EventEmitter {
|
||||
try {
|
||||
fs.writeFileSync(historyPath, JSON.stringify(history, null, 2));
|
||||
} catch (e) {
|
||||
console.error('[SelfUpdater] Failed to save history:', e.message);
|
||||
log.error('updater', e, { operation: 'saveHistory' });
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ const jwt = require('jsonwebtoken');
|
||||
const crypto = require('crypto');
|
||||
const credentialManager = require('./credential-manager');
|
||||
const cryptoUtils = require('../security/crypto-utils');
|
||||
const { log } = require('../utils/logging');
|
||||
|
||||
// JWT signing secret - derived from encryption key for consistency
|
||||
const JWT_SECRET = cryptoUtils.loadOrCreateKey();
|
||||
@@ -19,7 +20,7 @@ const API_KEY_METADATA_NAMESPACE = 'auth.metadata';
|
||||
class AuthManager {
|
||||
constructor() {
|
||||
this.keyMetadataCache = new Map(); // Cache for API key metadata
|
||||
console.log('[AuthManager] Initialized');
|
||||
log.info('auth', 'Initialized');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -44,10 +45,10 @@ class AuthManager {
|
||||
{ expiresIn }
|
||||
);
|
||||
|
||||
console.log(`[AuthManager] Generated JWT for user: ${payload.sub}, expires in: ${expiresIn}`);
|
||||
log.info('auth', 'Generated JWT', { user: payload.sub, expiresIn });
|
||||
return token;
|
||||
} catch (error) {
|
||||
console.error('[AuthManager] JWT generation failed:', error.message);
|
||||
log.error('auth', error, { operation: 'jwtGenerate' });
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -68,11 +69,11 @@ class AuthManager {
|
||||
};
|
||||
} catch (error) {
|
||||
if (error.name === 'TokenExpiredError') {
|
||||
console.log('[AuthManager] JWT token expired');
|
||||
log.info('auth', 'JWT token expired');
|
||||
} else if (error.name === 'JsonWebTokenError') {
|
||||
console.log('[AuthManager] JWT token invalid:', error.message);
|
||||
log.info('auth', 'JWT token invalid', { error: error.message });
|
||||
} else {
|
||||
console.error('[AuthManager] JWT verification failed:', error.message);
|
||||
log.error('auth', error, { operation: 'jwtVerify' });
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -116,7 +117,7 @@ class AuthManager {
|
||||
// Cache metadata
|
||||
this.keyMetadataCache.set(keyId, metadata);
|
||||
|
||||
console.log(`[AuthManager] Generated API key: ${name} (${keyId})`);
|
||||
log.info('auth', 'Generated API key', { name, keyId });
|
||||
|
||||
return {
|
||||
key: apiKey,
|
||||
@@ -126,7 +127,7 @@ class AuthManager {
|
||||
createdAt: metadata.createdAt
|
||||
};
|
||||
} catch (error) {
|
||||
console.error('[AuthManager] API key generation failed:', error.message);
|
||||
log.error('auth', error, { operation: 'apiKeyGenerate' });
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -154,30 +155,30 @@ class AuthManager {
|
||||
// Retrieve stored hash
|
||||
const storedHash = await credentialManager.retrieve(credentialKey);
|
||||
if (!storedHash) {
|
||||
console.log(`[AuthManager] API key not found: ${keyId}`);
|
||||
log.info('auth', 'API key not found', { keyId });
|
||||
return null;
|
||||
}
|
||||
|
||||
// Verify key matches stored hash
|
||||
const providedHash = crypto.createHash('sha256').update(key).digest('hex');
|
||||
if (!crypto.timingSafeEqual(Buffer.from(storedHash), Buffer.from(providedHash))) {
|
||||
console.log(`[AuthManager] API key hash mismatch: ${keyId}`);
|
||||
log.info('auth', 'API key hash mismatch', { keyId });
|
||||
return null;
|
||||
}
|
||||
|
||||
// Get metadata
|
||||
const metadata = await this.getKeyMetadata(keyId);
|
||||
if (!metadata) {
|
||||
console.log(`[AuthManager] API key metadata not found: ${keyId}`);
|
||||
log.info('auth', 'API key metadata not found', { keyId });
|
||||
return null;
|
||||
}
|
||||
|
||||
// Update last used timestamp (non-blocking)
|
||||
this.updateLastUsed(keyId, metadata).catch(err =>
|
||||
console.error(`[AuthManager] Failed to update lastUsed for ${keyId}:`, err.message)
|
||||
log.error('auth', err, { keyId, operation: 'updateLastUsed' })
|
||||
);
|
||||
|
||||
console.log(`[AuthManager] API key verified: ${metadata.name} (${keyId})`);
|
||||
log.info('auth', 'API key verified', { name: metadata.name, keyId });
|
||||
|
||||
return {
|
||||
keyId,
|
||||
@@ -185,7 +186,7 @@ class AuthManager {
|
||||
name: metadata.name
|
||||
};
|
||||
} catch (error) {
|
||||
console.error('[AuthManager] API key verification failed:', error.message);
|
||||
log.error('auth', error, { operation: 'apiKeyVerify' });
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -205,10 +206,10 @@ class AuthManager {
|
||||
|
||||
this.keyMetadataCache.delete(keyId);
|
||||
|
||||
console.log(`[AuthManager] Revoked API key: ${keyId}`);
|
||||
log.info('auth', 'Revoked API key', { keyId });
|
||||
return true;
|
||||
} catch (error) {
|
||||
console.error(`[AuthManager] Failed to revoke API key ${keyId}:`, error.message);
|
||||
log.error('auth', error, { keyId, operation: 'revoke' });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -233,7 +234,7 @@ class AuthManager {
|
||||
|
||||
return keys;
|
||||
} catch (error) {
|
||||
console.error('[AuthManager] Failed to list API keys:', error.message);
|
||||
log.error('auth', error, { operation: 'listApiKeys' });
|
||||
return [];
|
||||
}
|
||||
}
|
||||
@@ -262,7 +263,7 @@ class AuthManager {
|
||||
|
||||
return metadata;
|
||||
} catch (error) {
|
||||
console.error(`[AuthManager] Failed to get metadata for ${keyId}:`, error.message);
|
||||
log.error('auth', error, { keyId, operation: 'getMetadata' });
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -285,7 +286,7 @@ class AuthManager {
|
||||
|
||||
this.keyMetadataCache.set(keyId, updatedMetadata);
|
||||
} catch (error) {
|
||||
console.error(`[AuthManager] Failed to update lastUsed for ${keyId}:`, error.message);
|
||||
log.error('auth', error, { keyId, operation: 'updateLastUsed' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -294,7 +295,7 @@ class AuthManager {
|
||||
*/
|
||||
clearCache() {
|
||||
this.keyMetadataCache.clear();
|
||||
console.log('[AuthManager] Cache cleared');
|
||||
log.info('auth', 'Cache cleared');
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ const keychainManager = require('../security/keychain-manager');
|
||||
const cryptoUtils = require('../security/crypto-utils');
|
||||
const lockfile = require('proper-lockfile');
|
||||
const fs = require('fs');
|
||||
const { log } = require('../utils/logging');
|
||||
const path = require('path');
|
||||
const platformPaths = require('../../platform-paths');
|
||||
|
||||
@@ -33,7 +34,7 @@ class CredentialManager {
|
||||
stale: 30000
|
||||
};
|
||||
|
||||
console.log(`[CredentialManager] Initialized with ${this.useKeychain ? 'OS keychain' : 'encrypted file'} storage`);
|
||||
log.info('cred', 'Initialized', { storage: this.useKeychain ? 'keychain' : 'file' });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -60,19 +61,19 @@ class CredentialManager {
|
||||
// Store metadata separately in file
|
||||
await this.storeMetadata(key, metadata);
|
||||
this.cache.set(key, { value, exp: Date.now() + this.CACHE_TTL_MS });
|
||||
console.log(`[CredentialManager] Stored '${key}' in OS keychain`);
|
||||
log.info('cred', 'Stored credential in keychain', { key });
|
||||
return true;
|
||||
}
|
||||
console.warn(`[CredentialManager] Keychain storage failed for '${key}', falling back to encrypted file`);
|
||||
log.warn('cred', 'Keychain storage failed, falling back to encrypted file', { key });
|
||||
}
|
||||
|
||||
// Fallback to encrypted file storage
|
||||
await this.storeInFile(key, value, metadata);
|
||||
this.cache.set(key, { value, exp: Date.now() + this.CACHE_TTL_MS });
|
||||
console.log(`[CredentialManager] Stored '${key}' in encrypted file`);
|
||||
log.info('cred', 'Stored credential in encrypted file', { key });
|
||||
return true;
|
||||
} catch (error) {
|
||||
console.error(`[CredentialManager] Failed to store '${key}':`, error.message);
|
||||
log.error('cred', error, { key, operation: 'store' });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -109,7 +110,7 @@ class CredentialManager {
|
||||
}
|
||||
return value;
|
||||
} catch (error) {
|
||||
console.error(`[CredentialManager] Failed to retrieve '${key}':`, error.message);
|
||||
log.error('cred', error, { key, operation: 'retrieve' });
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -132,10 +133,10 @@ class CredentialManager {
|
||||
// Remove from file storage
|
||||
await this.deleteFromFile(key);
|
||||
|
||||
console.log(`[CredentialManager] Deleted '${key}'`);
|
||||
log.info('cred', 'Deleted credential', { key });
|
||||
return true;
|
||||
} catch (error) {
|
||||
console.error(`[CredentialManager] Failed to delete '${key}':`, error.message);
|
||||
log.error('cred', error, { key, operation: 'delete' });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -149,7 +150,7 @@ class CredentialManager {
|
||||
const credentials = await this.loadCredentialsFile();
|
||||
return Object.keys(credentials);
|
||||
} catch (error) {
|
||||
console.error('[CredentialManager] Failed to list credentials:', error.message);
|
||||
log.error('cred', error, { operation: 'list' });
|
||||
return [];
|
||||
}
|
||||
}
|
||||
@@ -175,7 +176,7 @@ class CredentialManager {
|
||||
async rotateEncryptionKey() {
|
||||
let release;
|
||||
try {
|
||||
console.log('[CredentialManager] Starting encryption key rotation...');
|
||||
log.info('cred', 'Starting encryption key rotation');
|
||||
|
||||
// Ensure file exists before locking
|
||||
this._ensureFileExists();
|
||||
@@ -186,7 +187,7 @@ class CredentialManager {
|
||||
const keys = Object.keys(credentials);
|
||||
|
||||
if (keys.length === 0) {
|
||||
console.log('[CredentialManager] No credentials to rotate');
|
||||
log.info('cred', 'No credentials to rotate');
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -219,10 +220,10 @@ class CredentialManager {
|
||||
// Clear cache to force reload
|
||||
this.cache.clear();
|
||||
|
||||
console.log(`[CredentialManager] Successfully rotated ${keys.length} credentials`);
|
||||
log.info('cred', 'Rotated credentials', { count: keys.length });
|
||||
return true;
|
||||
} catch (error) {
|
||||
console.error('[CredentialManager] Key rotation failed:', error.message);
|
||||
log.error('cred', error, { operation: 'rotate' });
|
||||
return false;
|
||||
} finally {
|
||||
if (release) {
|
||||
@@ -255,12 +256,12 @@ class CredentialManager {
|
||||
|
||||
if (migrated > 0) {
|
||||
this.cache.clear();
|
||||
console.log(`[CredentialManager] Migrated ${migrated} plaintext credentials to encrypted format`);
|
||||
log.info('cred', 'Migrated plaintext credentials', { count: migrated });
|
||||
}
|
||||
|
||||
return { migrated, skipped, total: migrated + skipped };
|
||||
} catch (error) {
|
||||
console.error('[CredentialManager] Migration failed:', error.message);
|
||||
log.error('cred', error, { operation: 'migrate' });
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -365,14 +366,11 @@ class CredentialManager {
|
||||
// Most common cause: the encryption key on disk is different from
|
||||
// the key that originally encrypted this entry (rotated by a
|
||||
// container recreate that didn't preserve CREDENTIALS_FILE env).
|
||||
console.warn(
|
||||
`[CredentialManager] '${key}' is present but cannot be decrypted ` +
|
||||
`(likely encryption-key mismatch): ${decryptErr.message}`
|
||||
);
|
||||
log.warn('cred', 'Credential present but cannot be decrypted (likely encryption-key mismatch)', { key, error: decryptErr.message });
|
||||
return { status: 'unreadable', value: null, error: decryptErr.message };
|
||||
}
|
||||
} catch (err) {
|
||||
console.error(`[CredentialManager] diagnose('${key}') failed:`, err.message);
|
||||
log.error('cred', err, { key, operation: 'diagnose' });
|
||||
return { status: 'malformed', value: null, error: err.message };
|
||||
}
|
||||
}
|
||||
@@ -404,7 +402,7 @@ class CredentialManager {
|
||||
const data = fs.readFileSync(CREDENTIALS_FILE, 'utf8');
|
||||
return JSON.parse(data);
|
||||
} catch (error) {
|
||||
console.error('[CredentialManager] Failed to load credentials file:', error.message);
|
||||
log.error('cred', error, { operation: 'loadFile' });
|
||||
return {};
|
||||
}
|
||||
}
|
||||
@@ -440,10 +438,10 @@ class CredentialManager {
|
||||
await this._lockedUpdate(() => backup.credentials);
|
||||
this.cache.clear();
|
||||
|
||||
console.log('[CredentialManager] Successfully imported backup');
|
||||
log.info('cred', 'Successfully imported backup');
|
||||
return true;
|
||||
} catch (error) {
|
||||
console.error('[CredentialManager] Failed to import backup:', error.message);
|
||||
log.error('cred', error, { operation: 'importBackup' });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,6 +8,7 @@ const fs = require('fs');
|
||||
const path = require('path');
|
||||
const lockfile = require('proper-lockfile');
|
||||
const platformPaths = require('../../platform-paths');
|
||||
const { log } = require('../utils/logging');
|
||||
|
||||
const LOCK_DIR = process.env.PORT_LOCK_DIR || path.join(platformPaths.dataDir, '.port-locks');
|
||||
const LOCK_TIMEOUT = 120000; // 2 minutes
|
||||
@@ -35,7 +36,7 @@ class PortLockManager {
|
||||
ensureLockDirectory() {
|
||||
if (!fs.existsSync(LOCK_DIR)) {
|
||||
fs.mkdirSync(LOCK_DIR, { recursive: true });
|
||||
console.log('[PortLockManager] Created lock directory:', LOCK_DIR);
|
||||
log.info('portlock', 'Created lock directory', { dir: LOCK_DIR });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -63,7 +64,7 @@ class PortLockManager {
|
||||
const releaseFunctions = [];
|
||||
|
||||
try {
|
||||
console.log(`[PortLockManager] Acquiring locks for ports: ${sortedPorts.join(', ')}`);
|
||||
log.info('portlock', 'Acquiring locks', { ports: sortedPorts });
|
||||
|
||||
// Acquire locks in sorted order to prevent deadlocks
|
||||
for (const port of sortedPorts) {
|
||||
@@ -83,7 +84,7 @@ class PortLockManager {
|
||||
acquiredLocks.push(port);
|
||||
releaseFunctions.push(release);
|
||||
|
||||
console.log(`[PortLockManager] Locked port ${port}`);
|
||||
log.info('portlock', 'Locked port', { port });
|
||||
}
|
||||
|
||||
// Store lock information
|
||||
@@ -93,18 +94,18 @@ class PortLockManager {
|
||||
timestamp: Date.now()
|
||||
});
|
||||
|
||||
console.log(`[PortLockManager] Successfully acquired all locks (ID: ${lockId})`);
|
||||
log.info('portlock', 'Acquired all locks', { lockId });
|
||||
return lockId;
|
||||
|
||||
} catch (error) {
|
||||
// Release any locks we managed to acquire
|
||||
console.error(`[PortLockManager] Failed to acquire all locks:`, error.message);
|
||||
log.error('portlock', error, { operation: 'acquire', lockId });
|
||||
|
||||
for (const release of releaseFunctions) {
|
||||
try {
|
||||
await release();
|
||||
} catch (releaseError) {
|
||||
console.error(`[PortLockManager] Error releasing lock during cleanup:`, releaseError.message);
|
||||
log.error('portlock', releaseError, { operation: 'releaseCleanup', lockId });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -120,11 +121,11 @@ class PortLockManager {
|
||||
const lockInfo = this.activeLocks.get(lockId);
|
||||
|
||||
if (!lockInfo) {
|
||||
console.warn(`[PortLockManager] Lock ID ${lockId} not found (may have been released already)`);
|
||||
log.warn('portlock', 'Lock ID not found', { lockId });
|
||||
return;
|
||||
}
|
||||
|
||||
console.log(`[PortLockManager] Releasing locks for ports: ${lockInfo.ports.join(', ')}`);
|
||||
log.info('portlock', 'Releasing locks', { lockId, ports: lockInfo.ports });
|
||||
|
||||
const errors = [];
|
||||
|
||||
@@ -133,16 +134,16 @@ class PortLockManager {
|
||||
await release();
|
||||
} catch (error) {
|
||||
errors.push(error.message);
|
||||
console.error(`[PortLockManager] Error releasing lock:`, error.message);
|
||||
log.error('portlock', error, { operation: 'release', lockId });
|
||||
}
|
||||
}
|
||||
|
||||
this.activeLocks.delete(lockId);
|
||||
|
||||
if (errors.length > 0) {
|
||||
console.warn(`[PortLockManager] Released locks with ${errors.length} errors`);
|
||||
log.warn('portlock', 'Released locks with errors', { lockId, errorCount: errors.length });
|
||||
} else {
|
||||
console.log(`[PortLockManager] Successfully released all locks (ID: ${lockId})`);
|
||||
log.info('portlock', 'Released all locks', { lockId });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -151,7 +152,7 @@ class PortLockManager {
|
||||
* Removes locks older than LOCK_STALE_THRESHOLD
|
||||
*/
|
||||
async cleanupStaleLocks() {
|
||||
console.log('[PortLockManager] Cleaning up stale locks...');
|
||||
log.info('portlock', 'Cleaning up stale locks');
|
||||
|
||||
this.ensureLockDirectory();
|
||||
|
||||
@@ -174,20 +175,20 @@ class PortLockManager {
|
||||
// Lock is stale or not locked, safe to remove
|
||||
fs.unlinkSync(lockFilePath);
|
||||
cleaned++;
|
||||
console.log(`[PortLockManager] Removed stale lock: ${file}`);
|
||||
log.info('portlock', 'Removed stale lock', { file });
|
||||
}
|
||||
} catch (error) {
|
||||
// File might not exist or might have been removed by another process
|
||||
if (error.code !== 'ENOENT') {
|
||||
errors++;
|
||||
console.warn(`[PortLockManager] Error checking lock ${file}:`, error.message);
|
||||
log.warn('portlock', 'Error checking lock', { file, error: error.message });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`[PortLockManager] Cleanup complete: ${cleaned} stale locks removed, ${errors} errors`);
|
||||
log.info('portlock', 'Cleanup complete', { cleaned, errors });
|
||||
} catch (error) {
|
||||
console.error('[PortLockManager] Error during cleanup:', error.message);
|
||||
log.error('portlock', error, { operation: 'cleanup' });
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -9,6 +9,7 @@ const EventEmitter = require('events');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const platformPaths = require('../../platform-paths');
|
||||
const { log } = require('../utils/logging');
|
||||
|
||||
const docker = new Docker();
|
||||
|
||||
@@ -59,17 +60,17 @@ class ResourceMonitor extends EventEmitter {
|
||||
*/
|
||||
start() {
|
||||
if (this.monitoring) {
|
||||
console.log('[ResourceMonitor] Already monitoring');
|
||||
log.info('monitor', 'Already monitoring');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[ResourceMonitor] Starting container monitoring');
|
||||
log.info('monitor', 'Starting container monitoring');
|
||||
this.monitoring = true;
|
||||
this.monitoringInterval = setInterval(() => this.collectStats(), MONITORING_INTERVAL);
|
||||
|
||||
// Hourly rollup — fires once an hour, computes the previous full hour
|
||||
this.hourlyRollupTimer = setInterval(() => {
|
||||
try { this.rollupHourly(); } catch (e) { console.error('[ResourceMonitor] hourly rollup error:', e.message); }
|
||||
try { this.rollupHourly(); } catch (e) { log.error('monitor', e, { rollup: 'hourly' }); }
|
||||
}, ROLLUP_HOURLY_INTERVAL);
|
||||
|
||||
// Daily rollup — schedule first run at the next midnight, then fire every 24h
|
||||
@@ -77,9 +78,9 @@ class ResourceMonitor extends EventEmitter {
|
||||
const nextMidnight = new Date(now.getFullYear(), now.getMonth(), now.getDate() + 1, 0, 0, 5);
|
||||
const msUntilMidnight = nextMidnight.getTime() - now.getTime();
|
||||
setTimeout(() => {
|
||||
try { this.rollupDaily(); } catch (e) { console.error('[ResourceMonitor] daily rollup error:', e.message); }
|
||||
try { this.rollupDaily(); } catch (e) { log.error('monitor', e, { rollup: 'daily' }); }
|
||||
this.dailyRollupTimer = setInterval(() => {
|
||||
try { this.rollupDaily(); } catch (e) { console.error('[ResourceMonitor] daily rollup error:', e.message); }
|
||||
try { this.rollupDaily(); } catch (e) { log.error('monitor', e, { rollup: 'daily' }); }
|
||||
}, ROLLUP_DAILY_INTERVAL);
|
||||
}, msUntilMidnight);
|
||||
|
||||
@@ -93,7 +94,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
stop() {
|
||||
if (!this.monitoring) return;
|
||||
|
||||
console.log('[ResourceMonitor] Stopping container monitoring');
|
||||
log.info('monitor', 'Stopping container monitoring');
|
||||
this.monitoring = false;
|
||||
|
||||
if (this.monitoringInterval) {
|
||||
@@ -131,7 +132,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
this.checkAlerts(containerInfo.Id, containerInfo.Names[0], stats);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`[ResourceMonitor] Error collecting stats for ${containerInfo.Names[0]}:`, error.message);
|
||||
log.error('monitor', error, { container: containerInfo.Names[0] });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -143,7 +144,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
this.saveStats();
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error collecting container stats:', error.message);
|
||||
log.error('monitor', error, { phase: 'collectStats' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -329,7 +330,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
// Send notification if manager is configured
|
||||
if (this.notificationManager) {
|
||||
this.notificationManager.sendAlert(alertPayload).catch(err => {
|
||||
console.error('[ResourceMonitor] Failed to send alert notification:', err.message);
|
||||
log.error('monitor', err, { phase: 'sendAlert' });
|
||||
});
|
||||
}
|
||||
|
||||
@@ -357,7 +358,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
*/
|
||||
async restartContainer(containerId, containerName, alerts) {
|
||||
try {
|
||||
console.log(`[ResourceMonitor] Auto-restarting ${containerName} due to alerts:`, alerts.map(a => a.type).join(', '));
|
||||
log.info('monitor', 'Auto-restarting container', { container: containerName, alerts: alerts.map(a => a.type) });
|
||||
|
||||
const container = docker.getContainer(containerId);
|
||||
await container.restart();
|
||||
@@ -377,11 +378,11 @@ class ResourceMonitor extends EventEmitter {
|
||||
timestamp: new Date().toISOString(),
|
||||
reason: alerts
|
||||
}).catch(err => {
|
||||
console.error('[ResourceMonitor] Failed to send auto-restart notification:', err.message);
|
||||
log.error('monitor', err, { phase: 'sendAutoRestart' });
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`[ResourceMonitor] Failed to restart ${containerName}:`, error.message);
|
||||
log.error('monitor', error, { container: containerName, phase: 'restart' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -390,7 +391,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
*/
|
||||
triggerWorkflows(eventType, eventData) {
|
||||
if (!this.workflowEngine) {
|
||||
console.log('[ResourceMonitor] Workflow engine not set, skipping workflow trigger');
|
||||
log.info('monitor', 'Workflow engine not set, skipping workflow trigger');
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -398,14 +399,14 @@ class ResourceMonitor extends EventEmitter {
|
||||
this.workflowEngine.triggerForEvent(eventType, eventData)
|
||||
.then(results => {
|
||||
if (results && results.length > 0) {
|
||||
console.log(`[ResourceMonitor] Triggered ${results.length} workflow(s) for ${eventType}`);
|
||||
log.info('monitor', `Triggered workflows for ${eventType}`, { count: results.length });
|
||||
}
|
||||
})
|
||||
.catch(err => {
|
||||
console.error('[ResourceMonitor] Workflow trigger error:', err.message);
|
||||
log.error('monitor', err, { phase: 'workflowTrigger' });
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error triggering workflows:', error.message);
|
||||
log.error('monitor', error, { phase: 'workflowTrigger' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -414,7 +415,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
*/
|
||||
setWorkflowEngine(workflowEngine) {
|
||||
this.workflowEngine = workflowEngine;
|
||||
console.log('[ResourceMonitor] Workflow engine configured');
|
||||
log.info('monitor', 'Workflow engine configured');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -562,10 +563,10 @@ class ResourceMonitor extends EventEmitter {
|
||||
if (fs.existsSync(ALERT_HISTORY_FILE)) {
|
||||
const data = JSON.parse(fs.readFileSync(ALERT_HISTORY_FILE, 'utf8'));
|
||||
this.alertHistory = Array.isArray(data) ? data : [];
|
||||
console.log(`[ResourceMonitor] Loaded ${this.alertHistory.length} alert history entries`);
|
||||
log.info('monitor', 'Loaded alert history', { count: this.alertHistory.length });
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error loading alert history:', error.message);
|
||||
log.error('monitor', error, { operation: 'loadAlertHistory' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -576,7 +577,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
try {
|
||||
fs.writeFileSync(ALERT_HISTORY_FILE, JSON.stringify(this.alertHistory, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error saving alert history:', error.message);
|
||||
log.error('monitor', error, { operation: 'saveAlertHistory' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -606,10 +607,10 @@ class ResourceMonitor extends EventEmitter {
|
||||
if (fs.existsSync(STATS_FILE)) {
|
||||
const data = JSON.parse(fs.readFileSync(STATS_FILE, 'utf8'));
|
||||
this.stats = new Map(Object.entries(data));
|
||||
console.log(`[ResourceMonitor] Loaded stats for ${this.stats.size} containers`);
|
||||
log.info('monitor', 'Loaded stats', { containerCount: this.stats.size });
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error loading stats:', error.message);
|
||||
log.error('monitor', error, { operation: 'loadStats' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -621,7 +622,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
const data = Object.fromEntries(this.stats);
|
||||
fs.writeFileSync(STATS_FILE, JSON.stringify(data, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error saving stats:', error.message);
|
||||
log.error('monitor', error, { operation: 'saveStats' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -633,10 +634,10 @@ class ResourceMonitor extends EventEmitter {
|
||||
if (fs.existsSync(ALERT_CONFIG_FILE)) {
|
||||
const data = JSON.parse(fs.readFileSync(ALERT_CONFIG_FILE, 'utf8'));
|
||||
this.alerts = new Map(Object.entries(data));
|
||||
console.log(`[ResourceMonitor] Loaded alert config for ${this.alerts.size} containers`);
|
||||
log.info('monitor', 'Loaded alert config', { containerCount: this.alerts.size });
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error loading alert config:', error.message);
|
||||
log.error('monitor', error, { operation: 'loadAlertConfig' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -648,7 +649,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
const data = Object.fromEntries(this.alerts);
|
||||
fs.writeFileSync(ALERT_CONFIG_FILE, JSON.stringify(data, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error saving alert config:', error.message);
|
||||
log.error('monitor', error, { operation: 'saveAlertConfig' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -902,10 +903,10 @@ class ResourceMonitor extends EventEmitter {
|
||||
if (fs.existsSync(STATS_HOURLY_FILE)) {
|
||||
const data = JSON.parse(fs.readFileSync(STATS_HOURLY_FILE, 'utf8'));
|
||||
this.hourlyHistory = new Map(Object.entries(data));
|
||||
console.log(`[ResourceMonitor] Loaded hourly rollups for ${this.hourlyHistory.size} containers`);
|
||||
log.info('monitor', 'Loaded hourly rollups', { containerCount: this.hourlyHistory.size });
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error loading hourly stats:', error.message);
|
||||
log.error('monitor', error, { operation: 'loadHourlyStats' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -917,7 +918,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
const data = Object.fromEntries(this.hourlyHistory);
|
||||
fs.writeFileSync(STATS_HOURLY_FILE, JSON.stringify(data, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error saving hourly stats:', error.message);
|
||||
log.error('monitor', error, { operation: 'saveHourlyStats' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -929,10 +930,10 @@ class ResourceMonitor extends EventEmitter {
|
||||
if (fs.existsSync(STATS_DAILY_FILE)) {
|
||||
const data = JSON.parse(fs.readFileSync(STATS_DAILY_FILE, 'utf8'));
|
||||
this.dailyHistory = new Map(Object.entries(data));
|
||||
console.log(`[ResourceMonitor] Loaded daily rollups for ${this.dailyHistory.size} containers`);
|
||||
log.info('monitor', 'Loaded daily rollups', { containerCount: this.dailyHistory.size });
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error loading daily stats:', error.message);
|
||||
log.error('monitor', error, { operation: 'loadDailyStats' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -944,7 +945,7 @@ class ResourceMonitor extends EventEmitter {
|
||||
const data = Object.fromEntries(this.dailyHistory);
|
||||
fs.writeFileSync(STATS_DAILY_FILE, JSON.stringify(data, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[ResourceMonitor] Error saving daily stats:', error.message);
|
||||
log.error('monitor', error, { operation: 'saveDailyStats' });
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ const fs = require('fs');
|
||||
const path = require('path');
|
||||
const https = require('https');
|
||||
const platformPaths = require('../../platform-paths');
|
||||
const { log } = require('../utils/logging');
|
||||
|
||||
const docker = new Docker();
|
||||
|
||||
@@ -33,7 +34,7 @@ class UpdateManager extends EventEmitter {
|
||||
start() {
|
||||
if (this.checking) return;
|
||||
|
||||
console.log('[UpdateManager] Starting update checks');
|
||||
log.info('update', 'Starting update checks');
|
||||
this.checking = true;
|
||||
|
||||
// Initial check
|
||||
@@ -52,7 +53,7 @@ class UpdateManager extends EventEmitter {
|
||||
stop() {
|
||||
if (!this.checking) return;
|
||||
|
||||
console.log('[UpdateManager] Stopping update checks');
|
||||
log.info('update', 'Stopping update checks');
|
||||
this.checking = false;
|
||||
|
||||
if (this.checkInterval) {
|
||||
@@ -70,7 +71,7 @@ class UpdateManager extends EventEmitter {
|
||||
*/
|
||||
triggerWorkflows(eventType, eventData) {
|
||||
if (!this.workflowEngine) {
|
||||
console.log('[UpdateManager] Workflow engine not set, skipping workflow trigger');
|
||||
log.info('update', 'Workflow engine not set, skipping workflow trigger');
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -78,14 +79,14 @@ class UpdateManager extends EventEmitter {
|
||||
this.workflowEngine.triggerForEvent(eventType, eventData)
|
||||
.then(results => {
|
||||
if (results && results.length > 0) {
|
||||
console.log(`[UpdateManager] Triggered ${results.length} workflow(s) for ${eventType}`);
|
||||
log.info('update', `Triggered workflows for ${eventType}`, { count: results.length });
|
||||
}
|
||||
})
|
||||
.catch(err => {
|
||||
console.error('[UpdateManager] Workflow trigger error:', err.message);
|
||||
log.error('update', err);
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('[UpdateManager] Error triggering workflows:', error.message);
|
||||
log.error('update', error);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -94,7 +95,7 @@ class UpdateManager extends EventEmitter {
|
||||
*/
|
||||
setWorkflowEngine(workflowEngine) {
|
||||
this.workflowEngine = workflowEngine;
|
||||
console.log('[UpdateManager] Workflow engine configured');
|
||||
log.info('update', 'Workflow engine configured');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -131,13 +132,13 @@ class UpdateManager extends EventEmitter {
|
||||
this.availableUpdates.delete(containerInfo.Id);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`[UpdateManager] Error checking ${containerInfo.Names[0]}:`, error.message);
|
||||
log.error('update', error, null, { containerName: containerInfo.Names[0] });
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`[UpdateManager] Found ${this.availableUpdates.size} updates available`);
|
||||
log.info('update', 'Checked for updates', { availableCount: this.availableUpdates.size });
|
||||
} catch (error) {
|
||||
console.error('[UpdateManager] Error checking for updates:', error.message);
|
||||
log.error('update', error);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -168,10 +169,10 @@ class UpdateManager extends EventEmitter {
|
||||
}
|
||||
|
||||
// gcr.io / quay.io / registry.gitlab.com — currently unsupported
|
||||
console.warn(`[UpdateManager] Custom registry not yet supported: ${remainder}`);
|
||||
log.warn('update', 'Custom registry not yet supported', { remainder });
|
||||
return null;
|
||||
} catch (error) {
|
||||
console.error(`[UpdateManager] Error getting digest for ${imageName}:`, error.message);
|
||||
log.error('update', error, null, { imageName });
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -338,7 +339,7 @@ class UpdateManager extends EventEmitter {
|
||||
async updateContainer(containerId, options = {}) {
|
||||
const startTime = Date.now();
|
||||
|
||||
console.log(`[UpdateManager] Starting update for container ${containerId}`);
|
||||
log.info('update', 'Starting update for container', { containerId });
|
||||
this.emit('update-start', { containerId, timestamp: new Date().toISOString() });
|
||||
|
||||
try {
|
||||
@@ -355,9 +356,9 @@ class UpdateManager extends EventEmitter {
|
||||
const oldImage = docker.getImage(oldImageId);
|
||||
const oldImageInspect = await oldImage.inspect();
|
||||
oldImageDigest = oldImageInspect.RepoDigests?.[0] || oldImageId;
|
||||
console.log(`[UpdateManager] Stored old image digest: ${oldImageDigest.substring(0, 40)}...`);
|
||||
log.info('update', 'Stored old image digest', { digestPrefix: oldImageDigest.substring(0, 40) });
|
||||
} catch (error) {
|
||||
console.warn(`[UpdateManager] Could not get old image digest: ${error.message}`);
|
||||
log.warn('update', 'Could not get old image digest', { error: error.message });
|
||||
}
|
||||
|
||||
// Create backup of current state
|
||||
@@ -380,19 +381,19 @@ class UpdateManager extends EventEmitter {
|
||||
this.triggerWorkflows('pre-update', { containerId, containerName, appId: containerName, imageName });
|
||||
|
||||
// Pull latest image
|
||||
console.log(`[UpdateManager] Pulling latest image: ${imageName}`);
|
||||
log.info('update', 'Pulling latest image', { imageName });
|
||||
await this.pullImage(imageName);
|
||||
|
||||
// Stop container
|
||||
console.log(`[UpdateManager] Stopping container: ${containerName}`);
|
||||
log.info('update', 'Stopping container', { containerName });
|
||||
await container.stop();
|
||||
|
||||
// Remove old container
|
||||
console.log(`[UpdateManager] Removing old container: ${containerName}`);
|
||||
log.info('update', 'Removing old container', { containerName });
|
||||
await container.remove();
|
||||
|
||||
// Create new container with same configuration
|
||||
console.log(`[UpdateManager] Creating new container: ${containerName}`);
|
||||
log.info('update', 'Creating new container', { containerName });
|
||||
const newContainer = await docker.createContainer({
|
||||
name: containerName,
|
||||
Image: imageName,
|
||||
@@ -401,11 +402,11 @@ class UpdateManager extends EventEmitter {
|
||||
});
|
||||
|
||||
// Start new container
|
||||
console.log(`[UpdateManager] Starting new container: ${containerName}`);
|
||||
log.info('update', 'Starting new container', { containerName });
|
||||
await newContainer.start();
|
||||
|
||||
// Extended verification with health checks and port accessibility
|
||||
console.log(`[UpdateManager] Performing extended verification...`);
|
||||
log.info('update', 'Performing extended verification');
|
||||
await this.verifyContainerExtended(newContainer, inspect, options.verifyTimeout || 60000);
|
||||
|
||||
// Get new image ID
|
||||
@@ -415,12 +416,12 @@ class UpdateManager extends EventEmitter {
|
||||
// Remove old image only after successful verification
|
||||
if (oldImageId !== newImageId) {
|
||||
try {
|
||||
console.log(`[UpdateManager] Removing old image: ${oldImageId.substring(0, 12)}`);
|
||||
log.info('update', 'Removing old image', { oldImageIdPrefix: oldImageId.substring(0, 12) });
|
||||
const oldImage = docker.getImage(oldImageId);
|
||||
await oldImage.remove({ force: false });
|
||||
console.log(`[UpdateManager] Old image removed successfully`);
|
||||
log.info('update', 'Old image removed successfully');
|
||||
} catch (error) {
|
||||
console.warn(`[UpdateManager] Could not remove old image (may be in use): ${error.message}`);
|
||||
log.warn('update', 'Could not remove old image (may be in use)', { error: error.message });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -442,7 +443,7 @@ class UpdateManager extends EventEmitter {
|
||||
this.availableUpdates.delete(containerId);
|
||||
|
||||
this.emit('update-complete', historyEntry);
|
||||
console.log(`[UpdateManager] Update completed in ${duration}ms`);
|
||||
log.info('update', 'Update completed', { durationMs: duration });
|
||||
|
||||
return historyEntry;
|
||||
} catch (error) {
|
||||
@@ -461,11 +462,11 @@ class UpdateManager extends EventEmitter {
|
||||
|
||||
// Attempt rollback
|
||||
if (options.autoRollback !== false) {
|
||||
console.log(`[UpdateManager] Attempting rollback for ${containerId}`);
|
||||
log.info('update', 'Attempting rollback', { containerId });
|
||||
try {
|
||||
await this.rollbackUpdate(containerId);
|
||||
} catch (rollbackError) {
|
||||
console.error(`[UpdateManager] Rollback failed:`, rollbackError.message);
|
||||
log.error('update', rollbackError);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -538,7 +539,7 @@ class UpdateManager extends EventEmitter {
|
||||
const maxAttempts = Math.floor(timeout / 2000); // Check every 2 seconds
|
||||
let lastError = null;
|
||||
|
||||
console.log(`[UpdateManager] Extended verification with ${maxAttempts} attempts over ${timeout/1000}s`);
|
||||
log.info('update', 'Extended verification', { maxAttempts, timeoutSec: timeout / 1000 });
|
||||
|
||||
for (let attempt = 0; attempt < maxAttempts; attempt++) {
|
||||
try {
|
||||
@@ -553,14 +554,14 @@ class UpdateManager extends EventEmitter {
|
||||
// Step 2: Check Docker health check if available
|
||||
if (inspect.State.Health) {
|
||||
if (inspect.State.Health.Status === 'healthy') {
|
||||
console.log(`[UpdateManager] Container health check: healthy`);
|
||||
log.info('update', 'Container health check: healthy');
|
||||
return true;
|
||||
} else if (inspect.State.Health.Status === 'unhealthy') {
|
||||
lastError = 'Container health check failed (unhealthy)';
|
||||
throw new Error(lastError);
|
||||
}
|
||||
// Status is 'starting' - continue waiting
|
||||
console.log(`[UpdateManager] Health check status: ${inspect.State.Health.Status} (attempt ${attempt + 1}/${maxAttempts})`);
|
||||
log.info('update', 'Health check status', { status: inspect.State.Health.Status, attempt: attempt + 1, maxAttempts });
|
||||
} else {
|
||||
// Step 3: No Docker health check - verify HTTP port accessibility
|
||||
const ports = this.extractPorts(inspect);
|
||||
@@ -578,22 +579,22 @@ class UpdateManager extends EventEmitter {
|
||||
|
||||
// Accept 2xx, 3xx, 4xx as "accessible" (server is responding)
|
||||
if (response.status >= 200 && response.status < 500) {
|
||||
console.log(`[UpdateManager] Port ${primaryPort.hostPort} is accessible (HTTP ${response.status})`);
|
||||
log.info('update', 'Port accessible', { hostPort: primaryPort.hostPort, httpStatus: response.status });
|
||||
|
||||
// Wait a bit more to ensure stability
|
||||
if (attempt >= 2) {
|
||||
console.log(`[UpdateManager] Container verified successfully`);
|
||||
log.info('update', 'Container verified successfully');
|
||||
return true;
|
||||
}
|
||||
}
|
||||
} catch (fetchError) {
|
||||
lastError = `Port ${primaryPort.hostPort} not accessible: ${fetchError.message}`;
|
||||
console.log(`[UpdateManager] ${lastError} (attempt ${attempt + 1}/${maxAttempts})`);
|
||||
log.info('update', lastError, { attempt: attempt + 1, maxAttempts });
|
||||
}
|
||||
} else {
|
||||
// No ports exposed - just verify it's running for a few cycles
|
||||
if (attempt >= 5) {
|
||||
console.log(`[UpdateManager] Container running without exposed ports (verified)`);
|
||||
log.info('update', 'Container running without exposed ports (verified)');
|
||||
return true;
|
||||
}
|
||||
}
|
||||
@@ -605,7 +606,7 @@ class UpdateManager extends EventEmitter {
|
||||
}
|
||||
} catch (error) {
|
||||
lastError = error.message;
|
||||
console.log(`[UpdateManager] Verification attempt ${attempt + 1} failed: ${lastError}`);
|
||||
log.info('update', 'Verification attempt failed', { attempt: attempt + 1, error: lastError });
|
||||
|
||||
if (attempt < maxAttempts - 1) {
|
||||
await new Promise(resolve => setTimeout(resolve, 2000));
|
||||
@@ -649,7 +650,7 @@ class UpdateManager extends EventEmitter {
|
||||
* Rollback to previous version
|
||||
*/
|
||||
async rollbackUpdate(containerId) {
|
||||
console.log(`[UpdateManager] Rolling back container ${containerId}`);
|
||||
log.info('update', 'Rolling back container', { containerId });
|
||||
|
||||
// Find last successful update in history
|
||||
const lastUpdate = this.history
|
||||
@@ -682,12 +683,12 @@ class UpdateManager extends EventEmitter {
|
||||
|
||||
await newContainer.start();
|
||||
|
||||
console.log(`[UpdateManager] Rollback completed for ${backup.containerName}`);
|
||||
log.info('update', 'Rollback completed', { containerName: backup.containerName });
|
||||
this.emit('rollback-complete', { containerId, containerName: backup.containerName });
|
||||
|
||||
return true;
|
||||
} catch (error) {
|
||||
console.error(`[UpdateManager] Rollback failed:`, error.message);
|
||||
log.error('update', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
@@ -704,11 +705,11 @@ class UpdateManager extends EventEmitter {
|
||||
|
||||
setTimeout(() => {
|
||||
this.updateContainer(containerId).catch(error => {
|
||||
console.error(`[UpdateManager] Scheduled update failed:`, error.message);
|
||||
log.error('update', error);
|
||||
});
|
||||
}, delay);
|
||||
|
||||
console.log(`[UpdateManager] Update scheduled for ${containerId} at ${scheduledTime}`);
|
||||
log.info('update', 'Update scheduled', { containerId, scheduledTime });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -784,7 +785,7 @@ class UpdateManager extends EventEmitter {
|
||||
changelog: this.formatChangelog(repoInfo, tags, imageTag)
|
||||
};
|
||||
} catch (error) {
|
||||
console.error(`[UpdateManager] Error fetching changelog for ${imageName}:`, error.message);
|
||||
log.error('update', error, null, { imageName });
|
||||
|
||||
// Return basic info even on error
|
||||
const [fullRepo] = imageName.split(':');
|
||||
@@ -940,7 +941,7 @@ class UpdateManager extends EventEmitter {
|
||||
|
||||
const count = Object.values(this.config.autoUpdate || {}).filter(c => c.enabled).length;
|
||||
if (count > 0) {
|
||||
console.log(`[UpdateManager] Auto-update scheduler started (${count} container(s) configured)`);
|
||||
log.info('update', 'Auto-update scheduler started', { containerCount: count });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -989,17 +990,17 @@ class UpdateManager extends EventEmitter {
|
||||
const update = this.availableUpdates.get(containerId);
|
||||
if (!update) continue;
|
||||
|
||||
console.log(`[UpdateManager] Auto-updating ${update.containerName} (schedule: ${cfg.schedule})`);
|
||||
log.info('update', 'Auto-updating container', { containerName: update.containerName, schedule: cfg.schedule });
|
||||
this.emit('auto-update-start', { containerId, containerName: update.containerName, schedule: cfg.schedule });
|
||||
|
||||
try {
|
||||
const result = await this.updateContainer(containerId, { autoRollback: cfg.autoRollback !== false });
|
||||
cfg.lastAutoUpdate = now.toISOString();
|
||||
this.saveConfig();
|
||||
console.log(`[UpdateManager] Auto-update completed for ${update.containerName}`);
|
||||
log.info('update', 'Auto-update completed', { containerName: update.containerName });
|
||||
this.emit('auto-update-complete', { containerId, containerName: update.containerName, result });
|
||||
} catch (error) {
|
||||
console.error(`[UpdateManager] Auto-update failed for ${update.containerName}:`, error.message);
|
||||
log.error('update', error, null, { containerName: update.containerName });
|
||||
cfg.lastAutoUpdate = now.toISOString(); // Don't retry same day
|
||||
this.saveConfig();
|
||||
this.emit('auto-update-failed', { containerId, containerName: update.containerName, error: error.message });
|
||||
@@ -1056,7 +1057,7 @@ class UpdateManager extends EventEmitter {
|
||||
return JSON.parse(fs.readFileSync(UPDATE_CONFIG_FILE, 'utf8'));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[UpdateManager] Error loading config:', error.message);
|
||||
log.error('update', error);
|
||||
}
|
||||
return { autoUpdate: {} };
|
||||
}
|
||||
@@ -1068,7 +1069,7 @@ class UpdateManager extends EventEmitter {
|
||||
try {
|
||||
fs.writeFileSync(UPDATE_CONFIG_FILE, JSON.stringify(this.config, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[UpdateManager] Error saving config:', error.message);
|
||||
log.error('update', error);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1081,7 +1082,7 @@ class UpdateManager extends EventEmitter {
|
||||
return JSON.parse(fs.readFileSync(UPDATE_HISTORY_FILE, 'utf8'));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[UpdateManager] Error loading history:', error.message);
|
||||
log.error('update', error);
|
||||
}
|
||||
return [];
|
||||
}
|
||||
@@ -1093,7 +1094,7 @@ class UpdateManager extends EventEmitter {
|
||||
try {
|
||||
fs.writeFileSync(UPDATE_HISTORY_FILE, JSON.stringify(this.history, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[UpdateManager] Error saving history:', error.message);
|
||||
log.error('update', error);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,392 @@
|
||||
/**
|
||||
* Disk Space Monitor
|
||||
*
|
||||
* Tracks Docker + system disk usage against a user-configured budget.
|
||||
* When usage exceeds thresholds, triggers automatic cleanup and notifications.
|
||||
*
|
||||
* Key concepts:
|
||||
* - diskBudgetGB: How much disk the user is willing to give DashCaddy (default 10)
|
||||
* - The monitor calculates Docker's footprint (images, volumes, containers, build cache)
|
||||
* - Breakdown shows where space goes so users can make informed decisions
|
||||
* - Auto-cleanup triggers at 80% (warning), 90% (aggressive), 95% (critical)
|
||||
*/
|
||||
|
||||
const EventEmitter = require('events');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { execFile } = require('child_process');
|
||||
const { promisify } = require('util');
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
const DEFAULT_BUDGET_GB = 10;
|
||||
const DEFAULT_CONFIG = {
|
||||
enabled: true,
|
||||
diskBudgetGB: DEFAULT_BUDGET_GB,
|
||||
warningThresholdPct: 80,
|
||||
criticalThresholdPct: 90,
|
||||
autoCleanup: true,
|
||||
cleanupAggressivePct: 95,
|
||||
};
|
||||
|
||||
class DiskSpaceMonitor extends EventEmitter {
|
||||
constructor({ log, config }) {
|
||||
super();
|
||||
this.log = log;
|
||||
this.config = config;
|
||||
this.lastSnapshot = null;
|
||||
this.lastCleanup = null;
|
||||
this.intervalHandle = null;
|
||||
this.diskConfig = { ...DEFAULT_CONFIG };
|
||||
this._loadConfig();
|
||||
}
|
||||
|
||||
/**
|
||||
* Load disk budget config from the site config file
|
||||
* Stored under `diskSpace` key in config.json
|
||||
*/
|
||||
_loadConfig() {
|
||||
try {
|
||||
const raw = this.config?.diskSpace;
|
||||
if (raw) {
|
||||
this.diskConfig = {
|
||||
...DEFAULT_CONFIG,
|
||||
...raw,
|
||||
};
|
||||
}
|
||||
} catch {
|
||||
// Use defaults
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Update disk space settings
|
||||
*/
|
||||
configure(updates) {
|
||||
const prev = { ...this.diskConfig };
|
||||
this.diskConfig = { ...this.diskConfig, ...updates };
|
||||
this._persistConfig();
|
||||
this.emit('config-changed', { prev, current: this.diskConfig });
|
||||
return this.diskConfig;
|
||||
}
|
||||
|
||||
_persistConfig() {
|
||||
// The config is persisted by the caller (settings route) which merges
|
||||
// into config.json. We just expose the current state.
|
||||
if (this.config) {
|
||||
this.config.diskSpace = this.diskConfig;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get a disk usage snapshot using `df` and `docker system df -v`
|
||||
*/
|
||||
async getSnapshot() {
|
||||
const [diskInfo, dockerInfo] = await Promise.all([
|
||||
this._getDiskInfo(),
|
||||
this._getDockerInfo(),
|
||||
]);
|
||||
|
||||
const snapshot = {
|
||||
timestamp: new Date().toISOString(),
|
||||
system: diskInfo,
|
||||
docker: dockerInfo,
|
||||
budget: {
|
||||
configuredGB: this.diskConfig.diskBudgetGB,
|
||||
dockerUsageGB: dockerInfo.totalGB,
|
||||
remainingBudgetGB: Math.max(0, this.diskConfig.diskBudgetGB - dockerInfo.totalGB),
|
||||
budgetUsedPct: Math.min(100, Math.round((dockerInfo.totalGB / this.diskConfig.diskBudgetGB) * 100)),
|
||||
status: this._getBudgetStatus(dockerInfo.totalGB),
|
||||
},
|
||||
config: { ...this.diskConfig },
|
||||
lastCleanup: this.lastCleanup,
|
||||
};
|
||||
|
||||
this.lastSnapshot = snapshot;
|
||||
|
||||
// Check thresholds and emit events
|
||||
this._checkThresholds(snapshot);
|
||||
|
||||
return snapshot;
|
||||
}
|
||||
|
||||
_getBudgetStatus(dockerUsageGB) {
|
||||
const pct = (dockerUsageGB / this.diskConfig.diskBudgetGB) * 100;
|
||||
if (pct >= this.diskConfig.cleanupAggressivePct) return 'critical';
|
||||
if (pct >= this.diskConfig.criticalThresholdPct) return 'aggressive';
|
||||
if (pct >= this.diskConfig.warningThresholdPct) return 'warning';
|
||||
return 'healthy';
|
||||
}
|
||||
|
||||
_checkThresholds(snapshot) {
|
||||
const { status, budgetUsedPct } = snapshot.budget;
|
||||
if (status === 'critical' || status === 'aggressive') {
|
||||
this.emit('budget-exceeded', snapshot);
|
||||
if (this.diskConfig.autoCleanup) {
|
||||
this.performCleanup(status === 'critical' ? 'aggressive' : 'standard').catch(() => {});
|
||||
}
|
||||
} else if (status === 'warning') {
|
||||
this.emit('budget-warning', snapshot);
|
||||
}
|
||||
}
|
||||
|
||||
async _getDiskInfo() {
|
||||
try {
|
||||
const { stdout } = await execFileAsync('df', ['-B1', '/']);
|
||||
const lines = stdout.trim().split('\n');
|
||||
const parts = lines[1].split(/\s+/);
|
||||
return {
|
||||
totalBytes: parseInt(parts[1], 10),
|
||||
usedBytes: parseInt(parts[2], 10),
|
||||
availableBytes: parseInt(parts[3], 10),
|
||||
usedPct: parseInt(parts[4], 10),
|
||||
mount: parts[5],
|
||||
totalGB: Math.round(parseInt(parts[1], 10) / 1073741824 * 10) / 10,
|
||||
usedGB: Math.round(parseInt(parts[2], 10) / 1073741824 * 10) / 10,
|
||||
availableGB: Math.round(parseInt(parts[3], 10) / 1073741824 * 10) / 10,
|
||||
};
|
||||
} catch {
|
||||
return { totalBytes: 0, usedBytes: 0, availableBytes: 0, usedPct: 0, totalGB: 0, usedGB: 0, availableGB: 0 };
|
||||
}
|
||||
}
|
||||
|
||||
async _getDockerInfo() {
|
||||
try {
|
||||
const { stdout } = await execFileAsync('docker', ['system', 'df', '--format', '{{json .}}']);
|
||||
const lines = stdout.trim().split('\n').filter(Boolean);
|
||||
|
||||
let images = { count: 0, totalGB: 0, reclaimableGB: 0 };
|
||||
let containers = { count: 0, totalGB: 0, reclaimableGB: 0 };
|
||||
let volumes = { count: 0, totalGB: 0, reclaimableGB: 0 };
|
||||
let buildCache = { count: 0, totalGB: 0, reclaimableGB: 0 };
|
||||
|
||||
for (const line of lines) {
|
||||
try {
|
||||
const d = JSON.parse(line);
|
||||
const type = d.Type?.toLowerCase() || '';
|
||||
const sizeGB = this._parseSizeToGB(d.Size);
|
||||
const reclaimGB = this._parseSizeToGB(d.Reclaimable);
|
||||
|
||||
if (type === 'images') images = { count: parseInt(d.TotalCount, 10) || 0, totalGB: sizeGB, reclaimableGB: reclaimGB };
|
||||
else if (type === 'containers') containers = { count: parseInt(d.TotalCount, 10) || 0, totalGB: sizeGB, reclaimableGB: reclaimGB };
|
||||
else if (type === 'local volumes') volumes = { count: parseInt(d.TotalCount, 10) || 0, totalGB: sizeGB, reclaimableGB: reclaimGB };
|
||||
else if (type === 'build cache') buildCache = { count: parseInt(d.TotalCount, 10) || 0, totalGB: sizeGB, reclaimableGB: reclaimGB };
|
||||
} catch { /* skip unparseable lines */ }
|
||||
}
|
||||
|
||||
const totalGB = Math.round((images.totalGB + containers.totalGB + volumes.totalGB + buildCache.totalGB) * 100) / 100;
|
||||
const reclaimableGB = Math.round((images.reclaimableGB + containers.reclaimableGB + volumes.reclaimableGB + buildCache.reclaimableGB) * 100) / 100;
|
||||
|
||||
return {
|
||||
images,
|
||||
containers,
|
||||
volumes,
|
||||
buildCache,
|
||||
totalGB,
|
||||
reclaimableGB,
|
||||
};
|
||||
} catch {
|
||||
return { images: {}, containers: {}, volumes: {}, buildCache: {}, totalGB: 0, reclaimableGB: 0 };
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse Docker's human-readable size strings (e.g., "2.519GB", "8.108MB", "0B")
|
||||
*/
|
||||
_parseSizeToGB(str) {
|
||||
if (!str || str === '0B') return 0;
|
||||
const match = str.match(/^([\d.]+)(B|KB|MB|GB|TB)$/i);
|
||||
if (!match) return 0;
|
||||
const value = parseFloat(match[1]);
|
||||
const unit = match[2].toUpperCase();
|
||||
const multipliers = { B: 1e-9, KB: 1e-6, MB: 1e-3, GB: 1, TB: 1e3 };
|
||||
return Math.round(value * (multipliers[unit] || 0) * 1000) / 1000;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get per-container log file sizes (the hidden disk hog)
|
||||
*/
|
||||
async _getContainerLogs() {
|
||||
try {
|
||||
const { stdout } = await execFileAsync('sh', ['-c', 'for f in /var/lib/docker/containers/*/*-json.log; do [ -f "$f" ] && stat -c "%s %n" "$f"; done 2>/dev/null | sort -rn | head -10']);
|
||||
const entries = [];
|
||||
for (const line of stdout.trim().split('\n').filter(Boolean)) {
|
||||
const [sizeStr, ...fileParts] = line.split(' ');
|
||||
const sizeBytes = parseInt(sizeStr, 10);
|
||||
entries.push({
|
||||
sizeBytes,
|
||||
sizeMB: Math.round(sizeBytes / 1048576 * 10) / 10,
|
||||
file: fileParts.join(' '),
|
||||
});
|
||||
}
|
||||
return entries;
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Perform cleanup
|
||||
* @param {string} level - 'standard' | 'aggressive' | 'logs-only'
|
||||
* @returns {Object} cleanup result with bytes reclaimed
|
||||
*/
|
||||
async performCleanup(level = 'standard') {
|
||||
const startTime = Date.now();
|
||||
const result = {
|
||||
level,
|
||||
startedAt: new Date(startTime).toISOString(),
|
||||
actions: [],
|
||||
bytesReclaimed: 0,
|
||||
};
|
||||
|
||||
try {
|
||||
// Always: truncate oversized container logs
|
||||
const logsBefore = await this._getContainerLogs();
|
||||
let logBytesFreed = 0;
|
||||
for (const log of logsBefore) {
|
||||
if (log.sizeBytes > 100 * 1048576) { // > 100MB
|
||||
try {
|
||||
await execFileAsync('truncate', ['-s', '0', log.file]);
|
||||
logBytesFreed += log.sizeBytes;
|
||||
result.actions.push({ action: 'truncate-log', file: log.file, freedBytes: log.sizeBytes });
|
||||
} catch { /* skip */ }
|
||||
}
|
||||
}
|
||||
result.bytesReclaimed += logBytesFreed;
|
||||
|
||||
// Always: vacuum journald to 200MB
|
||||
try {
|
||||
const { stdout } = await execFileAsync('journalctl', ['--vacuum-size=200M']);
|
||||
const freedMatch = stdout.match(/freed ([\d.]+[KMGT]?B)/i);
|
||||
if (freedMatch) {
|
||||
const freedBytes = this._humanToBytes(freedMatch[1]);
|
||||
result.bytesReclaimed += freedBytes;
|
||||
result.actions.push({ action: 'vacuum-journal', freedBytes, freedHuman: freedMatch[1] });
|
||||
}
|
||||
} catch { /* skip */ }
|
||||
|
||||
if (level === 'standard' || level === 'aggressive') {
|
||||
// Prune dangling images
|
||||
try {
|
||||
const { stdout } = await execFileAsync('docker', ['image', 'prune', '-f', '--filter', 'dangling=true']);
|
||||
const reclaimed = this._extractDockerReclaimed(stdout);
|
||||
result.bytesReclaimed += reclaimed;
|
||||
result.actions.push({ action: 'prune-dangling-images', freedBytes: reclaimed });
|
||||
} catch { /* skip */ }
|
||||
|
||||
// Prune unused volumes
|
||||
try {
|
||||
const { stdout } = await execFileAsync('docker', ['volume', 'prune', '-f']);
|
||||
const reclaimed = this._extractDockerReclaimed(stdout);
|
||||
result.bytesReclaimed += reclaimed;
|
||||
result.actions.push({ action: 'prune-unused-volumes', freedBytes: reclaimed });
|
||||
} catch { /* skip */ }
|
||||
|
||||
// Prune build cache (keep last 500MB)
|
||||
try {
|
||||
const { stdout } = await execFileAsync('docker', ['builder', 'prune', '-f', '--keep-storage', '500m']);
|
||||
const reclaimed = this._extractDockerReclaimed(stdout);
|
||||
result.bytesReclaimed += reclaimed;
|
||||
result.actions.push({ action: 'prune-build-cache', freedBytes: reclaimed });
|
||||
} catch { /* skip */ }
|
||||
}
|
||||
|
||||
if (level === 'aggressive') {
|
||||
// Remove ALL images not used by running containers
|
||||
try {
|
||||
const { stdout } = await execFileAsync('docker', ['image', 'prune', '-a', '-f']);
|
||||
const reclaimed = this._extractDockerReclaimed(stdout);
|
||||
result.bytesReclaimed += reclaimed;
|
||||
result.actions.push({ action: 'prune-all-unused-images', freedBytes: reclaimed });
|
||||
} catch { /* skip */ }
|
||||
|
||||
// Prune stopped containers older than 24h
|
||||
try {
|
||||
const { stdout } = await execFileAsync('docker', ['container', 'prune', '-f', '--filter', 'until=24h']);
|
||||
const reclaimed = this._extractDockerReclaimed(stdout);
|
||||
result.bytesReclaimed += reclaimed;
|
||||
result.actions.push({ action: 'prune-old-containers', freedBytes: reclaimed });
|
||||
} catch { /* skip */ }
|
||||
}
|
||||
|
||||
result.completedAt = new Date().toISOString();
|
||||
result.durationMs = Date.now() - startTime;
|
||||
result.bytesReclaimedGB = Math.round(result.bytesReclaimed / 1073741824 * 100) / 100;
|
||||
|
||||
this.lastCleanup = result;
|
||||
this.emit('cleanup-complete', result);
|
||||
|
||||
if (this.log) {
|
||||
this.log.info('disk', 'Disk cleanup completed', {
|
||||
level,
|
||||
bytesReclaimed: result.bytesReclaimed,
|
||||
GBReclaimed: result.bytesReclaimedGB,
|
||||
durationMs: result.durationMs,
|
||||
actions: result.actions.length,
|
||||
});
|
||||
}
|
||||
|
||||
return result;
|
||||
} catch (err) {
|
||||
result.error = err.message;
|
||||
result.completedAt = new Date().toISOString();
|
||||
if (this.log) {
|
||||
this.log.error('disk', 'Disk cleanup failed', { error: err.message, level });
|
||||
}
|
||||
return result;
|
||||
}
|
||||
}
|
||||
|
||||
_humanToBytes(str) {
|
||||
const match = str.match(/^([\d.]+)(B|KB|MB|GB|TB)$/i);
|
||||
if (!match) return 0;
|
||||
const value = parseFloat(match[1]);
|
||||
const unit = match[2].toUpperCase();
|
||||
const multipliers = { B: 1, KB: 1024, MB: 1048576, GB: 1073741824, TB: 1099511627776 };
|
||||
return Math.round(value * (multipliers[unit] || 0));
|
||||
}
|
||||
|
||||
_extractDockerReclaimed(stdout) {
|
||||
const match = stdout.match(/reclaimed\s+([\d.]+[KMGT]?B)/i) || stdout.match(/Total reclaimed space:\s*([\d.]+[KMGT]?B)/i);
|
||||
if (match) return this._humanToBytes(match[1]);
|
||||
return 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* Start periodic monitoring
|
||||
* @param {number} intervalMs - check interval (default 10 minutes)
|
||||
*/
|
||||
start(intervalMs = 600000) {
|
||||
if (this.intervalHandle) return;
|
||||
this.log?.info?.('disk', 'Disk space monitor started', { intervalMs });
|
||||
// Initial check
|
||||
this.getSnapshot().catch(() => {});
|
||||
this.intervalHandle = setInterval(() => {
|
||||
this.getSnapshot().catch(() => {});
|
||||
}, intervalMs);
|
||||
}
|
||||
|
||||
stop() {
|
||||
if (this.intervalHandle) {
|
||||
clearInterval(this.intervalHandle);
|
||||
this.intervalHandle = null;
|
||||
}
|
||||
}
|
||||
|
||||
getConfig() {
|
||||
return { ...this.diskConfig };
|
||||
}
|
||||
|
||||
async getDetailedBreakdown() {
|
||||
const [snapshot, containerLogs] = await Promise.all([
|
||||
this.getSnapshot(),
|
||||
this._getContainerLogs(),
|
||||
]);
|
||||
return {
|
||||
...snapshot,
|
||||
containerLogs,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = { DiskSpaceMonitor, DEFAULT_DISK_CONFIG: DEFAULT_CONFIG };
|
||||
@@ -6,6 +6,7 @@
|
||||
|
||||
const https = require('https');
|
||||
const http = require('http');
|
||||
const crypto = require('crypto');
|
||||
const EventEmitter = require('events');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
@@ -349,7 +350,7 @@ class HealthChecker extends EventEmitter {
|
||||
|
||||
// Create new incident
|
||||
const incident = {
|
||||
id: `incident-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`,
|
||||
id: `incident-${crypto.randomUUID()}`,
|
||||
serviceId,
|
||||
type,
|
||||
message,
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
const EventEmitter = require('events');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const { log } = require('../utils/logging');
|
||||
const platformPaths = require('../../platform-paths');
|
||||
|
||||
const WORKFLOWS_FILE = process.env.WORKFLOWS_FILE || path.join(platformPaths.dataDir, 'workflows-config.json');
|
||||
@@ -102,7 +103,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
this.enabled = new Map(Object.entries(data.enabled || {}));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[WorkflowEngine] Error loading config:', error.message);
|
||||
log.error('workflow', error, { operation: 'loadConfig' });
|
||||
}
|
||||
|
||||
// Default all workflows to enabled if not explicitly set
|
||||
@@ -123,7 +124,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
};
|
||||
fs.writeFileSync(WORKFLOWS_FILE, JSON.stringify(data, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[WorkflowEngine] Error saving config:', error.message);
|
||||
log.error('workflow', error, { operation: 'saveConfig' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -136,7 +137,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
this.history = JSON.parse(fs.readFileSync(WORKFLOW_HISTORY_FILE, 'utf8'));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[WorkflowEngine] Error loading history:', error.message);
|
||||
log.error('workflow', error, { operation: 'loadHistory' });
|
||||
this.history = [];
|
||||
}
|
||||
}
|
||||
@@ -148,7 +149,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
try {
|
||||
fs.writeFileSync(WORKFLOW_HISTORY_FILE, JSON.stringify(this.history, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[WorkflowEngine] Error saving history:', error.message);
|
||||
log.error('workflow', error, { operation: 'saveHistory' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -174,11 +175,11 @@ class WorkflowEngine extends EventEmitter {
|
||||
|
||||
const job = setInterval(() => {
|
||||
this.executeWorkflow(workflowId, { trigger: 'scheduled', timestamp: new Date().toISOString() })
|
||||
.catch(err => console.error(`[WorkflowEngine] Scheduled workflow ${workflowId} failed:`, err.message));
|
||||
.catch(err => log.error('workflow', err, { workflowId, phase: 'scheduled' }));
|
||||
}, workflow.interval);
|
||||
|
||||
this.scheduledJobs.set(workflowId, job);
|
||||
console.log(`[WorkflowEngine] Scheduled workflow '${workflowId}' every ${workflow.interval}ms`);
|
||||
log.info('workflow', 'Scheduled workflow', { workflowId, intervalMs: workflow.interval });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -201,14 +202,14 @@ class WorkflowEngine extends EventEmitter {
|
||||
}
|
||||
|
||||
if (!this.enabled.get(workflowId)) {
|
||||
console.log(`[WorkflowEngine] Workflow ${workflowId} is disabled, skipping`);
|
||||
log.info('workflow', 'Workflow disabled, skipping', { workflowId });
|
||||
return { skipped: true, reason: 'disabled' };
|
||||
}
|
||||
|
||||
const executionId = `${workflowId}-${Date.now()}`;
|
||||
const startTime = Date.now();
|
||||
|
||||
console.log(`[WorkflowEngine] Executing workflow: ${workflowId}`);
|
||||
log.info('workflow', 'Executing workflow', { workflowId });
|
||||
this.emit('workflow-start', { workflowId, executionId, triggerData });
|
||||
|
||||
const results = await this._runActions(workflow.actions, triggerData);
|
||||
@@ -237,7 +238,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
this.saveHistory();
|
||||
|
||||
this.emit('workflow-complete', historyEntry);
|
||||
console.log(`[WorkflowEngine] Workflow ${workflowId} completed in ${duration}ms, success: ${allSucceeded}`);
|
||||
log.info('workflow', 'Workflow completed', { workflowId, durationMs: duration, success: allSucceeded });
|
||||
|
||||
return historyEntry;
|
||||
}
|
||||
@@ -269,7 +270,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
const result = await this.executeAction(action, actionContext);
|
||||
results.push({ action: action.type, success: true, result });
|
||||
} catch (error) {
|
||||
console.error(`[WorkflowEngine] Action ${action.type} failed:`, error.message);
|
||||
log.error('workflow', error, { actionType: action.type });
|
||||
results.push({
|
||||
action: action.type,
|
||||
success: false,
|
||||
@@ -322,7 +323,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
return this.collectMetrics(context.containerId, action.period);
|
||||
|
||||
default:
|
||||
console.warn(`[WorkflowEngine] Unknown action type: ${action.type}`);
|
||||
log.warn('workflow', 'Unknown action type', { actionType: action.type });
|
||||
return { skipped: true, reason: `Unknown action type: ${action.type}` };
|
||||
}
|
||||
}
|
||||
@@ -428,7 +429,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
throw new Error('Container ID not provided');
|
||||
}
|
||||
|
||||
console.log(`[WorkflowEngine] Restarting container: ${containerId}`);
|
||||
log.info('workflow', 'Restarting container', { containerId });
|
||||
const container = docker.getContainer(containerId);
|
||||
await container.restart();
|
||||
|
||||
@@ -448,7 +449,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
throw new Error('App ID not provided');
|
||||
}
|
||||
|
||||
console.log(`[WorkflowEngine] Creating backup for: ${appId}`);
|
||||
log.info('workflow', 'Creating backup', { appId });
|
||||
|
||||
// Use backup manager's executeBackup if available
|
||||
const backupName = `${appId}-${label}`;
|
||||
@@ -477,11 +478,11 @@ class WorkflowEngine extends EventEmitter {
|
||||
async notify(message, channel) {
|
||||
const notification = this.ctx.notification;
|
||||
if (!notification) {
|
||||
console.warn('[WorkflowEngine] Notification manager not available');
|
||||
log.warn('workflow', 'Notification manager not available');
|
||||
return { notified: false, reason: 'no notification manager' };
|
||||
}
|
||||
|
||||
console.log(`[WorkflowEngine] Sending notification: ${message}`);
|
||||
log.info('workflow', 'Sending notification', { message });
|
||||
notification.send('workflow', 'Workflow Notification', message, 'info');
|
||||
|
||||
return { notified: true, message };
|
||||
@@ -548,7 +549,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
}
|
||||
}
|
||||
|
||||
console.log(`[WorkflowEngine] Workflow ${workflowId} ${enabled ? 'enabled' : 'disabled'}`);
|
||||
log.info('workflow', 'Workflow toggled', { workflowId, enabled });
|
||||
return { workflowId, enabled };
|
||||
}
|
||||
|
||||
@@ -581,7 +582,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
const conditionMet = this.evaluateCondition(workflow.condition, eventData);
|
||||
return conditionMet;
|
||||
} catch (e) {
|
||||
console.warn(`[WorkflowEngine] Condition evaluation failed for ${id}:`, e.message);
|
||||
log.warn('workflow', 'Condition evaluation failed', { workflowId: id, error: e.message });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -641,7 +642,7 @@ class WorkflowEngine extends EventEmitter {
|
||||
for (const [workflowId] of this.scheduledJobs) {
|
||||
this.stopScheduledWorkflow(workflowId);
|
||||
}
|
||||
console.log('[WorkflowEngine] All scheduled workflows stopped');
|
||||
log.info('workflow', 'All scheduled workflows stopped');
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ const crypto = require('crypto');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const platformPaths = require('../../platform-paths');
|
||||
const { log } = require('../utils/logging');
|
||||
|
||||
// Encryption settings
|
||||
const ALGORITHM = 'aes-256-gcm';
|
||||
@@ -65,7 +66,7 @@ function loadOrCreateKey() {
|
||||
// Check for key in environment variable first
|
||||
if (process.env.DASHCADDY_ENCRYPTION_KEY) {
|
||||
encryptionKey = Buffer.from(process.env.DASHCADDY_ENCRYPTION_KEY, 'hex');
|
||||
console.log('[Crypto] Using encryption key from environment variable');
|
||||
log.info('crypto', 'Using encryption key from environment variable');
|
||||
return encryptionKey;
|
||||
}
|
||||
|
||||
@@ -75,16 +76,16 @@ function loadOrCreateKey() {
|
||||
const keyData = fs.readFileSync(KEY_FILE, 'utf8').trim();
|
||||
if (keyData.length >= 64) {
|
||||
encryptionKey = Buffer.from(keyData, 'hex');
|
||||
console.log('[Crypto] Loaded encryption key from file');
|
||||
log.info('crypto', 'Loaded encryption key from file');
|
||||
// First-run bootstrap: if .bak doesn't exist yet, write the current
|
||||
// key to it. This ensures the silent recovery path is available from
|
||||
// the very next restart without requiring an explicit rotateKey().
|
||||
if (!fs.existsSync(KEY_FILE + '.bak')) {
|
||||
try {
|
||||
fs.writeFileSync(KEY_FILE + '.bak', keyData, { mode: 0o600 });
|
||||
console.log(`[Crypto] Seeded ${KEY_FILE}.bak with current key for future fallback`);
|
||||
log.info('crypto', 'Seeded .bak key file for future fallback');
|
||||
} catch (e) {
|
||||
console.warn('[Crypto] Could not seed .bak key file:', e.message);
|
||||
log.warn('crypto', 'Could not seed .bak key file', { error: e.message });
|
||||
}
|
||||
}
|
||||
// Try fallback to .bak key if primary can't decrypt existing credentials.
|
||||
@@ -98,14 +99,14 @@ function loadOrCreateKey() {
|
||||
encryptionKey = tryFallbackToBackupKey(Buffer.from(keyData, 'hex'), Buffer.from(backupData, 'hex'));
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn('[Crypto] Could not check backup key:', e.message);
|
||||
log.warn('crypto', 'Could not check backup key', { error: e.message });
|
||||
}
|
||||
}
|
||||
return encryptionKey;
|
||||
}
|
||||
// File exists but key is invalid/empty - will generate new one below
|
||||
} catch (error) {
|
||||
console.error('[Crypto] Error loading key file:', error.message);
|
||||
log.error('crypto', error, { operation: 'loadKey' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -115,10 +116,10 @@ function loadOrCreateKey() {
|
||||
try {
|
||||
// Save key to file with restricted permissions
|
||||
fs.writeFileSync(KEY_FILE, encryptionKey.toString('hex'), { mode: 0o600 });
|
||||
console.log('[Crypto] Generated and saved new encryption key');
|
||||
log.info('crypto', 'Generated and saved new encryption key');
|
||||
} catch (error) {
|
||||
console.warn('[Crypto] Could not save key to file:', error.message);
|
||||
console.warn('[Crypto] Key will be regenerated on restart - credentials will need to be re-entered');
|
||||
log.warn('crypto', 'Could not save key to file', { error: error.message });
|
||||
log.warn('crypto', 'Key will be regenerated on restart - credentials will need to be re-entered');
|
||||
}
|
||||
|
||||
return encryptionKey;
|
||||
@@ -171,12 +172,7 @@ function tryFallbackToBackupKey(primaryKey, backupKey) {
|
||||
|
||||
if (tryDecrypt(primaryKey)) return primaryKey;
|
||||
if (tryDecrypt(backupKey)) {
|
||||
console.warn(
|
||||
'[Crypto] Primary encryption key failed to decrypt credentials; ' +
|
||||
'fell back to .encryption-key.bak. The current primary key was set ' +
|
||||
'without preserving the original. Consider rotating the key explicitly ' +
|
||||
'via the credential-manager API to avoid this warning next restart.'
|
||||
);
|
||||
log.warn('crypto', 'Primary encryption key failed to decrypt credentials; fell back to .encryption-key.bak. Consider rotating the key explicitly via the credential-manager API.');
|
||||
return backupKey;
|
||||
}
|
||||
return primaryKey; // neither works — credential-manager.diagnose() will report 'unreadable'
|
||||
@@ -291,7 +287,7 @@ function decryptFields(obj, fields = null) {
|
||||
try {
|
||||
result[field] = decrypt(result[field]);
|
||||
} catch (error) {
|
||||
console.error(`[Crypto] Failed to decrypt field '${field}':`, error.message);
|
||||
log.error('crypto', error, { field, operation: 'decryptField' });
|
||||
// Leave the field as-is if decryption fails
|
||||
}
|
||||
}
|
||||
@@ -315,7 +311,7 @@ function migrateToEncrypted(credentials, sensitiveFields) {
|
||||
return credentials; // Already encrypted
|
||||
}
|
||||
|
||||
console.log('[Crypto] Migrating plaintext credentials to encrypted format');
|
||||
log.info('crypto', 'Migrating plaintext credentials to encrypted format');
|
||||
return encryptFields(credentials, sensitiveFields);
|
||||
}
|
||||
|
||||
@@ -340,10 +336,10 @@ function readEncryptedFile(filePath, sensitiveFields = ['password', 'token', 'ap
|
||||
}
|
||||
|
||||
// Plain text data - migrate it
|
||||
console.log(`[Crypto] Found plaintext data in ${filePath}, will encrypt on next save`);
|
||||
log.info('crypto', 'Found plaintext data', { filePath });
|
||||
return parsed;
|
||||
} catch (error) {
|
||||
console.error(`[Crypto] Error reading ${filePath}:`, error.message);
|
||||
log.error('crypto', error, { filePath, operation: 'readFile' });
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -357,7 +353,7 @@ function readEncryptedFile(filePath, sensitiveFields = ['password', 'token', 'ap
|
||||
function writeEncryptedFile(filePath, credentials, sensitiveFields = ['password', 'token', 'apiKey', 'secret']) {
|
||||
const encrypted = encryptFields(credentials, sensitiveFields);
|
||||
fs.writeFileSync(filePath, JSON.stringify(encrypted, null, 2), 'utf8');
|
||||
console.log(`[Crypto] Saved encrypted credentials to ${filePath}`);
|
||||
log.info('crypto', 'Saved encrypted credentials', { filePath });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -377,7 +373,7 @@ function rotateKey() {
|
||||
try {
|
||||
fs.writeFileSync(KEY_FILE + '.bak', oldKey.toString('hex'), { mode: 0o600 });
|
||||
} catch (error) {
|
||||
console.warn(`[Crypto] Could not save backup key to ${KEY_FILE}.bak:`, error.message);
|
||||
log.warn('crypto', 'Could not save backup key', { error: error.message });
|
||||
}
|
||||
|
||||
try {
|
||||
|
||||
@@ -9,6 +9,7 @@ const path = require('path');
|
||||
const https = require('https');
|
||||
const Docker = require('dockerode');
|
||||
const platformPaths = require('../../platform-paths');
|
||||
const { log } = require('../utils/logging');
|
||||
|
||||
const docker = new Docker();
|
||||
|
||||
@@ -19,7 +20,7 @@ class DockerSecurity {
|
||||
constructor() {
|
||||
this.config = this.loadConfig();
|
||||
this.mode = VERIFICATION_MODE;
|
||||
console.log(`[DockerSecurity] Initialized in ${this.mode} mode`);
|
||||
log.info('security', 'Docker security initialized', { mode: this.mode });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -32,7 +33,7 @@ class DockerSecurity {
|
||||
return JSON.parse(data);
|
||||
}
|
||||
} catch (error) {
|
||||
console.warn(`[DockerSecurity] Failed to load config: ${error.message}`);
|
||||
log.warn('security', 'Failed to load config', { error: error.message });
|
||||
}
|
||||
|
||||
// Default configuration
|
||||
@@ -51,7 +52,7 @@ class DockerSecurity {
|
||||
try {
|
||||
fs.writeFileSync(SECURITY_CONFIG_FILE, JSON.stringify(this.config, null, 2));
|
||||
} catch (error) {
|
||||
console.error(`[DockerSecurity] Failed to save config: ${error.message}`);
|
||||
log.error('security', error, { operation: 'saveConfig' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -110,7 +111,7 @@ class DockerSecurity {
|
||||
repository = repository.split(':')[0];
|
||||
}
|
||||
|
||||
console.log(`[DockerSecurity] Fetching manifest for ${registry}/${repository}:${tag}`);
|
||||
log.info('security', 'Fetching manifest', { registry, repository, tag });
|
||||
|
||||
return new Promise((resolve, reject) => {
|
||||
const isDockerHub = registry === 'registry-1.docker.io';
|
||||
@@ -216,7 +217,7 @@ class DockerSecurity {
|
||||
if (this.config.updateTrustedOnPull) {
|
||||
this.config.trustedDigests[imageName] = actualDigest;
|
||||
this.saveConfig();
|
||||
console.log(`[DockerSecurity] Added trusted digest for ${imageName}`);
|
||||
log.info('security', 'Added trusted digest', { imageName });
|
||||
}
|
||||
}
|
||||
} else if (actualDigest === trustedDigest) {
|
||||
@@ -250,26 +251,26 @@ class DockerSecurity {
|
||||
* @returns {Promise<object>} Verification result
|
||||
*/
|
||||
async verifyPulledImage(imageName) {
|
||||
console.log(`[DockerSecurity] Verifying image: ${imageName}`);
|
||||
log.info('security', 'Verifying image', { imageName });
|
||||
|
||||
try {
|
||||
const actualDigest = await this.getImageDigest(imageName);
|
||||
const result = await this.verifyImageDigest(imageName, actualDigest);
|
||||
|
||||
if (result.action === 'reject') {
|
||||
console.error(`[DockerSecurity] REJECTED: ${result.reason}`);
|
||||
log.error('security', 'Image REJECTED', { imageName, reason: result.reason });
|
||||
throw new Error(`Image verification failed: ${result.reason}`);
|
||||
} else if (result.action === 'warn') {
|
||||
console.warn(`[DockerSecurity] WARNING: ${result.reason}`);
|
||||
console.warn(`[DockerSecurity] Expected: ${result.trustedDigest}`);
|
||||
console.warn(`[DockerSecurity] Actual: ${result.actualDigest}`);
|
||||
log.warn('security', 'Image WARNING', { imageName, reason: result.reason });
|
||||
log.warn('security', 'Expected digest', { imageName, digest: result.trustedDigest });
|
||||
log.warn('security', 'Actual digest', { imageName, digest: result.actualDigest });
|
||||
} else {
|
||||
console.log(`[DockerSecurity] ACCEPTED: ${result.reason}`);
|
||||
log.info('security', 'Image ACCEPTED', { imageName, reason: result.reason });
|
||||
}
|
||||
|
||||
return result;
|
||||
} catch (error) {
|
||||
console.error(`[DockerSecurity] Verification error: ${error.message}`);
|
||||
log.error('security', error, { imageName, operation: 'verify' });
|
||||
|
||||
if (this.mode === 'strict') {
|
||||
throw error;
|
||||
@@ -294,7 +295,7 @@ class DockerSecurity {
|
||||
setTrustedDigest(imageName, digest) {
|
||||
this.config.trustedDigests[imageName] = digest;
|
||||
this.saveConfig();
|
||||
console.log(`[DockerSecurity] Updated trusted digest for ${imageName}`);
|
||||
log.info('security', 'Updated trusted digest', { imageName });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -304,7 +305,7 @@ class DockerSecurity {
|
||||
removeTrustedDigest(imageName) {
|
||||
delete this.config.trustedDigests[imageName];
|
||||
this.saveConfig();
|
||||
console.log(`[DockerSecurity] Removed trusted digest for ${imageName}`);
|
||||
log.info('security', 'Removed trusted digest', { imageName });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -325,7 +326,7 @@ class DockerSecurity {
|
||||
this.mode = mode;
|
||||
this.config.verificationMode = mode;
|
||||
this.saveConfig();
|
||||
console.log(`[DockerSecurity] Verification mode set to: ${mode}`);
|
||||
log.info('security', 'Verification mode set', { mode });
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -37,6 +37,7 @@
|
||||
*/
|
||||
|
||||
const fs = require('fs');
|
||||
const { log } = require('../utils/logging');
|
||||
const path = require('path');
|
||||
const os = require('os');
|
||||
const platformPaths = require('../../platform-paths');
|
||||
@@ -95,7 +96,7 @@ function createTail({ filePath, stateFile, onLine, label = 'tail', pollMs = 1000
|
||||
for (const line of lines) {
|
||||
if (line.trim()) {
|
||||
try { onLine(line); } catch (e) {
|
||||
console.error(`[${label}] onLine threw:`, e.message);
|
||||
log.error('events', e, { worker: label, phase: 'onLine' });
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -106,7 +107,7 @@ function createTail({ filePath, stateFile, onLine, label = 'tail', pollMs = 1000
|
||||
setTimeout(tick, pollMs);
|
||||
});
|
||||
stream.on('error', (e) => {
|
||||
console.error(`[${label}] read error:`, e.message);
|
||||
log.error('events', e, { worker: label, phase: 'read' });
|
||||
setTimeout(tick, pollMs * 5);
|
||||
});
|
||||
});
|
||||
@@ -267,11 +268,11 @@ function startFail2banWorker({ log } = {}) {
|
||||
function startAll({ log } = {}) {
|
||||
const workers = [];
|
||||
try { workers.push(startCaddyWorker({ log })); }
|
||||
catch (e) { console.error('[workers] caddy worker failed to start:', e.message); }
|
||||
catch (e) { log.error('events', e, { worker: 'caddy', phase: 'start' }); }
|
||||
try { workers.push(startSharedBansWorker({ log })); }
|
||||
catch (e) { console.error('[workers] shared_bans worker failed to start:', e.message); }
|
||||
catch (e) { log.error('events', e, { worker: 'shared_bans', phase: 'start' }); }
|
||||
try { workers.push(startFail2banWorker({ log })); }
|
||||
catch (e) { console.error('[workers] fail2ban worker failed to start:', e.message); }
|
||||
catch (e) { log.error('events', e, { worker: 'fail2ban', phase: 'start' }); }
|
||||
return {
|
||||
stop() { workers.forEach(w => { try { w.stop(); } catch {} }); },
|
||||
workers,
|
||||
|
||||
@@ -7,6 +7,7 @@
|
||||
const { execSync, execFileSync } = require('child_process');
|
||||
const os = require('os');
|
||||
const crypto = require('crypto');
|
||||
const { log } = require('../utils/logging');
|
||||
|
||||
const SERVICE_NAME = 'DashCaddy';
|
||||
const ACCOUNT_PREFIX = 'dashcaddy';
|
||||
@@ -44,7 +45,7 @@ class KeychainManager {
|
||||
}
|
||||
return false;
|
||||
} catch {
|
||||
console.warn('[Keychain] OS keychain not available, will use encrypted file storage');
|
||||
log.warn('keychain', 'OS keychain not available, will use encrypted file storage');
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -72,7 +73,7 @@ class KeychainManager {
|
||||
}
|
||||
return false;
|
||||
} catch (error) {
|
||||
console.error(`[Keychain] Failed to store ${key}:`, error.message);
|
||||
log.error('keychain', error, { key, operation: 'store' });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
@@ -99,7 +100,7 @@ class KeychainManager {
|
||||
}
|
||||
return null;
|
||||
} catch (error) {
|
||||
console.error(`[Keychain] Failed to retrieve ${key}:`, error.message);
|
||||
log.error('keychain', error, { key, operation: 'retrieve' });
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -126,7 +127,7 @@ class KeychainManager {
|
||||
}
|
||||
return false;
|
||||
} catch (error) {
|
||||
console.error(`[Keychain] Failed to delete ${key}:`, error.message);
|
||||
log.error('keychain', error, { key, operation: 'delete' });
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ const fs = require('fs');
|
||||
const fsp = require('fs').promises;
|
||||
const path = require('path');
|
||||
const { DOCKER } = require('../utilities/constants');
|
||||
const { log } = require('../utils/logging');
|
||||
|
||||
const docker = new Docker();
|
||||
|
||||
@@ -63,7 +64,7 @@ class LogDigest extends EventEmitter {
|
||||
// Collect logs every hour
|
||||
this.collectInterval = setInterval(() => {
|
||||
this._collectHourlyLogs().catch(e =>
|
||||
console.error('[LogDigest] Hourly collection failed:', e.message)
|
||||
log.error('logdigest', e, { phase: 'hourlyCollect' })
|
||||
);
|
||||
}, DOCKER.DIGEST.COLLECT_INTERVAL);
|
||||
|
||||
@@ -71,7 +72,7 @@ class LogDigest extends EventEmitter {
|
||||
this._scheduleDailyDigest();
|
||||
|
||||
// Run initial collection after 2 minutes
|
||||
setTimeout(() => {
|
||||
this._initialTimeout = setTimeout(() => {
|
||||
if (this.running) {
|
||||
this._collectHourlyLogs().catch(() => {});
|
||||
}
|
||||
@@ -89,6 +90,10 @@ class LogDigest extends EventEmitter {
|
||||
clearTimeout(this.digestTimeout);
|
||||
this.digestTimeout = null;
|
||||
}
|
||||
if (this._initialTimeout) {
|
||||
clearTimeout(this._initialTimeout);
|
||||
this._initialTimeout = null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -195,7 +200,7 @@ class LogDigest extends EventEmitter {
|
||||
hourSummary.services[appId] = serviceSummary;
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('[LogDigest] Container enumeration failed:', e.message);
|
||||
log.error('logdigest', e, { phase: 'enumerateContainers' });
|
||||
}
|
||||
|
||||
// Add to ring buffer
|
||||
@@ -258,7 +263,7 @@ class LogDigest extends EventEmitter {
|
||||
const delay = next.getTime() - now.getTime();
|
||||
this.digestTimeout = setTimeout(() => {
|
||||
this.generateDailyDigest().catch(e =>
|
||||
console.error('[LogDigest] Daily digest generation failed:', e.message)
|
||||
log.error('logdigest', e, { phase: 'dailyDigest' })
|
||||
);
|
||||
// Reschedule for tomorrow
|
||||
if (this.running) this._scheduleDailyDigest();
|
||||
|
||||
@@ -9,6 +9,7 @@ const { execSync } = require('child_process');
|
||||
const crypto = require('crypto');
|
||||
const EventEmitter = require('events');
|
||||
const platformPaths = require('../../platform-paths');
|
||||
const { log } = require('../utils/logging');
|
||||
|
||||
// Format bytes to human readable string
|
||||
function formatBytes(bytes) {
|
||||
@@ -38,7 +39,7 @@ class BackupManager extends EventEmitter {
|
||||
start() {
|
||||
if (this.running) return;
|
||||
|
||||
console.log('[BackupManager] Starting backup scheduler');
|
||||
log.info('backup', 'Starting backup scheduler');
|
||||
this.running = true;
|
||||
|
||||
// Schedule all configured backups
|
||||
@@ -55,7 +56,7 @@ class BackupManager extends EventEmitter {
|
||||
stop() {
|
||||
if (!this.running) return;
|
||||
|
||||
console.log('[BackupManager] Stopping backup scheduler');
|
||||
log.info('backup', 'Stopping backup scheduler');
|
||||
this.running = false;
|
||||
|
||||
// Clear all scheduled jobs
|
||||
@@ -91,7 +92,7 @@ class BackupManager extends EventEmitter {
|
||||
if (!isNaN(minutes) && minutes > 0) {
|
||||
intervalMs = minutes * 60 * 1000;
|
||||
} else {
|
||||
console.error(`[BackupManager] Invalid schedule for ${name}: ${backup.schedule}`);
|
||||
log.warn('backup', 'Invalid schedule', { name, schedule: backup.schedule });
|
||||
return;
|
||||
}
|
||||
}
|
||||
@@ -100,17 +101,17 @@ class BackupManager extends EventEmitter {
|
||||
// Schedule the job
|
||||
const job = setInterval(() => {
|
||||
this.executeBackup(name, backup).catch(error => {
|
||||
console.error(`[BackupManager] Scheduled backup ${name} failed:`, error.message);
|
||||
log.error('backup', error, { name });
|
||||
});
|
||||
}, intervalMs);
|
||||
|
||||
this.scheduledJobs.set(name, job);
|
||||
console.log(`[BackupManager] Scheduled backup '${name}' every ${backup.schedule}`);
|
||||
log.info('backup', 'Scheduled backup', { name, schedule: backup.schedule });
|
||||
|
||||
// Run immediately if configured
|
||||
if (backup.runImmediately) {
|
||||
this.executeBackup(name, backup).catch(error => {
|
||||
console.error(`[BackupManager] Initial backup ${name} failed:`, error.message);
|
||||
log.error('backup', error, { name, phase: 'initial' });
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -122,7 +123,7 @@ class BackupManager extends EventEmitter {
|
||||
const startTime = Date.now();
|
||||
const backupId = `${name}-${Date.now()}`;
|
||||
|
||||
console.log(`[BackupManager] Starting backup: ${name}`);
|
||||
log.info('backup', 'Starting backup', { name });
|
||||
|
||||
this.emit('backup-start', { name, backupId, timestamp: new Date().toISOString() });
|
||||
|
||||
@@ -151,7 +152,7 @@ class BackupManager extends EventEmitter {
|
||||
const location = await this.saveToDestination(finalData, dest, backupId);
|
||||
savedLocations.push(location);
|
||||
} catch (error) {
|
||||
console.error(`[BackupManager] Failed to save to ${dest.type}:`, error.message);
|
||||
log.error('backup', error, { destType: dest.type });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -192,7 +193,7 @@ class BackupManager extends EventEmitter {
|
||||
}
|
||||
|
||||
this.emit('backup-complete', historyEntry);
|
||||
console.log(`[BackupManager] Backup ${name} completed in ${duration}ms`);
|
||||
log.info('backup', 'Backup completed', { name, durationMs: duration });
|
||||
|
||||
return historyEntry;
|
||||
} catch (error) {
|
||||
@@ -263,7 +264,7 @@ class BackupManager extends EventEmitter {
|
||||
return JSON.parse(fs.readFileSync(servicesFile, 'utf8'));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[BackupManager] Error backing up services:', error.message);
|
||||
log.error('backup', error, { source: 'services' });
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -278,7 +279,7 @@ class BackupManager extends EventEmitter {
|
||||
return JSON.parse(fs.readFileSync(configFile, 'utf8'));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[BackupManager] Error backing up config:', error.message);
|
||||
log.error('backup', error, { source: 'config' });
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -291,7 +292,7 @@ class BackupManager extends EventEmitter {
|
||||
const credentialManager = require('../managers/credential-manager');
|
||||
return credentialManager.exportBackup();
|
||||
} catch (error) {
|
||||
console.error('[BackupManager] Error backing up credentials:', error.message);
|
||||
log.error('backup', error, { source: 'credentials' });
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -304,7 +305,7 @@ class BackupManager extends EventEmitter {
|
||||
const resourceMonitor = require('../managers/resource-monitor');
|
||||
return resourceMonitor.exportStats();
|
||||
} catch (error) {
|
||||
console.error('[BackupManager] Error backing up stats:', error.message);
|
||||
log.error('backup', error, { source: 'stats' });
|
||||
}
|
||||
return null;
|
||||
}
|
||||
@@ -374,7 +375,7 @@ class BackupManager extends EventEmitter {
|
||||
});
|
||||
}
|
||||
} catch (volumeError) {
|
||||
console.error(`[BackupManager] Error backing up volume ${volume.Name}:`, volumeError.message);
|
||||
log.error('backup', volumeError, { volume: volume.Name });
|
||||
backupResults.push({
|
||||
name: volume.Name,
|
||||
status: 'failed',
|
||||
@@ -390,7 +391,7 @@ class BackupManager extends EventEmitter {
|
||||
volumes: backupResults
|
||||
};
|
||||
} catch (error) {
|
||||
console.error('[BackupManager] Error backing up volumes:', error.message);
|
||||
log.error('backup', error, { source: 'volumes' });
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -461,9 +462,9 @@ class BackupManager extends EventEmitter {
|
||||
timestamp: new Date().toISOString()
|
||||
});
|
||||
|
||||
console.log(`[BackupManager] Volume ${volumeName} restored successfully`);
|
||||
log.info('backup', 'Volume restored', { volume: volumeName });
|
||||
} catch (restoreError) {
|
||||
console.error(`[BackupManager] Error restoring volume ${volBackup.name}:`, restoreError.message);
|
||||
log.error('backup', restoreError, { volume: volBackup.name });
|
||||
restoreResults.push({
|
||||
name: volBackup.name,
|
||||
status: 'failed',
|
||||
@@ -849,7 +850,7 @@ class BackupManager extends EventEmitter {
|
||||
throw new Error('Backup verification failed: checksum mismatch');
|
||||
}
|
||||
|
||||
console.log('[BackupManager] Backup verified successfully');
|
||||
log.info('backup', 'Backup verified successfully');
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -860,7 +861,7 @@ class BackupManager extends EventEmitter {
|
||||
* Restore from backup
|
||||
*/
|
||||
async restoreBackup(backupId, options = {}) {
|
||||
console.log(`[BackupManager] Starting restore from backup: ${backupId}`);
|
||||
log.info('backup', 'Starting restore', { backupId });
|
||||
|
||||
this.emit('restore-start', { backupId, timestamp: new Date().toISOString() });
|
||||
|
||||
@@ -922,7 +923,7 @@ class BackupManager extends EventEmitter {
|
||||
timestamp: new Date().toISOString()
|
||||
});
|
||||
|
||||
console.log('[BackupManager] Restore completed successfully');
|
||||
log.info('backup', 'Restore completed successfully');
|
||||
return { success: true, restored };
|
||||
} catch (error) {
|
||||
this.emit('restore-failed', {
|
||||
@@ -940,7 +941,7 @@ class BackupManager extends EventEmitter {
|
||||
restoreServices(services) {
|
||||
const servicesFile = platformPaths.servicesFile;
|
||||
fs.writeFileSync(servicesFile, JSON.stringify(services, null, 2));
|
||||
console.log('[BackupManager] Services restored');
|
||||
log.info('backup', 'Services restored');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -949,7 +950,7 @@ class BackupManager extends EventEmitter {
|
||||
restoreConfig(config) {
|
||||
const configFile = platformPaths.configFile;
|
||||
fs.writeFileSync(configFile, JSON.stringify(config, null, 2));
|
||||
console.log('[BackupManager] Config restored');
|
||||
log.info('backup', 'Config restored');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -958,7 +959,7 @@ class BackupManager extends EventEmitter {
|
||||
restoreCredentials(credentials) {
|
||||
const credentialManager = require('../managers/credential-manager');
|
||||
credentialManager.importBackup(credentials);
|
||||
console.log('[BackupManager] Credentials restored');
|
||||
log.info('backup', 'Credentials restored');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -967,7 +968,7 @@ class BackupManager extends EventEmitter {
|
||||
restoreStats(stats) {
|
||||
const resourceMonitor = require('../managers/resource-monitor');
|
||||
resourceMonitor.importStats(stats);
|
||||
console.log('[BackupManager] Stats restored');
|
||||
log.info('backup', 'Stats restored');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -975,7 +976,7 @@ class BackupManager extends EventEmitter {
|
||||
*/
|
||||
async enforceStorageLimit(name, maxBytes) {
|
||||
const maxStr = formatBytes(maxBytes);
|
||||
console.log("[BackupManager] Enforcing storage limit: " + maxStr + " for \"" + name + "\"");
|
||||
log.info('backup', 'Enforcing storage limit', { name, limit: maxStr });
|
||||
|
||||
const backups = this.history
|
||||
.filter(b => b.name === name && b.status === 'success')
|
||||
@@ -994,10 +995,10 @@ class BackupManager extends EventEmitter {
|
||||
}
|
||||
}
|
||||
|
||||
console.log("[BackupManager] Current total size: " + formatBytes(totalSize) + ", limit: " + maxStr);
|
||||
log.info('backup', 'Current storage usage', { totalSize: formatBytes(totalSize), limit: maxStr });
|
||||
|
||||
if (totalSize <= maxBytes) {
|
||||
console.log("[BackupManager] Storage limit OK (" + formatBytes(totalSize) + " <= " + maxStr + ")");
|
||||
log.info('backup', 'Storage limit OK', { totalSize: formatBytes(totalSize), limit: maxStr });
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1013,10 +1014,10 @@ class BackupManager extends EventEmitter {
|
||||
const sz = backup.size || 0;
|
||||
totalSize -= sz;
|
||||
freed += sz;
|
||||
console.log("[BackupManager] Deleted " + formatBytes(sz) + ": " + path);
|
||||
log.info('backup', 'Deleted old backup file', { size: formatBytes(sz), path });
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("[BackupManager] Error deleting " + path + ": " + error.message);
|
||||
log.error('backup', error, { path });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1024,7 +1025,7 @@ class BackupManager extends EventEmitter {
|
||||
}
|
||||
|
||||
this.saveHistory();
|
||||
console.log("[BackupManager] Storage limit enforced. Freed " + formatBytes(freed) + ", now " + formatBytes(totalSize));
|
||||
log.info('backup', 'Storage limit enforced', { freed: formatBytes(freed), totalSize: formatBytes(totalSize) });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1051,9 +1052,9 @@ class BackupManager extends EventEmitter {
|
||||
// Remove from history
|
||||
this.history = this.history.filter(b => b.id !== backup.id);
|
||||
|
||||
console.log(`[BackupManager] Deleted old backup: ${backup.id}`);
|
||||
log.info('backup', 'Deleted old backup', { backupId: backup.id });
|
||||
} catch (error) {
|
||||
console.error(`[BackupManager] Error deleting backup ${backup.id}:`, error.message);
|
||||
log.error('backup', error, { backupId: backup.id });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1109,7 +1110,7 @@ class BackupManager extends EventEmitter {
|
||||
return JSON.parse(fs.readFileSync(BACKUP_CONFIG_FILE, 'utf8'));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[BackupManager] Error loading config:', error.message);
|
||||
log.error('backup', error, { operation: 'loadConfig' });
|
||||
}
|
||||
|
||||
return {
|
||||
@@ -1125,7 +1126,7 @@ class BackupManager extends EventEmitter {
|
||||
try {
|
||||
fs.writeFileSync(BACKUP_CONFIG_FILE, JSON.stringify(this.config, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[BackupManager] Error saving config:', error.message);
|
||||
log.error('backup', error, { operation: 'saveConfig' });
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1138,7 +1139,7 @@ class BackupManager extends EventEmitter {
|
||||
return JSON.parse(fs.readFileSync(BACKUP_HISTORY_FILE, 'utf8'));
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[BackupManager] Error loading history:', error.message);
|
||||
log.error('backup', error, { operation: 'loadHistory' });
|
||||
}
|
||||
return [];
|
||||
}
|
||||
@@ -1150,7 +1151,7 @@ class BackupManager extends EventEmitter {
|
||||
try {
|
||||
fs.writeFileSync(BACKUP_HISTORY_FILE, JSON.stringify(this.history, null, 2));
|
||||
} catch (error) {
|
||||
console.error('[BackupManager] Error saving history:', error.message);
|
||||
log.error('backup', error, { operation: 'saveHistory' });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -3,12 +3,161 @@
|
||||
* Validates config.json structure to catch typos and invalid values early.
|
||||
*/
|
||||
|
||||
const VALID_TIMEZONES_SAMPLE = [
|
||||
'UTC', 'America/New_York', 'America/Chicago', 'America/Denver', 'America/Los_Angeles',
|
||||
'Europe/London', 'Europe/Paris', 'Europe/Berlin', 'Asia/Tokyo', 'Asia/Shanghai',
|
||||
'Asia/Singapore', 'Australia/Sydney', 'Pacific/Auckland'
|
||||
const VALID_THEMES = ['dark', 'light', 'blue'];
|
||||
const VALID_ROUTING_MODES = ['subdomain', 'subdirectory'];
|
||||
const VALID_DNS_PROVIDERS = ['technitium', 'cloudflare', 'rfc2136', 'manual'];
|
||||
|
||||
const KNOWN_KEYS = [
|
||||
'tld', 'caName', 'dns', 'dnsServers', 'dashboardHost', 'timezone', 'theme',
|
||||
'updatedAt', 'timestamp', 'logo', 'logoPosition', 'favicon', 'weather',
|
||||
'setupComplete', 'setupCompleted', 'setupMode', 'onboardingCompleted',
|
||||
'configurationType', 'defaults', 'customLogo', 'customFavicon',
|
||||
'dashboardTitle', 'tailscale', 'license', 'skipped',
|
||||
'routingMode', 'domain', 'email', 'defaultIP', 'pylon',
|
||||
'customLogoDark', 'customLogoLight'
|
||||
];
|
||||
|
||||
/**
|
||||
* @param {string[]} arr
|
||||
* @param {string} val
|
||||
* @returns {boolean}
|
||||
*/
|
||||
function isInArray(arr, val) {
|
||||
return arr.includes(val);
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{errors:string[], warnings:string[]}} ctx
|
||||
* @param {object} config
|
||||
*/
|
||||
function validateTld(ctx, config) {
|
||||
if (config.tld === undefined) return;
|
||||
if (typeof config.tld !== 'string') {
|
||||
ctx.errors.push('tld must be a string');
|
||||
return;
|
||||
}
|
||||
const tld = config.tld.startsWith('.') ? config.tld : '.' + config.tld;
|
||||
if (!/^\.[a-z0-9][a-z0-9-]*$/.test(tld)) {
|
||||
ctx.errors.push(`tld "${config.tld}" contains invalid characters (use lowercase alphanumeric)`);
|
||||
}
|
||||
if (tld.length > 20) {
|
||||
ctx.warnings.push(`tld "${config.tld}" is unusually long`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{errors:string[], warnings:string[]}} ctx
|
||||
* @param {object} config
|
||||
*/
|
||||
function validateDns(ctx, config) {
|
||||
if (config.dns === undefined) return;
|
||||
if (typeof config.dns !== 'object' || config.dns === null) {
|
||||
ctx.errors.push('dns must be an object');
|
||||
return;
|
||||
}
|
||||
if (config.dns.ip !== undefined && typeof config.dns.ip !== 'string') {
|
||||
ctx.errors.push('dns.ip must be a string');
|
||||
}
|
||||
if (config.dns.ip && !/^[\d.]+$/.test(config.dns.ip) && !/^[a-zA-Z0-9.-]+$/.test(config.dns.ip)) {
|
||||
ctx.errors.push(`dns.ip "${config.dns.ip}" is not a valid IP address or hostname`);
|
||||
}
|
||||
if (config.dns.port !== undefined) {
|
||||
const port = parseInt(config.dns.port, 10);
|
||||
if (isNaN(port) || port < 1 || port > 65535) {
|
||||
ctx.errors.push(`dns.port "${config.dns.port}" is not a valid port number (1-65535)`);
|
||||
}
|
||||
}
|
||||
if (config.dns.servers !== undefined) {
|
||||
if (typeof config.dns.servers !== 'object' || config.dns.servers === null) {
|
||||
ctx.errors.push('dns.servers must be an object');
|
||||
}
|
||||
}
|
||||
if (config.dns.provider !== undefined) {
|
||||
if (typeof config.dns.provider !== 'string') {
|
||||
ctx.errors.push('dns.provider must be a string');
|
||||
} else if (!isInArray(VALID_DNS_PROVIDERS, config.dns.provider)) {
|
||||
ctx.warnings.push(`dns.provider "${config.dns.provider}" is not one of: ${VALID_DNS_PROVIDERS.join(', ')}. It may still work if a custom adapter is installed.`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{errors:string[], warnings:string[]}} ctx
|
||||
* @param {object} config
|
||||
*/
|
||||
function validateDashboardHost(ctx, config) {
|
||||
if (config.dashboardHost === undefined) return;
|
||||
if (typeof config.dashboardHost !== 'string') {
|
||||
ctx.errors.push('dashboardHost must be a string');
|
||||
} else if (config.dashboardHost && !/^[a-zA-Z0-9][a-zA-Z0-9.-]*$/.test(config.dashboardHost)) {
|
||||
ctx.errors.push(`dashboardHost "${config.dashboardHost}" contains invalid characters`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{errors:string[], warnings:string[]}} ctx
|
||||
* @param {object} config
|
||||
*/
|
||||
function validateTimezone(ctx, config) {
|
||||
if (config.timezone === undefined) return;
|
||||
if (typeof config.timezone !== 'string') {
|
||||
ctx.errors.push('timezone must be a string');
|
||||
} else if (config.timezone) {
|
||||
try {
|
||||
Intl.DateTimeFormat(undefined, { timeZone: config.timezone });
|
||||
} catch {
|
||||
ctx.errors.push(`timezone "${config.timezone}" is not a recognized IANA timezone`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{errors:string[], warnings:string[]}} ctx
|
||||
* @param {object} config
|
||||
*/
|
||||
function validateTheme(ctx, config) {
|
||||
if (config.theme === undefined) return;
|
||||
if (!isInArray(VALID_THEMES, config.theme)) {
|
||||
ctx.warnings.push(`theme "${config.theme}" is not one of: ${VALID_THEMES.join(', ')}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{errors:string[], warnings:string[]}} ctx
|
||||
* @param {object} config
|
||||
*/
|
||||
function validateRoutingMode(ctx, config) {
|
||||
if (config.routingMode === undefined) return;
|
||||
if (!isInArray(VALID_ROUTING_MODES, config.routingMode)) {
|
||||
ctx.errors.push(`routingMode "${config.routingMode}" is not one of: ${VALID_ROUTING_MODES.join(', ')}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{errors:string[], warnings:string[]}} ctx
|
||||
* @param {object} config
|
||||
*/
|
||||
function validateDomain(ctx, config) {
|
||||
if (config.domain === undefined) return;
|
||||
if (typeof config.domain !== 'string') {
|
||||
ctx.errors.push('domain must be a string');
|
||||
} else if (config.domain && !/^[a-z0-9][a-z0-9.-]*\.[a-z]{2,}$/i.test(config.domain)) {
|
||||
ctx.warnings.push(`domain "${config.domain}" may not be a valid domain name`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{warnings:string[]}} ctx
|
||||
* @param {object} config
|
||||
*/
|
||||
function validateKnownKeys(ctx, config) {
|
||||
for (const key of Object.keys(config)) {
|
||||
if (!isInArray(KNOWN_KEYS, key)) {
|
||||
ctx.warnings.push(`Unknown config key "${key}" — possible typo?`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a config object and return errors/warnings.
|
||||
* @param {object} config - The config object to validate
|
||||
@@ -17,123 +166,20 @@ const VALID_TIMEZONES_SAMPLE = [
|
||||
function validateConfig(config) {
|
||||
const errors = [];
|
||||
const warnings = [];
|
||||
const ctx = { errors, warnings };
|
||||
|
||||
if (!config || typeof config !== 'object') {
|
||||
return { valid: false, errors: ['Config must be a non-null object'], warnings };
|
||||
}
|
||||
|
||||
// TLD validation
|
||||
if (config.tld !== undefined) {
|
||||
if (typeof config.tld !== 'string') {
|
||||
errors.push('tld must be a string');
|
||||
} else {
|
||||
const tld = config.tld.startsWith('.') ? config.tld : '.' + config.tld;
|
||||
if (!/^\.[a-z0-9][a-z0-9-]*$/.test(tld)) {
|
||||
errors.push(`tld "${config.tld}" contains invalid characters (use lowercase alphanumeric)`);
|
||||
}
|
||||
if (tld.length > 20) {
|
||||
warnings.push(`tld "${config.tld}" is unusually long`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// DNS config validation
|
||||
if (config.dns !== undefined) {
|
||||
if (typeof config.dns !== 'object' || config.dns === null) {
|
||||
errors.push('dns must be an object');
|
||||
} else {
|
||||
if (config.dns.ip !== undefined && typeof config.dns.ip !== 'string') {
|
||||
errors.push('dns.ip must be a string');
|
||||
}
|
||||
if (config.dns.ip && !/^[\d.]+$/.test(config.dns.ip) && !/^[a-zA-Z0-9.-]+$/.test(config.dns.ip)) {
|
||||
errors.push(`dns.ip "${config.dns.ip}" is not a valid IP address or hostname`);
|
||||
}
|
||||
if (config.dns.port !== undefined) {
|
||||
const port = parseInt(config.dns.port, 10);
|
||||
if (isNaN(port) || port < 1 || port > 65535) {
|
||||
errors.push(`dns.port "${config.dns.port}" is not a valid port number (1-65535)`);
|
||||
}
|
||||
}
|
||||
if (config.dns.servers !== undefined) {
|
||||
if (typeof config.dns.servers !== 'object' || config.dns.servers === null) {
|
||||
errors.push('dns.servers must be an object');
|
||||
}
|
||||
}
|
||||
// DNS provider validation
|
||||
if (config.dns.provider !== undefined) {
|
||||
const validProviders = ['technitium', 'cloudflare', 'rfc2136', 'manual'];
|
||||
if (typeof config.dns.provider !== 'string') {
|
||||
errors.push('dns.provider must be a string');
|
||||
} else if (!validProviders.includes(config.dns.provider)) {
|
||||
warnings.push(`dns.provider "${config.dns.provider}" is not one of: ${validProviders.join(', ')}. It may still work if a custom adapter is installed.`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Dashboard host validation
|
||||
if (config.dashboardHost !== undefined) {
|
||||
if (typeof config.dashboardHost !== 'string') {
|
||||
errors.push('dashboardHost must be a string');
|
||||
} else if (config.dashboardHost && !/^[a-zA-Z0-9][a-zA-Z0-9.-]*$/.test(config.dashboardHost)) {
|
||||
errors.push(`dashboardHost "${config.dashboardHost}" contains invalid characters`);
|
||||
}
|
||||
}
|
||||
|
||||
// Timezone validation
|
||||
if (config.timezone !== undefined) {
|
||||
if (typeof config.timezone !== 'string') {
|
||||
errors.push('timezone must be a string');
|
||||
} else if (config.timezone) {
|
||||
// Basic format check — full validation would require Intl API
|
||||
try {
|
||||
Intl.DateTimeFormat(undefined, { timeZone: config.timezone });
|
||||
} catch {
|
||||
errors.push(`timezone "${config.timezone}" is not a recognized IANA timezone`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Theme validation
|
||||
if (config.theme !== undefined) {
|
||||
const validThemes = ['dark', 'light', 'blue'];
|
||||
if (!validThemes.includes(config.theme)) {
|
||||
warnings.push(`theme "${config.theme}" is not one of: ${validThemes.join(', ')}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Routing mode validation
|
||||
if (config.routingMode !== undefined) {
|
||||
const validModes = ['subdomain', 'subdirectory'];
|
||||
if (!validModes.includes(config.routingMode)) {
|
||||
errors.push(`routingMode "${config.routingMode}" is not one of: ${validModes.join(', ')}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Domain validation
|
||||
if (config.domain !== undefined) {
|
||||
if (typeof config.domain !== 'string') {
|
||||
errors.push('domain must be a string');
|
||||
} else if (config.domain && !/^[a-z0-9][a-z0-9.-]*\.[a-z]{2,}$/i.test(config.domain)) {
|
||||
warnings.push(`domain "${config.domain}" may not be a valid domain name`);
|
||||
}
|
||||
}
|
||||
|
||||
// Warn on unknown top-level keys
|
||||
const knownKeys = [
|
||||
'tld', 'caName', 'dns', 'dnsServers', 'dashboardHost', 'timezone', 'theme',
|
||||
'updatedAt', 'timestamp', 'logo', 'logoPosition', 'favicon', 'weather',
|
||||
'setupComplete', 'setupCompleted', 'setupMode', 'onboardingCompleted',
|
||||
'configurationType', 'defaults', 'customLogo', 'customFavicon',
|
||||
'dashboardTitle', 'tailscale', 'license', 'skipped',
|
||||
'routingMode', 'domain', 'email', 'defaultIP', 'pylon',
|
||||
'customLogoDark', 'customLogoLight'
|
||||
];
|
||||
for (const key of Object.keys(config)) {
|
||||
if (!knownKeys.includes(key)) {
|
||||
warnings.push(`Unknown config key "${key}" — possible typo?`);
|
||||
}
|
||||
}
|
||||
validateTld(ctx, config);
|
||||
validateDns(ctx, config);
|
||||
validateDashboardHost(ctx, config);
|
||||
validateTimezone(ctx, config);
|
||||
validateTheme(ctx, config);
|
||||
validateRoutingMode(ctx, config);
|
||||
validateDomain(ctx, config);
|
||||
validateKnownKeys(ctx, config);
|
||||
|
||||
return { valid: errors.length === 0, errors, warnings };
|
||||
}
|
||||
|
||||
@@ -113,6 +113,41 @@ module.exports = function configureMiddleware(app, {
|
||||
next();
|
||||
});
|
||||
|
||||
// ── Tailscale authentication helpers ──
|
||||
|
||||
const PROBE_PATHS_TAILSCALE = new Set([
|
||||
'/health', '/health/live', '/health/ready', '/healthz', '/readyz',
|
||||
]);
|
||||
|
||||
function isTailScaleProbePath(reqPath) {
|
||||
return PROBE_PATHS_TAILSCALE.has(reqPath) || reqPath.startsWith('/probe/');
|
||||
}
|
||||
|
||||
function extractTailscaleIPs(req) {
|
||||
const clientIP = req.ip || req.socket?.remoteAddress || '';
|
||||
const forwardedFor = req.headers['x-forwarded-for'];
|
||||
const realIP = req.headers['x-real-ip'];
|
||||
const ipsToCheck = [clientIP, forwardedFor, realIP].filter(Boolean);
|
||||
const fromTailscale = ipsToCheck.some(ip =>
|
||||
isTailscaleIP(ip.toString().split(',')[0].trim()));
|
||||
const clientTailscaleIP = ipsToCheck
|
||||
.map(ip => ip.toString().split(',')[0].trim())
|
||||
.find(ip => isTailscaleIP(ip));
|
||||
return { clientIP, ipsToCheck, fromTailscale, clientTailscaleIP };
|
||||
}
|
||||
|
||||
async function isIPInTailnet(clientTailscaleIP) {
|
||||
const status = await getTailscaleStatus();
|
||||
if (!status) return true; // no status = can't verify = allow
|
||||
|
||||
const knownIPs = new Set();
|
||||
for (const ip of (status.Self?.TailscaleIPs || [])) knownIPs.add(ip);
|
||||
for (const peer of Object.values(status.Peer || {})) {
|
||||
for (const ip of (peer.TailscaleIPs || [])) knownIPs.add(ip);
|
||||
}
|
||||
return knownIPs.has(clientTailscaleIP);
|
||||
}
|
||||
|
||||
// ── Tailscale authentication middleware (optional) ──
|
||||
const tailscaleAuthMiddleware = async (req, res, next) => {
|
||||
if (!tailscaleConfig.enabled || !tailscaleConfig.requireAuth) {
|
||||
@@ -121,25 +156,11 @@ module.exports = function configureMiddleware(app, {
|
||||
|
||||
// Probe endpoints bypass Tailscale auth — k8s/Docker healthchecks
|
||||
// don't carry a Tailscale identity header.
|
||||
if (req.path === '/health'
|
||||
|| req.path === '/health/live'
|
||||
|| req.path === '/health/ready'
|
||||
|| req.path === '/healthz'
|
||||
|| req.path === '/readyz'
|
||||
|| req.path.startsWith('/probe/')) {
|
||||
if (isTailScaleProbePath(req.path) || req.path.startsWith('/api/v1/tailscale/')) {
|
||||
return next();
|
||||
}
|
||||
|
||||
if (req.path.startsWith('/api/v1/tailscale/')) {
|
||||
return next();
|
||||
}
|
||||
|
||||
const clientIP = req.ip || req.socket?.remoteAddress || '';
|
||||
const forwardedFor = req.headers['x-forwarded-for'];
|
||||
const realIP = req.headers['x-real-ip'];
|
||||
|
||||
const ipsToCheck = [clientIP, forwardedFor, realIP].filter(Boolean);
|
||||
const fromTailscale = ipsToCheck.some(ip => isTailscaleIP(ip.toString().split(',')[0].trim()));
|
||||
const { clientIP, fromTailscale, clientTailscaleIP } = extractTailscaleIPs(req);
|
||||
|
||||
if (!fromTailscale) {
|
||||
return errorResponse(res, 403, '[DC-120] Access denied. This dashboard requires Tailscale connection.', {
|
||||
@@ -148,28 +169,15 @@ module.exports = function configureMiddleware(app, {
|
||||
});
|
||||
}
|
||||
|
||||
if (tailscaleConfig.allowedTailnet) {
|
||||
if (tailscaleConfig.allowedTailnet && clientTailscaleIP) {
|
||||
try {
|
||||
const status = await getTailscaleStatus();
|
||||
if (status) {
|
||||
const clientTailscaleIP = ipsToCheck
|
||||
.map(ip => ip.toString().split(',')[0].trim())
|
||||
.find(ip => isTailscaleIP(ip));
|
||||
|
||||
if (clientTailscaleIP) {
|
||||
const knownIPs = new Set();
|
||||
for (const ip of (status.Self?.TailscaleIPs || [])) knownIPs.add(ip);
|
||||
for (const peer of Object.values(status.Peer || {})) {
|
||||
for (const ip of (peer.TailscaleIPs || [])) knownIPs.add(ip);
|
||||
}
|
||||
if (!knownIPs.has(clientTailscaleIP)) {
|
||||
const inTailnet = await isIPInTailnet(clientTailscaleIP);
|
||||
if (!inTailnet) {
|
||||
return errorResponse(res, 403, '[DC-121] Access denied. Device not in allowed tailnet.', {
|
||||
requiresTailscale: true,
|
||||
clientIP
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch (e) {
|
||||
log.warn('tailscale', 'Tailnet verification failed, allowing request', { error: e.message });
|
||||
}
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
/**
|
||||
* Graceful shutdown coordinator — DashCaddy
|
||||
*
|
||||
* Extracts the SIGTERM/SIGINT handler from server.js into a testable,
|
||||
* reusable module that:
|
||||
* 1. Calls server.close() to drain in-flight HTTP connections
|
||||
* 2. Stops each manager in a deterministic order
|
||||
* 3. Emits a 'shutdown' event so additional listeners can do cleanup
|
||||
* 4. Force-exits after a configurable drain timeout if connections don't drain
|
||||
* 5. Is idempotent — a second SIGTERM during shutdown does not re-run handlers
|
||||
*
|
||||
* Spec: DC-067 (production-grade backlog). Docker sends SIGTERM on stop;
|
||||
* without this coordinator, in-flight API calls drop.
|
||||
*
|
||||
* Exports:
|
||||
* - createShutdownCoordinator({ server, log, drainTimeoutMs, managers })
|
||||
* Returns an EventEmitter with: { shutdown, isShuttingDown, on, emit, ... }
|
||||
* - installSignalHandlers(coordinator, signals = ['SIGTERM', 'SIGINT'])
|
||||
* Registers the OS-level handlers. Idempotent.
|
||||
*/
|
||||
'use strict';
|
||||
|
||||
const EventEmitter = require('events');
|
||||
|
||||
const DEFAULT_DRAIN_TIMEOUT_MS = 10_000;
|
||||
|
||||
class ShutdownCoordinator extends EventEmitter {
|
||||
constructor({ server, log, drainTimeoutMs, managers }) {
|
||||
super();
|
||||
if (!server) throw new Error('createShutdownCoordinator: server is required');
|
||||
if (!log || typeof log.info !== 'function' || typeof log.warn !== 'function'
|
||||
|| typeof log.error !== 'function') {
|
||||
throw new Error('createShutdownCoordinator: log must have info(), warn(), and error() methods');
|
||||
}
|
||||
this.server = server;
|
||||
this.log = log;
|
||||
this.drainTimeoutMs = Number.isFinite(drainTimeoutMs) && drainTimeoutMs > 0
|
||||
? drainTimeoutMs
|
||||
: DEFAULT_DRAIN_TIMEOUT_MS;
|
||||
this.managers = Array.isArray(managers) ? managers : [];
|
||||
this._shuttingDown = false;
|
||||
this._forceTimer = null;
|
||||
}
|
||||
|
||||
isShuttingDown() {
|
||||
return this._shuttingDown;
|
||||
}
|
||||
|
||||
async _stopManager(m) {
|
||||
try {
|
||||
await m.stop();
|
||||
this.log.info('shutdown', `manager stopped: ${m.name}`);
|
||||
} catch (err) {
|
||||
this.log.warn('shutdown', `manager stop failed: ${m.name}`, { error: err.message });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stop each manager sequentially in declaration order. Each manager's
|
||||
* stop() is awaited so that a downstream manager is not stopped until
|
||||
* its upstream dependency has finished draining.
|
||||
*
|
||||
* IMPORTANT: this runs AFTER server.close() returns (see shutdown()).
|
||||
* We must wait for in-flight HTTP requests to complete before tearing
|
||||
* down the services that serve them — otherwise those requests fail
|
||||
* mid-drain with "service not found" / "monitor not running" errors.
|
||||
*/
|
||||
async _stopManagersInOrder() {
|
||||
for (const m of this.managers) {
|
||||
await this._stopManager(m);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Emit an event but swallow listener exceptions so one bad listener
|
||||
* can't abort the shutdown sequence. Logs each failure with the
|
||||
* listener's name (set via `listener.name`) if available.
|
||||
*/
|
||||
_safeEmit(event, ...args) {
|
||||
const listeners = this.listeners(event);
|
||||
for (const listener of listeners) {
|
||||
try {
|
||||
listener.apply(this, args);
|
||||
} catch (err) {
|
||||
const name = listener.name || '<anonymous>';
|
||||
this.log.error('shutdown', `event listener for '${event}' threw`,
|
||||
{ listener: name, error: err.message });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
shutdown(signal) {
|
||||
if (this._shuttingDown) {
|
||||
this.log.info('shutdown', `${signal || 'shutdown'} received — already shutting down, ignoring`);
|
||||
return;
|
||||
}
|
||||
this._shuttingDown = true;
|
||||
this.log.info('shutdown', `${signal || 'shutdown'} received, draining (timeout=${this.drainTimeoutMs}ms)...`);
|
||||
|
||||
// Emit 'shutdown' event first so any listeners can observe the signal
|
||||
// before the drain begins. NOTE: listeners should NOT tear down their
|
||||
// state here — that happens in the 'closed' event after server.close.
|
||||
// _safeEmit swallows listener exceptions so a buggy listener can't
|
||||
// abort the entire shutdown sequence.
|
||||
this._safeEmit('shutdown', signal);
|
||||
|
||||
// Close the HTTP server FIRST. Stops accepting new connections, waits
|
||||
// for in-flight requests to complete naturally. Only AFTER close fires
|
||||
// do we tear down managers — otherwise in-flight requests could fail
|
||||
// when the services they call have already been stopped.
|
||||
let serverClosed = false;
|
||||
let managersStopped = false;
|
||||
try {
|
||||
this.server.close(async () => {
|
||||
serverClosed = true;
|
||||
this.log.info('shutdown', 'HTTP server closed cleanly');
|
||||
// Now that in-flight requests are done, stop managers in order.
|
||||
// We do NOT clear the force-exit timer yet — if a manager's stop()
|
||||
// hangs, the timer is the safety net that prevents the process
|
||||
// from living forever in a half-shut-down state.
|
||||
try {
|
||||
await this._stopManagersInOrder();
|
||||
} catch (err) {
|
||||
// _stopManager already logs per-manager failures, but a top-level
|
||||
// throw (e.g. from the for-loop itself) is still possible.
|
||||
this.log.error('shutdown', 'manager shutdown loop threw', { error: err.message });
|
||||
}
|
||||
managersStopped = true;
|
||||
// Manager drain complete — NOW we can clear the safety timer.
|
||||
if (this._forceTimer) {
|
||||
clearTimeout(this._forceTimer);
|
||||
this._forceTimer = null;
|
||||
}
|
||||
this._safeEmit('closed', signal);
|
||||
process.exit(0);
|
||||
});
|
||||
} catch (err) {
|
||||
this.log.error('shutdown', 'server.close threw', { error: err.message });
|
||||
}
|
||||
|
||||
// Force-exit safety net. Fires when EITHER:
|
||||
// (a) server.close never fires (HTTP server stuck draining), or
|
||||
// (b) server.close fired but managers hung during stop()
|
||||
// We only suppress when managersStopped === true (full drain complete).
|
||||
// serverClosed alone is NOT enough — managers could still be running.
|
||||
this._forceTimer = setTimeout(() => {
|
||||
if (managersStopped) return; // full shutdown complete
|
||||
if (!serverClosed) {
|
||||
this.log.warn('shutdown', `drain timeout (${this.drainTimeoutMs}ms) reached before HTTP server closed, force-exiting`);
|
||||
} else {
|
||||
this.log.warn('shutdown', `drain timeout (${this.drainTimeoutMs}ms) reached after HTTP close (manager hung), force-exiting`);
|
||||
}
|
||||
process.exit(0);
|
||||
}, this.drainTimeoutMs);
|
||||
if (this._forceTimer && typeof this._forceTimer.unref === 'function') {
|
||||
this._forceTimer.unref();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function createShutdownCoordinator(opts) {
|
||||
return new ShutdownCoordinator(opts);
|
||||
}
|
||||
|
||||
/**
|
||||
* Install OS-level signal handlers. Idempotent: second call for the same
|
||||
* signal does NOT register a duplicate listener. Tracks registered signals
|
||||
* on the coordinator itself so a future caller can introspect.
|
||||
*
|
||||
* @param {ShutdownCoordinator} coordinator
|
||||
* @param {string[]} signals - signals to handle (default: SIGTERM, SIGINT)
|
||||
*/
|
||||
function installSignalHandlers(coordinator, signals) {
|
||||
if (!coordinator || typeof coordinator.shutdown !== 'function') {
|
||||
throw new Error('installSignalHandlers: coordinator required');
|
||||
}
|
||||
if (!Array.isArray(coordinator._installedSignals)) {
|
||||
coordinator._installedSignals = [];
|
||||
}
|
||||
const sigs = Array.isArray(signals) && signals.length > 0
|
||||
? signals
|
||||
: ['SIGTERM', 'SIGINT'];
|
||||
for (const sig of sigs) {
|
||||
if (coordinator._installedSignals.includes(sig)) continue;
|
||||
process.on(sig, () => coordinator.shutdown(sig));
|
||||
coordinator._installedSignals.push(sig);
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
createShutdownCoordinator,
|
||||
installSignalHandlers,
|
||||
DEFAULT_DRAIN_TIMEOUT_MS,
|
||||
ShutdownCoordinator, // exported for tests
|
||||
};
|
||||
@@ -9,10 +9,11 @@
|
||||
*
|
||||
* Priority:
|
||||
* 1. internet → https://www.google.com
|
||||
* 2. isExternal + externalUrl → use as-is
|
||||
* 3. service.url → prepend https:// if no protocol
|
||||
* 4. dnsServers config → http://{ip}:{port}
|
||||
* 5. fallback → buildServiceUrl(id)
|
||||
* 2. healthCheckUrl → use as-is (bypass SSO/Caddy for direct container health checks)
|
||||
* 3. isExternal + externalUrl → use as-is
|
||||
* 4. service.url → prepend https:// if no protocol
|
||||
* 5. dnsServers config → http://{ip}:{port}
|
||||
* 6. fallback → buildServiceUrl(id)
|
||||
*
|
||||
* @param {string} id - service identifier
|
||||
* @param {Object|null} service - service object from services.json (may be null for top-card services)
|
||||
@@ -22,6 +23,7 @@
|
||||
*/
|
||||
function resolveServiceUrl(id, service, siteConfig, buildServiceUrl) {
|
||||
if (id === 'internet') return 'https://www.google.com';
|
||||
if (service?.healthCheckUrl) return service.healthCheckUrl;
|
||||
if (service?.isExternal && service.externalUrl) return service.externalUrl;
|
||||
if (service?.url) return service.url.startsWith('http') ? service.url : `https://${service.url}`;
|
||||
const dnsServer = siteConfig?.dnsServers?.[id];
|
||||
|
||||
@@ -835,6 +835,22 @@ start_caddy() {
|
||||
fi
|
||||
}
|
||||
|
||||
# DC-037: Make API source reachable from both the install path
|
||||
# (${SITES_DIR}/dashcaddy-api, where this installer writes files) and the
|
||||
# /opt/dashcaddy/dashcaddy-api path that the auto-updater and several runtime
|
||||
# helpers default to. Without this, a first auto-update lands on a fresh host
|
||||
# that wrote its API files to ${SITES_DIR}/dashcaddy-api but tried to read
|
||||
# from /opt/dashcaddy/dashcaddy-api and crashes with
|
||||
# `cp: cannot create directory '/etc/dashcaddy/sites/dashcaddy-api/routes':
|
||||
# No such file or directory` because the trailing parent path is missing.
|
||||
# `ln -sfn` is idempotent (safe on re-runs; does not fail if the link already
|
||||
# points to the same target) and replaces any stale link.
|
||||
install_api_symlink() {
|
||||
mkdir -p /opt/dashcaddy
|
||||
ln -sfn "${API_DIR}" /opt/dashcaddy/dashcaddy-api
|
||||
ok "API symlink: /opt/dashcaddy/dashcaddy-api -> ${API_DIR}"
|
||||
}
|
||||
|
||||
# ============================================================================
|
||||
# Firewall
|
||||
# ============================================================================
|
||||
@@ -1091,6 +1107,7 @@ main() {
|
||||
# ---- Step 7: Start Caddy ----
|
||||
step "Starting web server"
|
||||
start_caddy
|
||||
install_api_symlink
|
||||
|
||||
print_success "$(elapsed "$start_time")"
|
||||
}
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
#!/bin/bash
|
||||
# DashCaddy Docker Space Management
|
||||
# Runs via cron to keep Docker disk usage under control
|
||||
# Prevents the overlay2 + dangling volumes + stale images that fill the disk
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
MAX_DISK_PCT=85 # Alert if disk usage exceeds this
|
||||
LOG_PREFIX="[dc-disk]"
|
||||
|
||||
# 1. Remove dangling (untagged) images
|
||||
echo "$LOG_PREFIX Pruning dangling images..."
|
||||
docker image prune -f --filter "dangling=true" 2>/dev/null || true
|
||||
|
||||
# 2. Remove unused volumes (volumes not attached to any container)
|
||||
echo "$LOG_PREFIX Pruning unused volumes..."
|
||||
docker volume prune -f 2>/dev/null || true
|
||||
|
||||
# 3. Remove old build cache
|
||||
echo "$LOG_PREFIX Pruning build cache..."
|
||||
docker builder prune -f --keep-storage 500m 2>/dev/null || true
|
||||
|
||||
# 4. Remove stopped containers older than 7 days
|
||||
echo "$LOG_PREFIX Pruning old stopped containers..."
|
||||
docker container prune -f --filter "until=168h" 2>/dev/null || true
|
||||
|
||||
# 5. Remove images not used by any container (keep only running images)
|
||||
# Only remove images older than 7 days to avoid breaking recent updates
|
||||
echo "$LOG_PREFIX Pruning unused images (>7 days old)..."
|
||||
docker image prune -a -f --filter "until=168h" --filter "dangling=false" 2>/dev/null || true
|
||||
|
||||
# 6. Truncate container log files that are bigger than 100MB
|
||||
echo "$LOG_PREFIX Checking container logs..."
|
||||
for logfile in /var/lib/docker/containers/*/*-json.log; do
|
||||
if [ -f "$logfile" ]; then
|
||||
size=$(stat -c%s "$logfile" 2>/dev/null || echo 0)
|
||||
if [ "$size" -gt 104857600 ]; then # 100MB
|
||||
echo "$LOG_PREFIX Truncating $(basename $logfile) ($(( size / 1048576 ))MB)"
|
||||
truncate -s 0 "$logfile"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# 7. Vacuum journald logs to 200MB
|
||||
echo "$LOG_PREFIX Vacuuming journal logs..."
|
||||
journalctl --vacuum-size=200M 2>/dev/null || true
|
||||
|
||||
# 8. Clear pip/npm caches that grow over time
|
||||
echo "$LOG_PREFIX Clearing stale caches..."
|
||||
rm -rf /root/.cache/pip/cache/html 2>/dev/null || true
|
||||
rm -rf /root/.cache/npm/_cacache 2>/dev/null || true
|
||||
|
||||
# 9. Report disk usage
|
||||
USAGE=$(df / | tail -1 | awk '{print $5}' | tr -d '%')
|
||||
FREE_GB=$(df -h / | tail -1 | awk '{print $4}')
|
||||
echo "$LOG_PREFIX Disk usage: ${USAGE}% (${FREE_GB} free)"
|
||||
|
||||
if [ "$USAGE" -gt "$MAX_DISK_PCT" ]; then
|
||||
echo "$LOG_PREFIX WARNING: Disk usage above ${MAX_DISK_PCT}%!"
|
||||
# More aggressive: remove ALL images not used by running containers
|
||||
echo "$LOG_PREFIX Aggressive prune: removing all unused images..."
|
||||
docker image prune -a -f 2>/dev/null || true
|
||||
fi
|
||||
|
||||
echo "$LOG_PREFIX Done."
|
||||
Reference in New Issue
Block a user