e33bc9143892bcf6f31b93ff039f570babc006ff
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Hermes
|
0721b1cb04 |
fix(security): audit-logger PII masking parity with unified logger (DC-110) [glm-grade=A]
audit-logger.js (StateManager write path into audit-log.json) starred only 6 sensitive keys at the middleware layer; email-bearing resource paths (/invites/<email>/accept), DC-048 details.userEmail attribution, and emails in non-sensitive body keys landed RAW — while the parallel unified-logger path has masked at every sink since DC-095. DC-110 closes the parity gap using the SAME canonical primitives (sa****@example.com): log() masks resource (maskEmailsInString) and deep-masks details (maskEmails) at the single write-point, covering middleware AND direct route calls. The security-event mirror now uses the masked entry.resource for target/message (judge round-1 fix-first: the raw parameter leaked emails into security-events.jsonl). logging.js exports maskEmails (export-only). maskEmails clones — caller details objects are never mutated. Judge: GLM-5.3 cold read (standing Sami authorization 2026-08-17; Codex quota dead until 2026-08-29). Round 1 (deleg_ebb83285) C fix-first — caught the event-store mirror leak. Round 2 (deleg_923f9076) after in-commit fix + mirror test: grade A, ship. Verdict URN urn:ump:mwtxoj6dbdq7bfeba3l2am2rgx34zjimcjde5f6sxz6tozsjbskq (GET readback verified: grade A, topic codex-judge-verdict). Deferred (judge-accepted): one-time scrub of historical raw-email lines in the live 16MB security-events.jsonl — queued follow-up. Tests: 123 suites / 2801 green (+6 DC-110 pins: resource+details mask, non-mutation, middleware e2e with *** survival, idempotence, no-email regression, masked mirror target/message). |
||
|
Hermes
|
9322831f1b | fix(security): quoted local-part email mask — strip delimiter quotes, split on last @ (DC-109) [glm-grade=A] | ||
|
Hermes
|
c429b8fdd7 |
fix(security): redact-on-rotate for error.log archive + README PII docs (DC-108) [glm-grade=B]
DC-095 masks emails at every live log sink; the rotated archive was the remaining belt-and-braces gap — any future sink that forgets masking would persist raw PII in error.log.1 for a full rotation cycle. appendErrorLog now scrubs the freshly rotated archive with the SAME canonical mask (sa****@example.com) via atomic rewrite (sibling .redact-<pid> temp, wx open preserving mode else 0600, fsync, rename). Scrub failure is caught and logged; the new error line is still appended. Stale crash-leftover .redact-<pid> temps are swept best-effort on every rotation. Also documents scripts/redact-log-pii.js usage in README (queue item e). Judge: GLM-5.3 cold read (deleg_e7cd7f8c), round-1 grade B ship — both nits addressed in-commit: stale-temp sweep (new 5th test pins it), quoted-local-part mask edge deferred as pre-existing DC-095 primitive. Tests: 122 suites / 2790 green (+5 DC-108 pins; was 2785 post-DC-107). |
||
|
Hermes
|
8d42eae6ac |
feat(maintenance): one-shot PII redaction tool for pre-DC-095 log files (DC-098) [glm-grade=A]
- scripts/redact-log-pii.js: atomic in-place email redaction reusing the canonical DC-095 masker (no second regex), dry-run/keep-raw modes, dir walk with skip-set, post-verify (exit 2 if raw addresses remain). - src/utils/logging.js: export EMAIL_RE/maskEmailAddress/maskEmailsInString (additive; no logger behavior change). - __tests__/redact-log-pii.test.js: 11 tests (shape, idempotence, clean-untouched, dry-run, keep-raw, skip-set, passthroughs, exit codes). - Judge: GLM-5.3 cold read, round-1 A/ship (deleg_4e684a18), URN urn:ump:7y2q5upoht7xq2mhlum764y2h36qpsgufyqsgx4cbijfpmfpmrcq. - Suite: 120/120 suites, 2751 tests green. |
||
|
Hermes
|
83ef84d218 |
feat(logging): central email PII masking across all log sinks [glm-grade=A]
DC-095: mask email addresses at every logger output choke point so raw
PII never reaches stdout/stderr, error.log, or audit-log.json regardless
of what a call site interpolates — msg strings, data payloads, error
messages/stacks, audit details, and error.log request lines (path/UA).
- Masked shape sa****@domain matches AuthProvider.maskEmail (UI-consistent)
- Bounded-quantifier regex: local {1,64} (incl. quoted local-parts),
domain {0,253}, TLD {2,24} — adversarial 40KB string 3.3s -> 17ms,
hostnames/versions/docker-refs untouched, idempotent under re-mask
- memo-Map recursion: DAG shared references get the same masked clone
(WeakSet seen-guard leaked the raw original on 2nd reference); cycles
resolve to in-progress clone
- Non-plain objects with own enumerable props cloned proto-preserving
(Object.create) so class-instance email fields are masked; Date/RegExp
pass through
- sanitize(): audit details mask email substrings in non-sensitive keys
(invite/auth POST bodies no longer land raw in audit-log.json)
- 18-test suite covers sinks + adversarial judge findings (ReDoS timing,
DAG, quoted locals, instances, request-line path/UA)
Judge: GLM-5.3 cold-read stand-in (Codex quota-dead until 2026-08-24,
substitution authorized by Sami 2026-08-17). Rounds C -> C -> A.
Verdict: urn:ump:zorj7vcrnw2t2jhhcp2g6wz4simvhyjdqwu2mjsifxlkzb2dwjmq
Suite: 117 suites / 2724 tests green.
|
||
|
Hermes
|
3137d4c16d |
[glm-grade=B] fix(logging): surface AggregateError causes + .cause chains in error.log (DC-056)
Live preflight at 2026-08-18T08:42Z surfaced a real entry in error.log:
[2026-08-18T06:49:03.345Z] [ERR] update:
context: {"imageName":"ipfs/kubo:latest"}
The line was terminated with a literal empty <message> because
AggregateError.message is empty by spec — registry-1.docker.io multi-A
timeouts (and any Promise.any / multi-fetch failure) leaked through with
no actionable signal. The only clue was a JSON context tail, and even that
didn't say WHY. Operators / incident-triage scripts that grep error.log by
line content couldn't tell the difference between a registry outage and
DNS resolution failure.
**Fix** (dashcaddy-api/src/utils/logging.js, +70 lines):
- describeErrorChain(err, depth, seen) flattens .errors[] (AggregateError)
and .cause chains into readable lines, each carrying Name [CODE]: message.
- writeErrorLog builds both the headline (replacing bare error.message with
the formatted chain[0]) and a tail diagnostic block listing chain[1..].
Backwards-compat preserved: headline still matches [ERR] ${ctx}: <head>.
- Cycle guard via WeakSet seen: pathological err.cause = err no longer
infinite-recurses on the error-path (round-1 GLM polish).
- Hard depth cap MAX_CHAIN_DEPTH=16: pathological deep chains truncate
with a marker, never crash writeErrorLog (round-1 GLM polish).
- Defensive head line for empty err.message: falls back to error.name
so AggregateError with no inline message still renders `Error` instead
of a literal empty after .
**Tests** (__tests__/utils-logging-aggregate-error.test.js, NEW, 209 lines):
13 cases covering plain Error, EPIPE code tag, custom subclass name,
empty message fallback, AggregateError (single + nested), .cause chain,
req field, extra JSON, separator invariant, circular .cause, depth-truncation,
circular .errors[].
GLM judge round 1 (deleg_59155c78, 43.77s): GRADE=B with 2 polish
suggestions (cycle guard + depth cap) — folded into the same commit per
conjoint-commit anti-pattern. Round 2: not needed (the polish is in).
Full suite: 89 suites / 1975 tests pass (+13 net new). ESLint clean.
|
||
|
Krystie
|
e99413150e |
[glm-grade=B] fix: dead dashboard WS, corrupted error.log, auth-polling storm, re-auth freeze + CVE bumps
Adversarial audit 2026-08-16 (GLM-5.3 delegate, 2 rounds, 141 tool calls):
P0-1: Dashboard WebSocket (/api/v1/ws) dead on EVERY boot since DC-076.
server.js passed module exports (DependencyManager class, {AutoRestartManager}
namespace, SSLMonitor class) instead of createApp()'s live instances — first
.on() threw ERR_INVALID_ARG_TYPE, catch swallowed it. Fix: app.locals.ctx
exposed in src/app.js; server.js passes all 8 real EventEmitter instances.
P0-2: error.log corrupted since 2026-07-14. errorMiddleware called
logError(FILE, SIZE, path, err, meta) — 5 args into a 3-arg wrapper —
logging 'Error: 5242880' garbage every ~60s and DISCARDING the real error
object. Fix: correct 3-arg call + legacy-shape guard in logErrorWrapper +
~74 log.error sites swept to pass real error objects (AST-verified scope-
safe 71/71, 29/29 modules load clean).
P0-3: auth-polling storm (stranded grade=B commit never landed in prod):
401/403 behind TOTP gate hammered /api/v1/services/status + SSE reconnect
every 2-8s, with misleading direct-probe fallback marking services 'up'.
Fix landed + B-round MEDIUM follow-up: TOTP re-auth success now clears
_dcAuthLost, resumes SSE (new _sseResume clears the latch), and refreshes.
Also: eslintignore static-sites/ (33→0 errors); nodemailer 8→9.0.5 and
sharp 0.33→0.35.3 (3 high CVEs killed; jest green on new majors);
dockerode@5/uuid deferred (semver-major, Docker API surface).
Verification: 80/80 suites, 1837/1837 tests; ESLint 0 errors/743 warnings;
node --check all changed files; bundles rebuilt + SW cache bumped.
Judges: Codex quota-dead until Aug 19 (verified live) — GLM adversarial
delegate per operator directive 2026-08-07. Round 1: 98-call mechanical
verification (timed out pre-verdict). Round 2 (this grade): B, one MEDIUM
(re-auth freeze) — fixed in this commit as prescribed.
|
||
|
Hermes
|
f750d01ed0 |
DC-039: route all module file defaults through platformPaths.dataDir
Multiple modules derived file paths from __dirname, which is unstable in two
ways: (1) it moves whenever the file is reorganized under src/, and (2) it
points to the in-container source dir /app/src/<x> in production, which is
not bind-mounted, so writes would silently land in the image layer.
Affected modules (10 files): backup-manager, resource-monitor, update-manager,
docker-security, audit-logger, bundled-workflows, port-lock-manager, logging,
error-handler, license-keygen, plus crypto-utils and credential-manager which
already had multi-candidate resolvers but no centralised fallback.
Introduced platformPaths.dataDir (derived from SERVICES_FILE/CONFIG_FILE/
DNS_CREDENTIALS_FILE env vars when set, else path.dirname(servicesFile)) so
every module resolves the same canonical data directory. Each module now
fans the runtime files into the data dir while preserving per-file env-var
overrides for custom deployments.
Why a single resolver:
- one place to swap the default path scheme in v2.x without chasing
hardcoded __dirname joins
- a single source-of-truth for tests, backup tools, and the soon-to-be
added single-volume migration script
- prevents the class of DC-033 (self-updater 0.0.0) bugs where __dirname
drift in a subdirectory silently loses runtime state
Also fixed:
- audit-logger: AUDIT_LOG_FILE default was /app/src/security/audit-log.json
(writable in dev, image-layer in production). Now /app/data/audit-log.json
via platformPaths.dataDir, matching logging.js's same file. Same physical
path, no behavior change for callers that already set AUDIT_LOG_FILE.
- logging.js: LOG_DIR was __dirname (src/utils/) — error.log and
audit-log.json were being written into the source tree. Now
platformPaths.dataDir, matching every other persistent file.
- error-handler.js: ERROR_LOG_FILE hard-coded to __dirname/error.log
(src/utilities/error.log), redundant with logging.js's own default.
Now platformPaths.dataDir/error.log.
- host-registry / event-store / event-workers: simplified the
'platformPaths.dataDir || path.join(__dirname, ../../data)' pattern
to just platformPaths.dataDir (the legacy fallback is no longer
reachable — services.json lives at dataDir/services.json now).
- public-routes-drift.test.js: added 'routes/security.js' to the
direct-mount list so the /api/v1/security/events/ingest and
/api/v1/security/events/batch entries in PUBLIC_ROUTES are
recognized as mounted (was missing — fixed DC-044's drift-detection
test gap).
Tests: 1214/1214 pass (0 new failures, 1 new test for the corrected route
mount detection path). ESLint: 146 warnings + 4 errors — same baseline as
HEAD (no new warnings or errors introduced; one pre-existing require-await
on readline was removed as a drive-by in event-workers.js since the module
uses line-level fs reads, not readline). Container config files like
audit-log.json, container-stats.json, and workflow-history.json still
exist on the running container's image layer — Docker will pick up the
new defaults on the next recreate (the update path already moves
services.json+config.json+credentials.json via the data bind mount).
|
||
|
Hermes
|
1c8f55edc1 |
DC-018: return writeErrorLog promise from Logger.error()
Logger.error() called this._log('error',...) but dropped the return value.
_log returns the writeErrorLog(...) promise for error level, so every
await logError(...)/await log.error(...) caller was awaiting undefined —
the error.log disk write was fire-and-forget. This caused:
1. __tests__/logging.test.js 'captures request context' to flake in the
full suite (test read error.log before the un-awaited appendFile
completed; passed in isolation).
2. In production, 6 route handlers + the global boundAsyncHandler error
catcher all await logError(...) expecting the write to flush — error
entries could be lost on fast process exit/restart.
Fix: add 'return' so the promise propagates. Verified: logging test
passes 10/10 full-suite runs (was ~1/6 failure rate). No behavior change
for debug/info/warn (they never wrote to disk).
|
||
|
Hermes
|
bf515e5415 |
DC-010: convert res.json({success:true,...}) → ok(res, {...}) in 3 routes; refactor config/context/utils
Routes converted: browse.js, logs.js, sites.js. Each factory dep now receives the ok() response helper from src/utils/responses.js. Wired through the route factory destructuring in src/app.js so the helper is available wherever the route needs to send a success response. Also touched (incidental cleanup landed in the same patch because the cron session was exploring how ok/errorResponse are composed): - src/config/site.js: 28 lines net — response shape consistency - src/context/caddy.js, dns.js: 34 lines net — minor refactors - src/utils/http.js, logging.js: 46 lines net — ESLint hygiene and helper plumbing 750/750 tests pass, 0 new ESLint warnings. |
||
|
Krystie
|
f71e5c52d4 |
feat(api): unify logger — single source of truth for logs, errors, audit
Cherry-picks the unified logger design from the 171c1ad WIP (which Hermes
signed off on as 'Ship this') and applies all Hermes review fixes
(krystie-wip/logger-refactor, 2026-06-15).
src/utils/logging.js is now the single entry point for:
- log.info / log.warn / log.error / log.debug (with level filtering,
color-coded dev output, JSON prod output)
- log.audit() / log.auditMiddleware() (audit-log.json + SKIP_PATHS
+ sensitive-key redaction)
- logError(ctx, err, extra) (writes error.log with
rotation, request context extraction)
- safeErrorMessage(err) (DC-200 port collision,
No-such-container, ECONNREFUSED, etc.)
Existing src/security/audit-logger.js kept untouched — routes/errorlogs.js
still uses auditLogger.query/clear, no callers migrated.
Hermes' must-fixes (all addressed):
[1] Syntax error on logger.js:401 — old logger.js at repo root is gone;
refactored src/utils/logging.js is the new home, no Chinese IME bug.
[2] /health/live and /health/ready endpoints — untouched in src/app.js.
[3] Tests — added __tests__/logging.test.js (18 tests, all pass) covering
module loads, level filtering, sanitize/audit/auditMiddleware,
safeErrorMessage, and logError. Full suite: 897/897 pass across 31
suites (was 879 + 18 new).
Hermes' should-fixes:
[4] asyncHandler signature — KEPT 3-arg (logError, fn, context). 49 route
files still call it this way; src/app.js's boundAsyncHandler unchanged.
[5] platformPaths.pkiRootCert — UNTOUCHED, still used in src/app.js.
[6] Five managers (Dependency, AutoRestart, ConfigDrift, SSL, DNS) — ALL
FIVE still initialized at server boot (verified via test).
[7] ok(res, ...) helper — UNTOUCHED, all routes still use it.
[8] Network-intel helpers (isPrivateLan, isTailscaleIP) — UNTOUCHED in
src/app.js, no duplicate inline logic added.
- setLevel() now updates both GLOBAL_LEVEL and the singleton log._level,
so level-filter tests don't pollute later tests.
- Logger.audit() and Logger.error() now return promises so await works.
- Logger._log() awaits writeErrorLog so callers using await can rely on
the error.log being flushed.
- safeErrorMessage() handles null/undefined explicitly (regression fix —
String(null) returned 'null' before, now returns 'An internal error
occurred').
- src/app.js boundLogError() simplified to 3-arg form matching the
unified logError(ctx, err, extra) signature.
- createLogger(level) alias exported so existing src/app.js callers work.
- logError, safeErrorMessage, LOG_LEVELS still exported.
- asyncHandler still imported from ./utils/async-handler, not from logging.
- No changes to routes/* (audit-logger.js still consumed unchanged).
- jest: 897/897 tests pass across 31 suites
- node -e "require('./src/app.js')" loads cleanly
- node server.js boots through full init (all 5 managers start)
- Color-coded logger output visible in dev mode (no NODE_ENV)
- JSON output in production mode (NODE_ENV=production)
|
||
|
Krystie
|
fa7a78388a |
refactor(utils): Extract utilities from server.js
- Create src/utils/http.js - fetchT and HTTP helpers - Create src/utils/logging.js - Structured logging and error logging - Create src/utils/responses.js - Standard API responses - Create src/utils/async-handler.js - Async wrapper with error handling - Create src/utils/index.js - Consolidated exports Removes scattered helper functions from server.js |