Actions now retry up to 3 times with 2/4/8s exponential backoff before
giving up. Logs each retry attempt with attempt count. exhaustedRetries
field in failure result shows total attempts made.
All 1540 tests pass.
Replaced all 18 console calls in src/recipes/bundled-workflows.js with
log.info/warn/error tagged 'workflow'. Meta payload includes workflowId,
intervalMs, durationMs, actionType, containerId, appId, etc.
1539/1539 Jest tests pass. ESLint clean (0 new warnings).
The original DC-044 fix (b492e1c) repaired servicesStateManager.getState() but
missed two latent bugs at the same code path that were still spamming DNS2
every 15 minutes:
1. notify-on-failure fired unconditionally. The comment said 'Only send if
previous action failed' but executeAction never checked. Every
health-check-on-interval cycle ran notify regardless of outcome.
2. {{serviceId}} template never interpolated. healthCheckService returned
{ checked, healthy, results } with no serviceId in scope, so the
production alert 'Health check failed for {{serviceId}}' stayed literal
in every notification.
3. checkContainerHealth compared info.State.Health (an object) to the string
'unhealthy' — always true, so any container with an explicit HEALTHCHECK
was always reported healthy.
Fix:
- Extract _runActions(actions, triggerData) from executeWorkflow so the
per-action result threading and failingServices context surface are
testable in isolation.
- Gate notify-on-failure on previousResult.success === false. Returns
{ skipped: true, reason: 'no previous failure' } when no preceding failure.
- healthCheckService throws an Error with .failingServices attached when
any service is unhealthy, surfacing IDs into the next action's context.
- checkContainerHealth now reads info.State.Health.Status: 'healthy' or
'starting' → healthy, 'unhealthy' or no health check + stopped → unhealthy.
- Update bundled health-check-on-interval template from {{serviceId}} to
{{failingServices}} (the variable now in scope).
Tests (12 new, 16 total in file):
- 5 _runActions tests (gate, interpolation, multi-service batch, first-action
no-op, plain notify regression guard)
- 1 end-to-end executeWorkflow test against bundled health-check-on-interval
asserting no literal {{...}} tokens reach notification.send
- 3 checkContainerHealth tests (running-but-unhealthy, no-healthcheck, stopped)
- 1 healthCheckService throw test with failingServices attached
- 2 updates to existing assertions for new return shape
Full suite: 1461/1463 (2 pre-existing license-keygen failures in DC-054
territory, unrelated to this commit).
Co-graded: Codex B urn:ump:b2nzzoulodwsullt3rhz4mtzou7fqgwiuoyrzxho67gdpwx3uvaa
The bundled-workflows.js:310 call site used a non-existent .getState()
method AND forgot to await. The Promise short-circuited via '|| []' to an
empty array, so every health-check-on-interval workflow ran every 5 min
reporting 'Action health-check failed: servicesStateManager.getState is
not a function' while silently iterating over zero services. Visible on
both DNS2 (production) and dc-contabo-de (test server) — same code, same
bug, same log spam.
Fix: 'await servicesStateManager.read().catch(() => []) || []' — uses the
actual async method, returns empty array on read() failure (corrupt or
missing state file shouldn't break the workflow), preserves the original
short-circuit guard.
New regression test __tests__/bundled-workflows-health-check.test.js with
5 cases:
1. uses .read() not the non-existent .getState() — does not throw
2. returns checked/healthy counts from read() output
3. gracefully degrades if read() throws — empty services list, no crash
4. servicesStateManager absent on ctx → no crash, empty result
5. single service (non-template serviceId) path still works
Tests: 1219/1219 pass (1214 baseline + 5 new). ESLint: clean for the new
file. Test fixture note: had to clearInterval the constructor's
scheduledJobs so Jest could exit cleanly — scheduled workflows are not
under test here.
Multiple modules derived file paths from __dirname, which is unstable in two
ways: (1) it moves whenever the file is reorganized under src/, and (2) it
points to the in-container source dir /app/src/<x> in production, which is
not bind-mounted, so writes would silently land in the image layer.
Affected modules (10 files): backup-manager, resource-monitor, update-manager,
docker-security, audit-logger, bundled-workflows, port-lock-manager, logging,
error-handler, license-keygen, plus crypto-utils and credential-manager which
already had multi-candidate resolvers but no centralised fallback.
Introduced platformPaths.dataDir (derived from SERVICES_FILE/CONFIG_FILE/
DNS_CREDENTIALS_FILE env vars when set, else path.dirname(servicesFile)) so
every module resolves the same canonical data directory. Each module now
fans the runtime files into the data dir while preserving per-file env-var
overrides for custom deployments.
Why a single resolver:
- one place to swap the default path scheme in v2.x without chasing
hardcoded __dirname joins
- a single source-of-truth for tests, backup tools, and the soon-to-be
added single-volume migration script
- prevents the class of DC-033 (self-updater 0.0.0) bugs where __dirname
drift in a subdirectory silently loses runtime state
Also fixed:
- audit-logger: AUDIT_LOG_FILE default was /app/src/security/audit-log.json
(writable in dev, image-layer in production). Now /app/data/audit-log.json
via platformPaths.dataDir, matching logging.js's same file. Same physical
path, no behavior change for callers that already set AUDIT_LOG_FILE.
- logging.js: LOG_DIR was __dirname (src/utils/) — error.log and
audit-log.json were being written into the source tree. Now
platformPaths.dataDir, matching every other persistent file.
- error-handler.js: ERROR_LOG_FILE hard-coded to __dirname/error.log
(src/utilities/error.log), redundant with logging.js's own default.
Now platformPaths.dataDir/error.log.
- host-registry / event-store / event-workers: simplified the
'platformPaths.dataDir || path.join(__dirname, ../../data)' pattern
to just platformPaths.dataDir (the legacy fallback is no longer
reachable — services.json lives at dataDir/services.json now).
- public-routes-drift.test.js: added 'routes/security.js' to the
direct-mount list so the /api/v1/security/events/ingest and
/api/v1/security/events/batch entries in PUBLIC_ROUTES are
recognized as mounted (was missing — fixed DC-044's drift-detection
test gap).
Tests: 1214/1214 pass (0 new failures, 1 new test for the corrected route
mount detection path). ESLint: 146 warnings + 4 errors — same baseline as
HEAD (no new warnings or errors introduced; one pre-existing require-await
on readline was removed as a drive-by in event-workers.js since the module
uses line-level fs reads, not readline). Container config files like
audit-log.json, container-stats.json, and workflow-history.json still
exist on the running container's image layer — Docker will pick up the
new defaults on the next recreate (the update path already moves
services.json+config.json+credentials.json via the data bind mount).
After the DC-005 module reorganization (41 files moved into src/ subdirs),
138 test suites failed because the refactor script's path-rewrite logic
missed three categories:
1. Files inside src/ doing 'require("./src/...")' — should be 'require("../...")'
2. Files in src/X/Y/ doing 'require("../../../src/...")' — should be 'require("../../...")'
3. Test files in __tests__/ with leftover 'require("../../../src/...")' paths
Root cause: the original refactor script ran before all files were moved,
so it computed relative paths against stale filesystem state.
Result:
- 30/30 test suites pass
- 879/879 tests pass (was: 18/30 suites, 614/687 tests)
Also fixed:
- routes/apps/restore.js: wrong responses import path
- routes/*/*.js: '../../src/utilities/X' → '../src/utilities/X' (depth 2 routes)
Hermes