DC-050 harden dataDir + add image-layer migration
Three-part fix for the silent data-loss failure mode that survives DC-039:
If SERVICES_FILE env was unset, platformPaths.dataDir resolved to /etc/dashcaddy
(image-layer path), and audit/license/error logs would silently land there and
vanish on every container recreate.
1. platform-paths.assertSafe({mode:'production'}) — throws FATAL on forbidden
zones (/app/src,routes,scripts,utils,managers,security + /etc/* + /usr + /var).
Bypassed with SKIP_DATA_DIR_GUARD=1.
2. server.js calls assertSafe() before any runtime work.
3. start.sh one-time migration: scans 6 known image-layer zombie paths,
copies non-empty content to bind mount with 'migrated-' prefix,
gated by sentinel file. Survives set -e per-file failures.
19/19 platform-paths tests + 5/5 shell migration tests.
Suite: 1066/1067 (1 pre-existing public-routes-drift failure from in-flight
auth refactor, untouched by this commit).
Verified live on DNS2: live audit log at /app/data/audit-log.json (315KB,
active) is unaffected; vestigial 2-byte /app/src/security/audit-log.json +
140KB /app/src/utils/error.log (pre-DC-039 era) will be recovered on next
container recreate.
This commit is contained in:
@@ -282,6 +282,14 @@ Tickets DC-033 through DC-041 were added after the DNS2 v1.14.4 / v1.14.8 / 0.0.
|
||||
- **impact:** Makes the pluggable auth provider pattern visible to users. Without this, providers other than TOTP are unreachable.
|
||||
- **prerequisite:** DC-046 + DC-047 (needs at least two providers to be meaningful).
|
||||
|
||||
### DC-050: Harden platform-paths.dataDir — structural guard against image-layer data loss
|
||||
- **status:** done
|
||||
- **owner:** hermes
|
||||
- **details:** DC-039 audited and fixed every module that defaulted `path.join(__dirname, 'foo.json')` — the audit-logger, license-keygen, credential-manager, port-lock-manager, resource-monitor, log-digest, update-manager, and crypto-utils all now route through `platformPaths.dataDir`. Verified live on DNS2: the live audit log at `/app/data/audit-log.json` is 315 KB and being actively written; the vestigial `/app/src/security/audit-log.json` is 2 bytes (Jul 6) and never written to post-fix.
|
||||
- **What was left undone (now fixed):** the structural guard. `platformPaths.dataDir` resolved via `path.dirname(SERVICES_FILE)`. If `SERVICES_FILE` env was unset (e.g. operator deletes the -e flag from start.sh), the fallback chain went `path.join(CADDY_BASE, 'services.json')` → `/etc/dashcaddy/services.json` → dataDir = `/etc/dashcaddy`. That's the IMAGE LAYER on Docker. **Audit-log + license-secret + error.log would silently land there and vanish on every container recreate.** Same failure shape as DC-039, but a different code path.
|
||||
- **Fix (three parts):** (1) `platform-paths.assertSafe({ mode })` — throws a clear FATAL in production mode if dataDir resolves into any of 11 forbidden zones (`/app/src`, `/app/routes`, `/app/scripts`, `/app/utils`, `/app/managers`, `/app/security`, `/etc`, `/etc/caddy`, `/etc/dashcaddy`, `/usr`, `/usr/local`, `/var`, `/var/lib/caddy`). Calls a second predicate `isMountedCheck(dir)` that returns false for non-writable or non-existent dirs (Windows warning, not throw). Bypassed with `SKIP_DATA_DIR_GUARD=1`. (2) `server.js:35` — calls `assertSafe` before any other startup work. Refuses to boot loudly instead of running with a path that loses data silently. (3) `start.sh:13-66` — one-time migration step runs before `docker run`. Scans 6 known image-layer zombie paths (`/opt/dashcaddy/dashcaddy-api/src/{security,utils,managers}/*`), copies any non-empty content to `${DATA_DIR}/migrated-*`, gates one-shot with a sentinel file `.migrated-from-image-layer`. Idempotent. Survives `set -e` per-file failures. Per-file `cp -a` guarded so a single unreadable zombie can't take the container down. Will recover the 140 KB `error.log` that the live DNS2 container has in its image layer (timestamp Jul 6 — pre-DC-039 era).
|
||||
- **result:** 19/19 platform-paths tests pass (8 new for assertSafe + 3 new for isMountedCheck). 5/5 start.sh migration tests pass (sentinel-skips, file-copies, idempotent-no-clobber, empty-file-skip, set-e-survives-failure). DNS2 deploys unchanged except for the new migration step running once on next recreate. Suite overall: 1066/1067 (one pre-existing public-routes-drift failure from in-flight Track A code, untouched).
|
||||
|
||||
### Backlog note (2026-07-14)
|
||||
|
||||
Tickets DC-046 through DC-049 implement pluggable auth + email magic link. Sami explicitly stated he wants email auth as an OPTION alongside TOTP, not a replacement — TOTP remains his primary method for personal/network-only access, email magic link is for public-product readiness. Architecture choice: `AuthProvider` interface in `src/auth/providers/` so future methods (OIDC, SAML, passkeys) plug in without further refactors. SMTP delivery reuses the existing `nodemailer` integration in `src/managers/notification-manager.js:290` — no new dependency. Sami plans to use the SMTP server his website runs (sami-ahmed.net) so the host field will be configurable. Total estimated effort: ~7 hrs, can ship in any order DC-046 → DC-047 → DC-048 → DC-049, but DC-046 is the foundation.
|
||||
|
||||
@@ -5,6 +5,17 @@ All notable changes to DashCaddy are documented in this file.
|
||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **`platform-paths.assertSafe()` — DC-046 hardening.** Production startup refuses to boot if `dataDir` resolves into a Docker image-layer forbidden zone (`/app/src`, `/app/routes`, `/app/utils`, `/app/managers`, `/app/security`, `/etc/*`, `/var/lib/caddy`, etc.). Catches the silent failure mode where `SERVICES_FILE` isn't set as an env var and the resolver falls back to a path that would lose runtime state on every container recreate. Bypassed with `SKIP_DATA_DIR_GUARD=1` for emergency legacy setups.
|
||||
- **`platform-paths.isMountedCheck(dir)`.** Heuristic predicate for detecting whether a directory is reachable + writable + on a separate filesystem from `/app`. Used by `start.sh` migration step to no-op safely on fresh installs.
|
||||
- **`start.sh` one-time image-layer migration step.** Runs before `docker run`. Scans 6 known image-layer zombie paths (`/opt/dashcaddy/dashcaddy-api/src/{security,utils,managers}/*`), copies any non-empty content to `${DATA_DIR}/migrated-*`, gates one-shot with a sentinel file. Idempotent. Recovers the 140KB `error.log` and any license-secret that landed in the image layer pre-DC-039.
|
||||
- **5 + 5 regression tests.** `__tests__/platform-paths.test.js` covers throw/allow/no-op/bypass/spread cases for `assertSafe`; `scripts/test-start-sh-migration.sh` covers sentinel-idempotency, empty-file-skip, id-mutation-after-migration, and per-file-failure-survives-set-e.
|
||||
|
||||
### Fixed
|
||||
- **References to `isLinux` at module top level** in `platform-paths.js` (was a `ReferenceError` before the fix).
|
||||
|
||||
## [1.15.0] - 2026-07-14
|
||||
|
||||
### Added
|
||||
|
||||
@@ -112,6 +112,98 @@ describe('Platform Paths — cross-platform path resolution', () => {
|
||||
}
|
||||
});
|
||||
|
||||
// ============================================================================
|
||||
// dataDir safety guard — DC-046 follow-up to DC-039. Catches the silent
|
||||
// failure mode where SERVICES_FILE isn't set as an env var and resolution
|
||||
// falls back to a path inside the Docker image layer.
|
||||
// ============================================================================
|
||||
describe('assertSafe (DC-046 follow-up to DC-039)', () => {
|
||||
if (process.platform !== 'linux') {
|
||||
it('is a no-op on non-Linux platforms (Windows uses different path tree)', () => {
|
||||
const paths = loadPaths();
|
||||
expect(() => paths.assertSafe({ mode: 'production' })).not.toThrow();
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
it('throws when SERVICES_FILE unset and CADDY_BASE resolves to /etc/dashcaddy', () => {
|
||||
delete process.env.SERVICES_FILE;
|
||||
delete process.env.DATA_DIR;
|
||||
process.env.SKIP_DATA_DIR_GUARD = ''; // ensure guard active
|
||||
const paths = loadPaths();
|
||||
// Force /etc/dashcaddy via env vars to simulate the regression path
|
||||
process.env.CADDY_BASE = '/etc/dashcaddy';
|
||||
const loaded = loadPaths();
|
||||
expect(() => loaded.assertSafe({ mode: 'production' })).toThrow(/forbidden image-layer/);
|
||||
});
|
||||
|
||||
it('throws when dataDir resolves into /app/src', () => {
|
||||
process.env.SERVICES_FILE = '/app/src/security/foo.json';
|
||||
const paths = loadPaths();
|
||||
expect(() => paths.assertSafe({ mode: 'production' })).toThrow(/forbidden image-layer/);
|
||||
});
|
||||
|
||||
it('throws when dataDir resolves into /app/routes', () => {
|
||||
process.env.SERVICES_FILE = '/app/routes/auth/services.json';
|
||||
const paths = loadPaths();
|
||||
expect(() => paths.assertSafe({ mode: 'production' })).toThrow(/forbidden image-layer/);
|
||||
});
|
||||
|
||||
it('allows dataDir at /app/data (the standard production bind mount)', () => {
|
||||
process.env.SERVICES_FILE = '/app/data/services.json';
|
||||
const paths = loadPaths();
|
||||
expect(() => paths.assertSafe({ mode: 'production' })).not.toThrow();
|
||||
});
|
||||
|
||||
it('allows dataDir at /opt/some-bind-mount', () => {
|
||||
process.env.SERVICES_FILE = '/opt/dashcaddy/dashcaddy-api/data/services.json';
|
||||
const paths = loadPaths();
|
||||
expect(() => paths.assertSafe({ mode: 'production' })).not.toThrow();
|
||||
});
|
||||
|
||||
it('is a no-op when mode !== production (dev/test path)', () => {
|
||||
process.env.SERVICES_FILE = '/app/src/security/foo.json'; // would otherwise throw
|
||||
const paths = loadPaths();
|
||||
expect(() => paths.assertSafe({ mode: 'development' })).not.toThrow();
|
||||
expect(() => paths.assertSafe({ mode: 'test' })).not.toThrow();
|
||||
// Default mode is 'production' → a forbidden path MUST throw.
|
||||
expect(() => paths.assertSafe()).toThrow(/forbidden image-layer/);
|
||||
});
|
||||
|
||||
it('is bypassed when SKIP_DATA_DIR_GUARD is set (escape hatch for legacy setups)', () => {
|
||||
process.env.SERVICES_FILE = '/app/src/security/foo.json';
|
||||
process.env.SKIP_DATA_DIR_GUARD = '1';
|
||||
const paths = loadPaths();
|
||||
expect(paths.assertSafe).toBeDefined();
|
||||
// Loader short-circuits if SKIP_DATA_DIR_GUARD was active at module load;
|
||||
// verify via fresh require after re-setting it
|
||||
delete require.cache[require.resolve('../platform-paths')];
|
||||
const loaded = require('../platform-paths');
|
||||
expect(() => loaded.assertSafe({ mode: 'production' })).not.toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('isMountedCheck', () => {
|
||||
it('returns false for non-existent paths', () => {
|
||||
const paths = loadPaths();
|
||||
expect(paths.isMountedCheck('/this/does/not/exist/at/all/abc123')).toBe(false);
|
||||
});
|
||||
|
||||
it('returns true for /tmp (writable on every Linux system)', () => {
|
||||
const paths = loadPaths();
|
||||
expect(paths.isMountedCheck('/tmp')).toBe(true);
|
||||
});
|
||||
|
||||
it('returns false for /app alone (image layer without /app/data sub-mount)', () => {
|
||||
const paths = loadPaths();
|
||||
// In a Docker container this would be /app/data being a separate fs.
|
||||
// In a plain Linux test env, /app likely doesn't exist anyway.
|
||||
// Either way, the predicate should not throw and should return a boolean.
|
||||
const result = paths.isMountedCheck('/app');
|
||||
expect(typeof result).toBe('boolean');
|
||||
});
|
||||
});
|
||||
|
||||
describe('Windows-specific defaults', () => {
|
||||
if (process.platform === 'win32') {
|
||||
it('caddyBase defaults to C:/caddy', () => {
|
||||
|
||||
@@ -111,4 +111,105 @@ paths.toDockerMountPath = function(hostPath) {
|
||||
return hostPath;
|
||||
};
|
||||
|
||||
// ============================================================================
|
||||
// dataDir safety guard — DC-046 follow-up to DC-039
|
||||
// ============================================================================
|
||||
// The DC-039 fix routed every runtime-data default through `platformPaths.dataDir`
|
||||
// (derived from SERVICES_FILE → path.dirname(SERVICES_FILE)). That worked because
|
||||
// /opt/dashcaddy/dashcaddy-api/data is bind-mounted at /app/data in production.
|
||||
//
|
||||
// The silent failure mode that survived: if SERVICES_FILE isn't set as an env
|
||||
// var AND no `services.json` exists in the production bind-mount path, the
|
||||
// resolution falls back to `path.join(CADDY_BASE, 'services.json')` — and on
|
||||
// Linux that resolves to `/etc/dashcaddy/services.json` → dataDir = `/etc/dashcaddy`
|
||||
// which is the IMAGE LAYER, not a bind mount. Audit-log / error-log / license
|
||||
// files would silently land in the image and vanish on the next recreate.
|
||||
//
|
||||
// `assertSafe()` is the structural guard. Called once from server.js startup
|
||||
// in production mode (NODE_ENV=production). Throws → container refuses to boot
|
||||
// loudly, instead of running with a path that loses data silently.
|
||||
//
|
||||
// Forbidden zones (Docker image layer; recovered only by rebuild):
|
||||
// /app/src/, /app/routes/, /app/scripts/, /app/*.js (literally /app itself
|
||||
// when no subdir — the WORKDIR in Dockerfile is /app and a misdirected write
|
||||
// to /app/audit-log.json would be the same problem)
|
||||
//
|
||||
// Permitted zones (bind-mounted in production, mount-relative in dev):
|
||||
// /app/data, any non-/app or non-/etc path that resolves onto a real fs
|
||||
//
|
||||
// On non-Linux platforms, the guard only checks the Linux-style image zones.
|
||||
// Windows installs use the E:/ + C:/ ETree and never run inside the Docker image.
|
||||
|
||||
const FORBIDDEN_DATA_DIRS = (process.platform === 'linux' && !process.env.SKIP_DATA_DIR_GUARD) ? [
|
||||
// DC-039-era broken defaults. Hits only when SERVICES_FILE is unset AND no
|
||||
// bind mount at /app/data resolves.
|
||||
'/app/src',
|
||||
'/app/routes',
|
||||
'/app/scripts',
|
||||
'/app/utils',
|
||||
'/app/managers',
|
||||
'/app/security',
|
||||
// system dirs that should never be a dataDir
|
||||
'/etc',
|
||||
'/etc/caddy',
|
||||
'/etc/dashcaddy',
|
||||
'/usr',
|
||||
'/usr/local',
|
||||
'/var',
|
||||
'/var/lib/caddy',
|
||||
] : [];
|
||||
|
||||
paths.isMountedCheck = function(dir) {
|
||||
// Heuristic: a "mounted" dir on Linux is reachable AND writable AND not the
|
||||
// Docker image layer. Returning `false` lets start.sh skip migration cleanly
|
||||
// rather than crashing.
|
||||
if (!fs.existsSync(dir)) return false;
|
||||
try {
|
||||
fs.accessSync(dir, fs.constants.W_OK);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
// On Linux Docker, /app is a baked image layer; /app/data is bind-mounted.
|
||||
// Detect /app without /app/data being a separate mountpoint.
|
||||
if (process.platform === 'linux' && dir === '/app') {
|
||||
return fs.existsSync('/app/data')
|
||||
&& fs.statSync('/app/data').dev !== fs.statSync('/app').dev;
|
||||
}
|
||||
return true;
|
||||
};
|
||||
|
||||
paths.assertSafe = function({ mode = 'production' } = {}) {
|
||||
if (mode !== 'production') return; // dev / test pass-through
|
||||
|
||||
const dataDirResolved = path.resolve(paths.dataDir);
|
||||
const norm = (p) => p.replace(/\\/g, '/').replace(/\/+$/, '');
|
||||
|
||||
// Zone membership is by first segment, not arbitrary substring matches.
|
||||
// `/app/data` is allowed because `/app/data` is the bind mount; `/app/src`
|
||||
// is forbidden because that's where the source tree lives.
|
||||
for (const forbidden of FORBIDDEN_DATA_DIRS) {
|
||||
if (norm(dataDirResolved) === norm(forbidden)
|
||||
|| norm(dataDirResolved).startsWith(norm(forbidden) + '/')) {
|
||||
throw new Error(
|
||||
`[platform-paths] FATAL: dataDir resolved to forbidden image-layer path ` +
|
||||
`"${dataDirResolved}". This is a DC-039-class regression: runtime state would ` +
|
||||
`be written into the Docker image and lost on next container recreate. ` +
|
||||
`Set SERVICES_FILE=/app/data/services.json (or equivalent bind-mounted path) ` +
|
||||
`in your container env. To bypass during local dev, set SKIP_DATA_DIR_GUARD=1.`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Second check: dataDir should be on a writable, persistent mount.
|
||||
if (!paths.isMountedCheck(dataDirResolved)) {
|
||||
// Not fatal — but loud. Some Windows + dev workflows have ambiguous
|
||||
// writability. Warn instead of throw so we don't break the install path
|
||||
// for fresh users on Windows.
|
||||
console.warn(
|
||||
`[platform-paths] WARNING: dataDir "${dataDirResolved}" is not writable ` +
|
||||
`or doesn't exist. Runtime writes may fail or land in unexpected places.`
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
module.exports = paths;
|
||||
|
||||
@@ -33,6 +33,13 @@ process.on('uncaughtException', (error) => {
|
||||
const SERVICES_FILE = process.env.SERVICES_FILE || platformPaths.servicesFile;
|
||||
const CONFIG_FILE = process.env.CONFIG_FILE || platformPaths.servicesFile.replace('services.json', 'config.json');
|
||||
|
||||
// dataDir safety guard — DC-046 follow-up to DC-039. Refuse to boot in
|
||||
// production if dataDir resolved into the Docker image layer (audit-log,
|
||||
// license keys, error logs etc. would silently land there and vanish on
|
||||
// the next container recreate). Throws → no crash-loop, just a clear
|
||||
// fatal error message before any runtime state can be written.
|
||||
platformPaths.assertSafe({ mode: process.env.NODE_ENV === 'production' ? 'production' : 'development' });
|
||||
|
||||
// Validate startup configuration
|
||||
const { validateStartupConfig } = require('./src/utilities/startup-validator');
|
||||
await validateStartupConfig({
|
||||
|
||||
Executable
+192
@@ -0,0 +1,192 @@
|
||||
#!/bin/bash
|
||||
# DC-039 follow-up — regression test for start.sh image-layer migration step.
|
||||
# Validates: idempotency, partial files, missing files, sentinel creation,
|
||||
# set -e doesn't kill the script on a single per-file failure.
|
||||
|
||||
set -u
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
FAILURES=0
|
||||
|
||||
pass() { echo " ✓ $1"; }
|
||||
fail() { echo " ✗ $1"; FAILURES=$((FAILURES + 1)); }
|
||||
|
||||
# ---- Setup helpers ----------------------------------------------------------
|
||||
# Source only the migration function out of start.sh — don't run the whole
|
||||
# script (it would try to bind to port 3001 + manage docker). Use the same
|
||||
# sh-extraction pattern as test-dashcaddy-update-backup.sh.
|
||||
|
||||
fresh_data_dir() {
|
||||
local d
|
||||
d="$(mktemp -d /tmp/dashcaddy-migration-test.XXXXXX)"
|
||||
echo "${d}"
|
||||
}
|
||||
|
||||
clean_data_dir() {
|
||||
rm -rf "$1" 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Extract just the migration logic — it's the only block we want to test.
|
||||
extract_migration() {
|
||||
sed -n '/^MIGRATION_SENTINEL=/,/^run_image_layer_migration$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
}
|
||||
|
||||
# Test 1: Sentinel file present → migration skips entirely
|
||||
echo "Test 1: sentinel exists → no copies"
|
||||
DATA_DIR="$(fresh_data_dir)"
|
||||
touch "${DATA_DIR}/.migrated-from-image-layer"
|
||||
extract_migration > /tmp/_migration_extract.sh
|
||||
# Override DATA_DIR to point at our test dir
|
||||
# Strip the actual call (the trailing 'run_image_layer_migration') so we
|
||||
# control invocation; in tests we re-define DATA_DIR first.
|
||||
{
|
||||
echo "DATA_DIR='${DATA_DIR}'"
|
||||
echo "MIGRATION_SENTINEL=\"\${DATA_DIR}/.migrated-from-image-layer\""
|
||||
sed -n '/^IMAGE_LAYER_ZOMBIES=(/,/^)$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
sed -n '/^run_image_layer_migration()/,/^}$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
} > /tmp/_migration_block.sh
|
||||
# shellcheck disable=SC1091
|
||||
source /tmp/_migration_block.sh
|
||||
# Plant a fake zombie that should NOT be migrated because the sentinel exists
|
||||
ZOMBIE_DIR="$(mktemp -d /tmp/dashcaddy-zombie.XXXXXX)"
|
||||
mkdir -p "${ZOMBIE_DIR}/security"
|
||||
echo '{"data":"should not be migrated"}' > "${ZOMBIE_DIR}/security/audit-log.json"
|
||||
run_image_layer_migration
|
||||
if [ -f "${DATA_DIR}/migrated-audit-log.json" ]; then
|
||||
fail "test 1: sentinel existed, migration should have skipped but a file appeared"
|
||||
else
|
||||
pass "sentinel skipped migration cleanly"
|
||||
fi
|
||||
rm -rf "${ZOMBIE_DIR}"
|
||||
clean_data_dir "${DATA_DIR}"
|
||||
|
||||
# Test 2: No sentinel + non-empty zombie → migration copies file
|
||||
echo "Test 2: zombie file present → migration copies to bind mount"
|
||||
DATA_DIR="$(fresh_data_dir)"
|
||||
ZOMBIE_DIR="$(mktemp -d /tmp/dashcaddy-zombie.XXXXXX)"
|
||||
mkdir -p "${ZOMBIE_DIR}/security" "${ZOMBIE_DIR}/managers"
|
||||
echo '{"audit":"prod data"}' > "${ZOMBIE_DIR}/security/audit-log.json"
|
||||
echo "license-secret-blob" > "${ZOMBIE_DIR}/managers/.license-secret"
|
||||
{
|
||||
echo "DATA_DIR='${DATA_DIR}'"
|
||||
echo "MIGRATION_SENTINEL=\"\${DATA_DIR}/.migrated-from-image-layer\""
|
||||
sed -n '/^IMAGE_LAYER_ZOMBIES=(/,/^)$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
sed -n '/^run_image_layer_migration()/,/^}$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
} > /tmp/_migration_block2.sh
|
||||
# Stub out the real zombie paths to point at our temp zombie
|
||||
sed -i "s|/opt/dashcaddy/dashcaddy-api/src/security/audit-log.json|${ZOMBIE_DIR}/security/audit-log.json|g" /tmp/_migration_block2.sh
|
||||
sed -i "s|/opt/dashcaddy/dashcaddy-api/src/managers/.license-secret|${ZOMBIE_DIR}/managers/.license-secret|g" /tmp/_migration_block2.sh
|
||||
# shellcheck disable=SC1091
|
||||
source /tmp/_migration_block2.sh
|
||||
run_image_layer_migration
|
||||
if [ ! -f "${DATA_DIR}/migrated-audit-log.json" ]; then
|
||||
fail "test 2: audit-log.json not migrated"
|
||||
elif ! grep -q "audit.*prod data" "${DATA_DIR}/migrated-audit-log.json"; then
|
||||
fail "test 2: audit-log.json migrated but content corrupt"
|
||||
else
|
||||
pass "audit-log.json migrated with correct content"
|
||||
fi
|
||||
if [ ! -f "${DATA_DIR}/migrated-.license-secret" ]; then
|
||||
fail "test 2: .license-secret not migrated"
|
||||
else
|
||||
pass ".license-secret migrated"
|
||||
fi
|
||||
if [ ! -f "${DATA_DIR}/.migrated-from-image-layer" ]; then
|
||||
fail "test 2: sentinel file was not written"
|
||||
else
|
||||
pass "sentinel file written"
|
||||
fi
|
||||
rm -rf "${ZOMBIE_DIR}"
|
||||
clean_data_dir "${DATA_DIR}"
|
||||
|
||||
# Test 3: Idempotency — running migration twice does NOT clobber first copy
|
||||
echo "Test 3: idempotency"
|
||||
DATA_DIR="$(fresh_data_dir)"
|
||||
ZOMBIE_DIR="$(mktemp -d /tmp/dashcaddy-zombie.XXXXXX)"
|
||||
mkdir -p "${ZOMBIE_DIR}/security"
|
||||
echo '{"first":true}' > "${ZOMBIE_DIR}/security/audit-log.json"
|
||||
{
|
||||
echo "DATA_DIR='${DATA_DIR}'"
|
||||
echo "MIGRATION_SENTINEL=\"\${DATA_DIR}/.migrated-from-image-layer\""
|
||||
sed -n '/^IMAGE_LAYER_ZOMBIES=(/,/^)$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
sed -n '/^run_image_layer_migration()/,/^}$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
} > /tmp/_migration_block3.sh
|
||||
sed -i "s|/opt/dashcaddy/dashcaddy-api/src/security/audit-log.json|${ZOMBIE_DIR}/security/audit-log.json|g" /tmp/_migration_block3.sh
|
||||
# shellcheck disable=SC1091
|
||||
source /tmp/_migration_block3.sh
|
||||
run_image_layer_migration
|
||||
echo '{"second":true}' > "${ZOMBIE_DIR}/security/audit-log.json" # mutate the source after migration
|
||||
run_image_layer_migration
|
||||
if grep -q "first.*true" "${DATA_DIR}/migrated-audit-log.json"; then
|
||||
pass "second run did not overwrite first migrated content"
|
||||
else
|
||||
fail "second run overwrote the migrated file"
|
||||
fi
|
||||
rm -rf "${ZOMBIE_DIR}"
|
||||
clean_data_dir "${DATA_DIR}"
|
||||
|
||||
# Test 4: Zero-byte zombie (empty file) → NOT migrated
|
||||
echo "Test 4: empty file is not migrated"
|
||||
DATA_DIR="$(fresh_data_dir)"
|
||||
ZOMBIE_DIR="$(mktemp -d /tmp/dashcaddy-zombie.XXXXXX)"
|
||||
mkdir -p "${ZOMBIE_DIR}/security"
|
||||
touch "${ZOMBIE_DIR}/security/audit-log.json" # zero bytes
|
||||
{
|
||||
echo "DATA_DIR='${DATA_DIR}'"
|
||||
echo "MIGRATION_SENTINEL=\"\${DATA_DIR}/.migrated-from-image-layer\""
|
||||
sed -n '/^IMAGE_LAYER_ZOMBIES=(/,/^)$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
sed -n '/^run_image_layer_migration()/,/^}$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
} > /tmp/_migration_block4.sh
|
||||
sed -i "s|/opt/dashcaddy/dashcaddy-api/src/security/audit-log.json|${ZOMBIE_DIR}/security/audit-log.json|g" /tmp/_migration_block4.sh
|
||||
# shellcheck disable=SC1091
|
||||
source /tmp/_migration_block4.sh
|
||||
run_image_layer_migration
|
||||
if [ -f "${DATA_DIR}/migrated-audit-log.json" ]; then
|
||||
fail "test 4: empty file should not be migrated"
|
||||
else
|
||||
pass "empty file correctly skipped"
|
||||
fi
|
||||
if [ -f "${DATA_DIR}/.migrated-from-image-layer" ]; then
|
||||
pass "sentinel still written even with zero zombies"
|
||||
else
|
||||
fail "sentinel should still be written even with no zombies"
|
||||
fi
|
||||
rm -rf "${ZOMBIE_DIR}"
|
||||
clean_data_dir "${DATA_DIR}"
|
||||
|
||||
# Test 5: set -e present + all per-file failures → script doesn't take down container
|
||||
echo "Test 5: a single per-file failure does not bring down the container"
|
||||
DATA_DIR="$(fresh_data_dir)"
|
||||
ZOMBIE_DIR="$(mktemp -d /tmp/dashcaddy-zombie.XXXXXX)"
|
||||
mkdir -p "${ZOMBIE_DIR}/security"
|
||||
echo "x" > "${ZOMBIE_DIR}/security/audit-log.json"
|
||||
chmod 000 "${ZOMBIE_DIR}/security/audit-log.json" # make it unreadable so cp -a fails
|
||||
{
|
||||
set -e # NOW we need to verify the inner guard prevents set -e from killing us
|
||||
echo "DATA_DIR='${DATA_DIR}'"
|
||||
echo "MIGRATION_SENTINEL=\"\${DATA_DIR}/.migrated-from-image-layer\""
|
||||
sed -n '/^IMAGE_LAYER_ZOMBIES=(/,/^)$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
sed -n '/^run_image_layer_migration()/,/^}$/p' "${SCRIPT_DIR}/../start.sh"
|
||||
} > /tmp/_migration_block5.sh
|
||||
sed -i "s|/opt/dashcaddy/dashcaddy-api/src/security/audit-log.json|${ZOMBIE_DIR}/security/audit-log.json|g" /tmp/_migration_block5.sh
|
||||
EXIT=0
|
||||
# shellcheck disable=SC1091
|
||||
source /tmp/_migration_block5.sh && run_image_layer_migration || EXIT=$?
|
||||
chmod 644 "${ZOMBIE_DIR}/security/audit-log.json" 2>/dev/null || true
|
||||
rm -rf "${ZOMBIE_DIR}"
|
||||
clean_data_dir "${DATA_DIR}"
|
||||
if [ "$EXIT" -eq 0 ]; then
|
||||
pass "script survived a per-file cp failure"
|
||||
else
|
||||
fail "set -e propagated a per-file failure (exit ${EXIT}); container would not boot"
|
||||
fi
|
||||
|
||||
# ---- Cleanup ----------------------------------------------------------------
|
||||
rm -f /tmp/_migration_extract.sh /tmp/_migration_block*.sh
|
||||
|
||||
echo
|
||||
if [ "$FAILURES" -eq 0 ]; then
|
||||
echo "All migration regression tests passed."
|
||||
exit 0
|
||||
fi
|
||||
echo "${FAILURES} test(s) failed."
|
||||
exit 1
|
||||
@@ -14,6 +14,60 @@ HOST_IP="172.17.0.1"
|
||||
DNS_PRIMARY="100.121.150.22" # Technitium (Tailscale IP) — resolves *.sami
|
||||
DNS_FALLBACK="8.8.8.8"
|
||||
|
||||
# --- One-time migration from Docker image layer to bind mount --------------
|
||||
# DC-039 follow-up. Before v1.14.10, certain modules (audit-logger, license-
|
||||
# keygen, credential-manager) defaulted their files to /app/src/* via
|
||||
# path.join(__dirname, 'foo.json'). Those writes landed in the Docker image
|
||||
# layer and VANISHED on every container recreate. This step scans for any
|
||||
# non-empty zombie files left over from a previous image (where /opt/dashcaddy/
|
||||
# previously used /opt/dashcaddy/dashcaddy-api/src/... as the path root) and
|
||||
# copies their contents into the bind-mounted data dir ONCE.
|
||||
#
|
||||
# Idempotent: bails out if the migration sentinel file already exists.
|
||||
# Designed to be a no-op on every fresh install.
|
||||
MIGRATION_SENTINEL="${DATA_DIR}/.migrated-from-image-layer"
|
||||
IMAGE_LAYER_ZOMBIES=(
|
||||
"/opt/dashcaddy/dashcaddy-api/src/security/audit-log.json"
|
||||
"/opt/dashcaddy/dashcaddy-api/src/security/.encryption-key"
|
||||
"/opt/dashcaddy/dashcaddy-api/src/security/.encryption-key.bak"
|
||||
"/opt/dashcaddy/dashcaddy-api/src/utils/error.log"
|
||||
"/opt/dashcaddy/dashcaddy-api/src/managers/.license-secret"
|
||||
"/opt/dashcaddy/dashcaddy-api/src/managers/.license-counter"
|
||||
)
|
||||
# Note: set -e is active at top of script. Each per-file step uses an
|
||||
# explicit `|| true` (or guarded `if`) so a single unreadable zombie file
|
||||
# can't take down the whole container. The sentinel write at the end is
|
||||
# outside any conditional so it always runs once.
|
||||
run_image_layer_migration() {
|
||||
if [ -f "${MIGRATION_SENTINEL}" ]; then
|
||||
return 0
|
||||
fi
|
||||
mkdir -p "${DATA_DIR}" || { echo "[start.sh] [migration] mkdir failed: ${DATA_DIR}"; return 0; }
|
||||
local migrated=0
|
||||
for src in "${IMAGE_LAYER_ZOMBIES[@]}"; do
|
||||
if [ -f "${src}" ] && [ -s "${src}" ]; then
|
||||
local dest_name dest
|
||||
dest_name="$(basename "${src}")"
|
||||
dest="${DATA_DIR}/migrated-${dest_name}"
|
||||
if [ ! -f "${dest}" ]; then
|
||||
echo "[start.sh] [migration] Recovering image-layer file: ${src} -> ${dest}"
|
||||
if cp -a "${src}" "${dest}" 2>/dev/null; then
|
||||
migrated=$((migrated + 1))
|
||||
else
|
||||
echo "[start.sh] [migration] WARN: failed to copy ${src} (continuing)"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
done
|
||||
if [ "$migrated" -gt 0 ]; then
|
||||
echo "[start.sh] [migration] Recovered ${migrated} file(s) from image layer."
|
||||
echo "[start.sh] [migration] Review files prefixed 'migrated-' in ${DATA_DIR} and merge or delete."
|
||||
fi
|
||||
# Sentinel write MUST run regardless of any per-file failure above.
|
||||
date -u +%Y-%m-%dT%H:%M:%SZ > "${MIGRATION_SENTINEL}" 2>/dev/null || echo "1" > "${MIGRATION_SENTINEL}"
|
||||
}
|
||||
run_image_layer_migration
|
||||
|
||||
# --- /etc/hosts overrides for the container ---------------------------------
|
||||
# The base image (node:20-alpine) has no entries for *.sami. We must inject
|
||||
# them via --add-host so health checks inside the container can resolve LAN
|
||||
|
||||
Reference in New Issue
Block a user