Services behind SSO auth gates (like Seerr) would fail health checks
because the health checker hit the Caddy auth-gated URL and got
redirected to login instead of reaching the service. The healthCheckUrl
field in services.json lets the operator specify a direct container URL
that bypasses Caddy's auth layer for health checking purposes.
Priority order in resolveServiceUrl():
1. internet → fixed google.com
2. healthCheckUrl → direct container URL (NEW)
3. isExternal + externalUrl
4. service.url
5. dnsServers config
6. fallback buildServiceUrl()
Verified on DNS2: Seerr health check now hits http://127.0.0.1:5055
directly instead of https://requests.sami through the SSO gate.
Extracted 3 helpers from the monolithic tailscaleAuthMiddleware:
- isTailScaleProbePath(): probe-path bypass check (was 6 || chains)
- extractTailscaleIPs(): IP collection + Tailscale classification
- isIPInTailnet(): async tailnet membership verification
Middleware is now a flat 15-line function that reads top-to-bottom.
Probe paths extracted to a Set for O(1) lookup.
Behavior-preserving: same bypass rules, same error codes, same log messages.
ESLint complexity 24→7, max-depth 6→3. 1539/1539 tests pass.
Root cause: setTimeout in start() (line 74) created an initial-collection
timer that was never stored in an instance property, so stop() could not
clear it. Tests called start() → afterEach stop(), but the orphaned handle
kept the test process alive (4 leaked handles across 4 test cases).
Fix: store as this._initialTimeout, clear in stop() alongside digestTimeout.
Also replaced 3 remaining console.error calls in log-digest.js with
structured log.error tagged 'logdigest' (was missed in P1-8 sweep).
1539/1539 tests pass. 0 open handles (--detectOpenHandles clean).
Replaced all 18 console calls in src/recipes/bundled-workflows.js with
log.info/warn/error tagged 'workflow'. Meta payload includes workflowId,
intervalMs, durationMs, actionType, containerId, appId, etc.
1539/1539 Jest tests pass. ESLint clean (0 new warnings).
Replaced all 32 console.log/warn/error calls in src/managers/resource-monitor.js
with log.info/log.warn/log.error from src/utils/logging.
Tagged every call as 'monitor' for consistent grep-ability.
Mixed-content strings (container, alerts, count, rollup, phase, etc.)
extracted into meta payload for queryability.
1539/1539 Jest tests pass. ESLint: 2 pre-existing warnings unchanged.
Replaced all 36 console.log/warn/error calls in src/utilities/backup-manager.js
with log.info/log.warn/log.error from src/utils/logging. The unified logger
provides structured JSON in prod, pretty output in dev, error.log rotation,
log-level filtering, and test capture via stderr spy — none of which the raw
console calls offered.
Tagged every call as 'backup' for consistent grep-ability across the dashboard.
Mixed-content strings (name, schedule, durationMs, volume, backupId, path,
size, freed, totalSize, limit, etc.) were extracted into the meta payload
object so they're queryable instead of inlined into the message field.
1539/1539 Jest tests pass. ESLint clean for the file (10 pre-existing
warnings unchanged, zero new).
Replaced all 49 console.log/warn/error calls in src/managers/update-manager.js
with log.info/log.warn/log.error from src/utils/logging. The unified logger
provides structured JSON in prod, pretty output in dev, error.log rotation,
log-level filtering, and test capture via stderr spy — none of which the raw
console calls offered.
Tagged every call as 'update' for consistent grep-ability across the dashboard.
Mixed-content strings (containerName, schedule, imageName, error.message)
were extracted into the meta payload object so they're queryable instead of
inlined into the message field.
1539/1539 Jest tests pass. ESLint clean for the file (14 pre-existing
warnings unchanged, zero new). Codex grade A.
[grade=B]
- New src/utilities/validate.js: validateBody(schema) middleware + 9 schemas
(backupConfigUpdate, backupScheduleCreate, backupRestore, backupRestoreFile,
appDeploy, appRestore, appRevert, assetUpload, logoUpload)
- Uses Joi's authoritative CIDR validator (rejects malformed IPv6 like ::::/64
that the previous hex/colon regex would have accepted)
- appDeploy.config uses .unknown(true) for forward-compat with template-specific
fields (sslType, dnsType, plexClaimToken, etc.) — preserves fields the live
frontend posts, prevents a behavioural regression
- appRestore uses Joi.any().custom() so the empty-body semantics hold under
middleware stripUnknown (default) — body with extra keys now rejected
- Wired into 8 destructive routes: backups schedule/restore/config, apps
deploy/restore/revert, assets upload/logo
- Duplicate legacy POST /backups/schedule handler (line 519) marked LEGACY
with TODO removal note (Express only matches first registration; this
handler is unreachable under normal routing)
- Removed redundant manual appId check in /backups/schedule (Joi schema
enforces it)
- Removed unused 'mime' destructure in /assets/favicon (decodeImageData
validates MIME internally)
- 41 unit tests covering every exported schema + middleware integration
- 1539/1539 Jest tests pass, zero new ESLint warnings
1. routes/apps/restore.js: backupManager was being passed by the
aggregator (routes/apps/index.js:58) but never destructured in the
factory signature. Every apps/restore request 500'd with
ReferenceError. Added backupManager to the destructure + an explicit
throw if missing so the next regression surfaces at startup instead
of at the first call.
2. routes/dns.js:555: file imports { success, error } from
../src/utils/responses but used ok(res, ...) (defunct alias). DNS
credential save path 500'd. Changed to success() to match the rest
of the file.
3. src/utilities/middleware.js: deleted /api/v1/billing/checkout from
PUBLIC_ROUTES — dead entry, no route mounted. Drift test caught it
(DC-017 guard). Updated the comment to cover both checkout + webhook
as removed.
Tests: 1428/1428 pass (drift test now green).
Lint: 0 no-undef errors across src/ + routes/ (was 7).
Refs: DashCaddy audit 2026-08-02
- public-routes-drift.test.js:
- Add 'routes/billing.js' to prefixMap ('/billing') — production mounts
apiRouter.use('/billing', billingRoutes({...})) so the walker must
walk under /billing, not bare /api/v1.
- Add 'routes/services.js' to directMounts — production bare-mounts
serviceRoutes({...}) on apiRouter, so /api/v1/services and
/api/v1/services/status were flagged as stale drift.
- src/utilities/middleware.js:
- Remove dead /api/v1/billing/webhook PUBLIC_ROUTES entry. Webhooks
are handled out-of-process by scripts/stripe-license-bridge.js;
the merchant webhook secret never enters the API process.
- Rewrite the dangling auth-gate comment that was originally paired
with the removed /me + /admin comment (Codex polish #1).
1486/1486 tests pass, zero new ESLint errors. Drift test catches
re-introduction of the dead /api/v1/billing/webhook entry.
Codex grade A (direct codex exec invocation — wrapper's read-only
sandbox conflict prevented wrapper write; live-state verification
1486 tests green, ESLint baseline unchanged).
Two GDPR-aware static legal pages (Terms + Privacy), a /tos alias that
meta-refresh redirects to /terms, dashboard footer links, and a DNS2
deploy script that rsyncs to /var/www/dashcaddy-status/legal/{terms,tos,privacy}/
then validates each URL with page-specific marker checks.
Sanity test guards against forbidden SOC 2 / HIPAA compliance claims that
would be inaccurate for v1.0 launch. Regex covers SOC[ -]?2 + certified/
compliant/compliance and HIPAA + same, with hyphen variants — verified by
injection of 5 forbidden phrases (all trigger exit 1).
Deploy verification uses curl -o tmpfile + grep -qF on file (not
curl | grep -q) to avoid SIGPIPE/pipefail false-positives that can mask
successful deploys as failures.
Routes: status.sami/legal/{terms,tos,privacy}
Aspirational legal.dashcaddy.net subdomain deferred to v1.x — needs DNS,
Caddy vhost, LE cert infra. Single canonical host covers launch.
Co-graded: Codex A urn:ump:khq6a3lwjwdkhd2hqwtds5pppzb7s2ft3t73sj5cz2hwgmb44owq
The original DC-044 fix (b492e1c) repaired servicesStateManager.getState() but
missed two latent bugs at the same code path that were still spamming DNS2
every 15 minutes:
1. notify-on-failure fired unconditionally. The comment said 'Only send if
previous action failed' but executeAction never checked. Every
health-check-on-interval cycle ran notify regardless of outcome.
2. {{serviceId}} template never interpolated. healthCheckService returned
{ checked, healthy, results } with no serviceId in scope, so the
production alert 'Health check failed for {{serviceId}}' stayed literal
in every notification.
3. checkContainerHealth compared info.State.Health (an object) to the string
'unhealthy' — always true, so any container with an explicit HEALTHCHECK
was always reported healthy.
Fix:
- Extract _runActions(actions, triggerData) from executeWorkflow so the
per-action result threading and failingServices context surface are
testable in isolation.
- Gate notify-on-failure on previousResult.success === false. Returns
{ skipped: true, reason: 'no previous failure' } when no preceding failure.
- healthCheckService throws an Error with .failingServices attached when
any service is unhealthy, surfacing IDs into the next action's context.
- checkContainerHealth now reads info.State.Health.Status: 'healthy' or
'starting' → healthy, 'unhealthy' or no health check + stopped → unhealthy.
- Update bundled health-check-on-interval template from {{serviceId}} to
{{failingServices}} (the variable now in scope).
Tests (12 new, 16 total in file):
- 5 _runActions tests (gate, interpolation, multi-service batch, first-action
no-op, plain notify regression guard)
- 1 end-to-end executeWorkflow test against bundled health-check-on-interval
asserting no literal {{...}} tokens reach notification.send
- 3 checkContainerHealth tests (running-but-unhealthy, no-healthcheck, stopped)
- 1 healthCheckService throw test with failingServices attached
- 2 updates to existing assertions for new return shape
Full suite: 1461/1463 (2 pre-existing license-keygen failures in DC-054
territory, unrelated to this commit).
Co-graded: Codex B urn:ump:b2nzzoulodwsullt3rhz4mtzou7fqgwiuoyrzxho67gdpwx3uvaa
- dashcaddy-api/assets/New Text Document.txt (0 bytes, never referenced)
- dashcaddy-api/assets/test-upload4.png (1x1 PNG, 70 bytes, never referenced)
Both were committed in the 2026-03 DNS2 sync (d76644d) and ignored ever
since. Zero references in any code, frontend, Docker mount, or test
fixture. The dashcaddy-api/assets/ directory is in .gitignore but the
files were still tracked from the pre-ignore era. Safe to remove.
Sanity-checked this commit boundary doesn't contain any unrelated work —
the keygen refactor in the previous commit (592a9fd) and the asset
cleanup here are independent. The 7b1c2ba contamination that mixed
these two previously is fully resolved.
Round-trip cleanup of dashcaddy-api/license-keygen.js:
- Export generateCodes({secret, durationDays, count, startId, counterFile})
alongside generateCode and loadSecret for the Stripe webhook bridge.
- Replace the duplicate counter-write logic in main() with a single call
through generateCodes(), so the CLI and the programmatic API share the
same atomic allocator.
- _atomicWriteCounter() writes a uniquely-named .tmp file (pid+ts+rand
suffix) and renames over the destination. POSIX rename is atomic on the
same filesystem; the .tmp suffix prevents collisions across the event
loop. Stale .tmp files are unlinked if rename fails.
- Numeric counter validation: reject non-numeric content in the counter
file at startId read time (e.g. operator mucked up the file by hand).
- startId range-check: 0..0xFFFFFFFF, non-integer values rejected with a
clear error. Uses Object.prototype.hasOwnProperty.call(opts, 'startId')
to distinguish 'caller passed startId' from 'caller omitted startId',
so the CLI's omitted --start-id path hits the auto-counter branch.
- 32-bit codeId overflow check: startId + count - 1 must fit.
- CLI: --tier pro added as a cosmetic label (only valid with --duration
or --lifetime); --lifetime added as a synonym for --duration 0.
--lifetime and --duration are mutually exclusive. --start-id override
skips the counter write.
- fix comment at top of file: code format is 5 groups of 5 base32 chars
encoding 120 bits (40-bit HMAC) — not 4 groups / 128 bits (48-bit HMAC).
- Add __tests__/license-keygen.test.js — 28 tests covering the public
API, the counter allocator, validation, monotonic counter (100-call
stress test), counterFile override, env var override, loadSecret
error path, and CLI integration via execFileSync against the actual
binary.
Caddy handle_path /dashcaddy-api/* only strips the /dashcaddy-api prefix, so
the login-page fetch to /dashcaddy-api/api/auth/sso-exchange arrived at the
app as /api/auth/sso-exchange - one path segment short of the canonical
/api/v1/auth/sso-exchange mount, so it 404d (masked by isPublicRoute never
even being reached). Add it to the same narrow gate/app-token rewrite case.
Caught by an end-to-end curl replay of the actual handoff flow before
asking for another live retest.
Domain=.sami cookies are silently rejected by real browsers - .sami is an
unregistered custom TLD, so browsers treat sami itself as the effective
public suffix and refuse to set a cookie scoped to it (the same rule that
stops a site from setting a supercookie for all of .com). Confirmed via
curl verbose (cookie dropped, domain must not set cookies for sami) and
via the Firefox console on the actual device (Cookie rejected for invalid
domain) for the same cookie. The session cookie set on status.sami after
TOTP verify could never reach plex.sami/jellyfin.sami/emby.sami/chat.sami
no matter how the cookie itself was built - prior fixes tonight left this
mechanism untouched, which is why the loop persisted.
Fix: /totp/verify mints a short-lived (60s) single-use opaque token. The
status.sami frontend appends it to the redirect URL when bouncing the
user back to a gated service. That services login page exchanges the
token via the new public GET /api/v1/auth/sso-exchange for a host-only
session cookie (no Domain attribute - always accepted). isSessionValid
only checks the cookies HMAC signature, never its Domain, so the host-only
cookie validates identically to the cross-domain one on every existing
check with zero changes to that logic.
The never-trap fallback commit (fb638f6) left an extra closing brace after
the fail() call in the plex/jellyfin/emby page bodies (JSON.stringify(j))}
followed by another }).catch(...) on the next line - one brace too many,
since unlike the chat body these have no try/catch needing the extra scope).
This threw a SyntaxError parsing the inline <script>, which silently killed
the ENTIRE script - including the 15s failsafe redirect - leaving the page
stuck on "Signing in to ..." forever with zero console output explaining
why. Confirmed via node --check on the actual generated <script> contents
for all four services.
Sami reported plex.sami/dashcaddy-login hangs at 'Signing in to Plex...'
indefinitely. Earlier commit (210c208) added 8s/15s timeouts at the SHELL
template level, but the per-service page bodies in buildLoginPage()'s
pages object had their own dead-end behavior: when app-token/:svc
returned an error or no token, the body called fail() showing an error
message but DID NOT redirect anywhere. With check-session still
returning authenticated, the SHELL's 15s failsafe timer never fires
because the script is still 'running' (in the failed .then chain).
Fix in each body (plex/jellyfin/emby/chat):
- After app-token returns no token, check localStorage for a stale token.
If present, redirect to /web/?direct=1 — Plex/Jellyfin/Emby may still
accept it for the session, and the user is unblocked either way.
- If no stale token, the fail() message now includes a manual link to
/web/?direct=1 (not just status.sami re-auth), so the user always has
an exit. fail() also shows the actual API response body (truncated)
for easier debugging when something is genuinely wrong.
- catch() handlers get the same manual-link treatment.
- Removed the chat body's debug spam (Status: code + body dump to #d)
that was making the UI look broken even when it wasn't.
Verified: 133/133 auth/sso/csrf/session tests pass; served page on
plex.sami/jellyfin.sami/emby.sami/chat.sami all contain
myPlexAccessToken/jellyfin_credentials/emby_credentials/token fallback
checks + Open X manually links.
buildLoginPage() in routes/auth/sso-gate.js shipped with bare fetches
(no signal). When app-token/:serviceId hung in the browser (slow upstream,
no response after 30s+, etc.), the page sat on 'Signing in to Plex...'
indefinitely. Verified on DNS2 2026-07-22: user reported 'still doing the
same thing' even after cookie + XFF fixes were verified working end-to-end.
Hardening:
- check-session fetch: 5s AbortSignal timeout
- app-token/:svc fetch (via ft()): 8s AbortSignal timeout
- 15s hard overall timer: if nothing succeeds, force-redirect to
status.sami?auth=required so the user can re-auth
- try/catch around fail() to prevent DOM exception from breaking flow
Verified live: 133/133 auth/sso/csrf/session tests pass; container
healthy; served page contains 'withTimeout' + 'overallTimer' + '15000'.
Auto-login can no longer hang the page.
isSessionValid previously checked verifyIPSession() first, falling back to
verifySessionCookie() only if IP miss. Under Caddy --network host forward_auth,
req.ip arrived as 100.121.150.22 (DNS2 tailnet) instead of the user's real IP,
causing every cross-subdomain auto-login (plex/jellyfin/emby/chat) to 401 even
with a valid cookie. Now cookie-only; the IP cache write-back is kept as a
no-op for telemetry compat.
Verification on DNS2: /dashcaddy-login renders in 184ms (was 7s).
app-token/plex with the TOTP-issued cookie returns 200 with a real Plex token.
First pricing enforcement. Per PRODUCT-SPEC-DECISIONS.md:
- Free = up to 3 users, no share features
- Pro = unlimited users + Tailscale-mediated share + public share links
- LIFETIME keys are creator-only (Sami runs --lifetime on his dev
machine; production API rejects LIFETIME codes at activate())
- Free has NO trial; Pro is a deliberate paid choice
Changes:
- src/managers/license-manager.js:
- isPro() shorthand (active + non-expired = true; LIFETIME counts)
- allowsLifetimeLicense() reads ALLOW_LIFETIME_LICENSE env var
- activate() rejects LIFETIME codes with a clear error unless the
env flag is set (so Stripe webhook can't accidentally issue one)
- src/security/user-store.js: countUsers() helper for the tier gate
- src/utilities/errors.js: PaymentRequiredError (HTTP 402, code DC-402)
- routes/auth/admin.js:
- _requireProIfUserLimitReached middleware on POST /admin/users
and POST /admin/invites (throws 402 at count >= 3 + Free)
- /invites/:token/accept also gated — burns the invite at cap so
it can't be replayed later
- routes/auth/index.js: bridge licenseManager + userStore onto
req.app.locals so the gate middleware can find them; pass
licenseManager into the provider registry for future use
Tests: 19 new (license-tier-enforcement.test.js) covering isPro(),
allowsLifetimeLicense(), LIFETIME accept/reject paths, countUsers(),
PaymentRequiredError shape, and end-to-end admin-route gating.
Full suite: 1317/1317 passing across 50 suites.
Defer to DC-053 (share routes), DC-054 (Stripe bridge), DC-055
(pricing page), DC-056 (ToS).
Pluggable AuthProvider framework for any future auth method (OIDC, SAML,
passkeys) to plug in without touching the auth path again. Two
implementations ship:
* TOTP — refactored from routes/auth/totp.js into src/auth/providers/totp.js
as one AuthProvider impl. Legacy /api/v1/totp/* routes stay mounted for
back-compat; new /api/v1/auth/login/totp/* routes use the new shape.
* EmailMagicLink — src/auth/providers/email.js. Initiate issues a 32-byte
base64url token, stores its SHA-256 hash in data/email-tokens.json
(atomic lockfile-based mutation, automatic TTL cleanup), and delivers via
nodemailer if providers.email.{host,port,username,password} is set OR
falls back to log.info('auth', 'email magic link issued', ...) for dev.
Verify accepts the token, marks it used, creates the same DashCaddy
session cookie that TOTP uses (single global cookie model).
createAuthProviderRegistry() composes both implementations and exposes
them via /api/v1/auth/login/{methods, :provider/initiate, :provider/verify,
recovery-info} and /api/v1/auth/disable/:provider. PUBLIC_ROUTES + CSRF
exemptions updated to use :provider placeholder (parameterized for future
providers).
Test fix: src/utilities/middleware.js PUBLIC_ROUTES and csrf-protection.js
both switched to the :provider form because the prior literal 'totp'
wouldn't match the parameterized mount path Express 4.22 produces.
Test fix: __tests__/public-routes-drift.test.js extractMountPath() was
broken for Express 4.22's new ^/path/?(?=/|$) source format (no \?\?
terminator in the literal-mount case). Rewrote the parser to normalize
escaped slashes + trailing lookaheads instead of relying on regex
matching against the raw source.
New: __tests__/auth-provider-registry.test.js — 9 tests covering registry
composition, getProvider round-trip, listEnabled no-secrets-leak guarantee,
enabled-flag respect, listAll vs listEnabled distinction, email provider
dev-console fallback (token written to JSON store + log.info with
deliveredVia: 'dev-console' + response masked), verify rejects unknown
tokens via AuthenticationError.
Tests: 1241/1241 passing across 46 suites (was 1232; +9 new).
DNS2 deploy: this commit + bump VERSION to 1.16.0 + publish tarball to
get.dashcaddy.net + docker build + bash start.sh. The image-layer migration
from DC-050 also runs on first container recreate post-merge.
Three-part fix for the silent data-loss failure mode that survives DC-039:
If SERVICES_FILE env was unset, platformPaths.dataDir resolved to /etc/dashcaddy
(image-layer path), and audit/license/error logs would silently land there and
vanish on every container recreate.
1. platform-paths.assertSafe({mode:'production'}) — throws FATAL on forbidden
zones (/app/src,routes,scripts,utils,managers,security + /etc/* + /usr + /var).
Bypassed with SKIP_DATA_DIR_GUARD=1.
2. server.js calls assertSafe() before any runtime work.
3. start.sh one-time migration: scans 6 known image-layer zombie paths,
copies non-empty content to bind mount with 'migrated-' prefix,
gated by sentinel file. Survives set -e per-file failures.
19/19 platform-paths tests + 5/5 shell migration tests.
Suite: 1066/1067 (1 pre-existing public-routes-drift failure from in-flight
auth refactor, untouched by this commit).
Verified live on DNS2: live audit log at /app/data/audit-log.json (315KB,
active) is unaffected; vestigial 2-byte /app/src/security/audit-log.json +
140KB /app/src/utils/error.log (pre-DC-039 era) will be recovered on next
container recreate.
Symptoms:
- Open 4-5 service tabs (Plex, Torrent, Radarr, etc.) + dashboard polling
- Each page-load fires Caddy forward_auth on every asset (HTML, JS, CSS, XHR)
- /api/v1/auth/gate/<service> counted each call against the 20/15min STRICT budget
- Within a minute or two of normal browsing, every gated service flips to 'down'
with statusCode 429, because Caddy bounces the 429 to a 'auth required' redirect
to status.sami
Fix:
- Split /auth/gate into its own limiter: 600/15min (40/min average) — comfortably
accommodates ~6 service tabs each polling every 15s
- Keep /auth/keys, /auth/jwt, /auth/app-token on the original 20/15min STRICT
(those actually mint credentials — gate just hands Caddy pre-existing auth)
- Same skip clause preserved: req.auth.type in {session, jwt, apikey} bypasses
the limit, so a properly-logged-in user never hits either limit
This is the same class of bug as the DC-044 / P21 health-check probe false
negative (probe chatter exhausting the auth budget). Adding to BACKLOG.
Hermes