Merge dc/DC-085-link-first-invite-dc-086-flicker-fix: DC-085 link-first invite + DC-086 badge flicker hysteresis + race hardening [glm-grade=A]
CI / Test & Lint (push) Canceled after 0s
CI / Security audit (push) Canceled after 0s

This commit is contained in:
Hermes
2026-08-22 14:27:54 -07:00
7 changed files with 847 additions and 56 deletions
+16
View File
@@ -401,6 +401,22 @@ Tickets DC-046 through DC-049 implement pluggable auth + email magic link. Sami
- **result:** Shipped codex-graded A. All 49 `console.*` sites in `src/managers/update-manager.js` now route through `log.info/log.warn/log.error` from `src/utils/logging` (tag = `'update'`). Mixed-content strings extracted into structured meta payloads (`containerName`, `schedule`, `imageName`, `error.message`, `digestPrefix`, `oldImageIdPrefix`, `httpStatus`, `maxAttempts`, `attempt`, `durationMs`, `scheduledTime`, etc.) so fields are queryable instead of inlined into the message. Errors now go through `log.error(ctx, errObj)` so they land in error.log with full stack trace + context, not just stderr. 1539/1539 Jest tests pass (78/78 update-manager tests still pass). ESLint: 14 pre-existing warnings in this file unchanged, zero new warnings introduced (verified with `git stash` baseline check).
### DC-086: Service-status flicker fix — asymmetric hysteresis on the badge
- **status:** in-progress
- **owner:** hermes
- **details:** Dashboard service badges perpetually flip between green and red for "a few seconds at a time, never stable" (Sami's report, 2026-08-20). Root cause: `src/monitoring/health-checker.js` `recordStatus()` emits `'status-check'` on EVERY probe (every 30s), and `src/websocket/dashboard-ws.js` forwards every probe as `'status-change'` to the browser with no diff. The frontend `live-events.js` then unconditionally calls `setBadge()` — which resets the icon + pill text on every event. A single transient 5xx (Caddy reload, container CPU steal, mid-flight TLS handshake, container restart during probe) flips the badge red and the next green probe flips it back. Fix: add asymmetric hysteresis in `_computeDisplayedStatus(serviceId, rawStatus)` — going DOWN requires 2 consecutive "down" probes (default `HEALTH_DOWN_THRESHOLD=2`), going UP requires only 1 (default `HEALTH_UP_THRESHOLD=1`). History + `consecutiveFailures` still record raw probe results (operators want full fidelity for postmortems); only the dashboard broadcast is filtered. `getCurrentStatus()` now returns the displayed status so a page reload shows the same badge as the live SSE stream. Both thresholds are env-var configurable so operators can tune. New tests in `__tests__/health-checker-hysteresis.test.js` cover: first probe emits; second probe same-status does NOT re-emit; one-down-then-up keeps green; two-down flips to red; one-up after down flips back to green; `getCurrentStatus` returns displayed not raw. Effort: ~30 min. Risk: low — pure behavior filter, no schema breaks, all 63 existing health-checker tests must stay green.
- **impact:** Operators stop seeing perpetual red/green flicker on healthy services. Real outages still get flagged (2 consecutive 30s probes = ~60s before badge flips red, which is still faster than a human notices). Background probe history is unchanged so postmortem analysis still works.
- **prerequisite:** None.
- **result:** _pending — ship + codex round_
### DC-085: Link-first invite — Discord-style "share it however you want"
- **status:** in-progress
- **owner:** hermes
- **details:** Today `POST /api/v1/auth/admin/invites` defaults to sending the invite link via SMTP; if SMTP is not configured it spams the server console with `[DC-048-DEV-INVITE-LINK]` log lines. Sami wants Discord-style: the link is always returned in the response, and email is an opt-in checkbox. Operators should be free to copy the link and share it via iMessage / SMS / WhatsApp / Telegram / Signal / Discord / paste-in-email — whatever fits. (1) Flip default `sendEmail !== false` to `sendEmail === true` in `routes/auth/admin.js` so omitting the field means "no email, just hand me the link." (2) Stop logging the raw invite URL to error.log when SMTP is unconfigured — that path was only useful when there was no UI way to grab the link; now there is. (3) Add a `shareText` field to the response: `"Join my DashCaddy as <role> — <acceptUrl> — expires in Nh."` for one-tap paste into any messenger. (4) Frontend: `status/js/admin.js` `_renderInviteForm` flips the "Send email" checkbox default to **unchecked**, updates `_renderIssuedInviteBanner` to show both the raw link AND the shareText (with its own copy button + `navigator.share()` native share-sheet button where available). (5) New tests in `__tests__/admin-invites.test.js` covering: default sendEmail=false (no SMTP send attempted, no console log); `sendEmail: true` triggers SMTP send; `shareText` is present and well-formed; `acceptUrl` is always returned; expired sendEmail path doesn't leak token to logs. Effort: ~1 hr. Risk: low — pure behavior flip + UI additive change.
- **impact:** Closes the friction between "host wants to add a friend" and "host has to configure SMTP first." Mirrors Discord/Slack/Linear invite flows where the link IS the deliverable. No new tier changes, no schema breaks.
- **prerequisite:** DC-048 (invite store + admin route), DC-052 (Pro gate stays).
- **result:** _pending — ship + codex round_
### DC-084: Remove redundant active Caddy health check from `arch.sami` site — eliminate 6 syslog spam lines/min
- **status:** done
- **owner:** hermes
@@ -0,0 +1,240 @@
/**
* Tests for DC-085: link-first invite (Discord-style "share it however you want").
*
* - default sendEmail omission = no email sent, link returned, no token in logs
* - sendEmail:true triggers SMTP send when configured
* - sendEmail:true + SMTP unconfigured = deliveredVia:'failed', no token leaked
* - shareText field present and well-formed in every response
* - acceptUrl always present (regardless of sendEmail)
* - role + ttl validation unchanged from DC-048
*
* Strategy: drive the route handler directly with mock req/res, mount the admin
* router against an isolated userStore + inviteStore + email-sender stub.
*/
'use strict';
const fs = require('fs');
const path = require('path');
const os = require('os');
const express = require('express');
function _tmpDir() {
return fs.mkdtempSync(path.join(os.tmpdir(), 'dashcaddy-admin-invites-test-'));
}
function _cleanup(dir) {
try { fs.rmSync(dir, { recursive: true, force: true }); } catch {}
}
// Stub email-sender so we can assert "was it called?" without an SMTP server.
// NOTE: the variable name MUST start with `mock` so Jest's hoisted `jest.mock()`
// call is allowed to reference it (Babel guard against out-of-scope access).
const mockEmailSender = {
isConfigured: jest.fn(() => false),
sendEmail: jest.fn(async () => undefined),
};
jest.mock('../src/auth/providers/email-sender', () => mockEmailSender);
describe('DC-085: link-first admin invites', () => {
let dir, app, request;
let logCalls; // captured { level, msg, meta } from our fake log
beforeEach(async () => {
jest.clearAllMocks();
dir = _tmpDir();
logCalls = [];
// Set up email auth enable flag so userStore mounts.
process.env.NODE_ENV = 'test';
const { createUserStore } = require('../src/security/user-store');
const userStore = createUserStore({ dataDir: dir });
// Bootstrap the admin so we have a session-attributable user.
await userStore.login({ email: 'admin@sami-host.me' });
// Build a tiny Express app with the admin router mounted, but skip the
// global auth gate (we inject req.user directly).
const adminRouter = require('../routes/auth/admin')({
asyncHandler: (fn) => (req, res, next) => Promise.resolve(fn(req, res, next)).catch(next),
errorResponse: (_res, code, msg) => ({ status: code, msg }),
log: {
info: (topic, msg, meta) => logCalls.push({ level: 'info', topic, msg, meta }),
warn: (topic, msg, meta) => logCalls.push({ level: 'warn', topic, msg, meta }),
error: (topic, msg, meta) => logCalls.push({ level: 'error', topic, msg, meta }),
},
session: { isSessionValid: () => true, create: () => {}, setCookie: () => {} },
dataDir: dir,
});
app = express();
app.use(express.json());
// Inject req.user = admin so /admin/* passes the role gate.
app.use((req, _res, next) => {
req.user = { id: 'admin-id', email: 'admin@sami-host.me', role: 'admin' };
req.app.locals = req.app.locals || {};
req.app.locals.siteConfig = {}; // no publicBaseUrl — route uses req.headers
req.app.locals.emailConfig = null; // SMTP not configured by default
next();
});
app.use('/api/v1/auth', adminRouter);
// Error handler — last in chain.
app.use((err, _req, res, _next) => {
const code = (err && err.statusCode) || 500;
res.status(code).json({
success: false,
error: err && err.message,
code: err && err.code,
});
});
request = require('supertest');
});
afterEach(() => _cleanup(dir));
test('default sendEmail (omitted) returns link and does NOT send email', async () => {
const res = await request(app)
.post('/api/v1/auth/admin/invites')
.send({ email: 'friend@example.com', role: 'operator' });
expect(res.status).toBe(200);
expect(res.body.success).toBe(true);
expect(mockEmailSender.sendEmail).not.toHaveBeenCalled();
expect(res.body.acceptUrl).toMatch(/\/api\/v1\/auth\/invites\/[^/]+\/accept$/);
expect(res.body.deliveredVia).toBe('manual');
});
test('default sendEmail does NOT log raw token to server log', async () => {
const res = await request(app)
.post('/api/v1/auth/admin/invites')
.send({ email: 'friend@example.com', role: 'operator' });
const acceptUrl = res.body.acceptUrl;
// Extract the token from the URL and verify it does NOT appear in any log call.
const token = acceptUrl.match(/invites\/([^/]+)\/accept/)[1];
const tokenLeaked = logCalls.some(c =>
typeof c.msg === 'string' && c.msg.includes(token)
);
expect(tokenLeaked).toBe(false);
// Also assert no log entry mentions the URL verbatim (the old
// `[DC-048-DEV-INVITE-LINK] url=...` spam).
const oldSpam = logCalls.find(c =>
typeof c.msg === 'string' && c.msg.includes('[DC-048-DEV-INVITE-LINK]')
);
expect(oldSpam).toBeUndefined();
});
test('shareText is present and well-formed in every response', async () => {
const res = await request(app)
.post('/api/v1/auth/admin/invites')
.send({ email: 'friend@example.com', role: 'operator', ttlHours: 24 });
expect(res.body.shareText).toBeDefined();
expect(res.body.shareText).toContain('Join my DashCaddy');
expect(res.body.shareText).toContain('operator');
expect(res.body.shareText).toContain(res.body.acceptUrl);
expect(res.body.shareText).toContain('expires in 24h');
});
test('acceptUrl is always returned regardless of sendEmail', async () => {
const r1 = await request(app)
.post('/api/v1/auth/admin/invites')
.send({ email: 'a@x.com', sendEmail: false });
const r2 = await request(app)
.post('/api/v1/auth/admin/invites')
.send({ email: 'b@x.com' });
expect(r1.body.acceptUrl).toBeTruthy();
expect(r2.body.acceptUrl).toBeTruthy();
});
test('sendEmail: true triggers SMTP send when configured', async () => {
// Build a SECOND app instance where emailConfig is a real-looking object,
// so isConfigured() returns true. The first app uses emailConfig=null.
mockEmailSender.isConfigured.mockReturnValueOnce(true);
mockEmailSender.sendEmail.mockResolvedValueOnce(undefined);
const app2 = express();
app2.use(express.json());
app2.use((req, _res, next) => {
req.user = { id: 'admin-id', email: 'admin@sami-host.me', role: 'admin' };
req.app.locals = req.app.locals || {};
req.app.locals.siteConfig = {};
req.app.locals.emailConfig = { host: 'smtp.test', from: 'noreply@test' };
next();
});
const { createUserStore } = require('../src/security/user-store');
const userStore2 = createUserStore({ dataDir: dir });
await userStore2.login({ email: 'admin@sami-host.me' });
const router2 = require('../routes/auth/admin')({
asyncHandler: (fn) => (req, res, next) => Promise.resolve(fn(req, res, next)).catch(next),
errorResponse: (_res, code, msg) => ({ status: code, msg }),
log: { info() {}, warn: (t, m, meta) => logCalls.push({ level: 'warn', topic: t, msg: m, meta }), error() {} },
session: { isSessionValid: () => true, create: () => {}, setCookie: () => {} },
dataDir: dir,
});
app2.use('/api/v1/auth', router2);
const res = await request(app2)
.post('/api/v1/auth/admin/invites')
.send({ email: 'friend@example.com', role: 'viewer', sendEmail: true });
expect(res.status).toBe(200);
expect(mockEmailSender.sendEmail).toHaveBeenCalledTimes(1);
const [_cfg, to, subject, text, html] = mockEmailSender.sendEmail.mock.calls[0];
expect(to).toBe('friend@example.com');
expect(subject).toMatch(/invited/i);
expect(text).toContain(res.body.acceptUrl);
expect(html).toContain(res.body.acceptUrl);
expect(res.body.deliveredVia).toBe('email');
});
test('sendEmail: true + SMTP unconfigured returns deliveredVia:failed and does NOT leak token', async () => {
mockEmailSender.isConfigured.mockReturnValueOnce(false);
const res = await request(app)
.post('/api/v1/auth/admin/invites')
.send({ email: 'friend@example.com', role: 'operator', sendEmail: true });
expect(res.status).toBe(200);
expect(mockEmailSender.sendEmail).not.toHaveBeenCalled();
expect(res.body.deliveredVia).toBe('failed');
// acceptUrl + shareText still present so the operator can share manually.
expect(res.body.acceptUrl).toBeTruthy();
expect(res.body.shareText).toBeTruthy();
// Token does NOT appear in any log call.
const token = res.body.acceptUrl.match(/invites\/([^/]+)\/accept/)[1];
const tokenLeaked = logCalls.some(c =>
typeof c.msg === 'string' && c.msg.includes(token)
);
expect(tokenLeaked).toBe(false);
});
test('invalid role silently defaults to operator (DC-048 behavior preserved)', async () => {
// DC-048: the route's `(role && VALID_ROLES.has(role)) ? role : 'operator'`
// silently substitutes default rather than throwing. This test pins that
// behavior so a future "strict role validation" change is a deliberate
// decision, not a silent regression.
const res = await request(app)
.post('/api/v1/auth/admin/invites')
.send({ email: 'a@x.com', role: 'superuser' });
expect(res.status).toBe(200);
expect(res.body.role).toBe('operator');
expect(mockEmailSender.sendEmail).not.toHaveBeenCalled();
});
test('email validation: missing email still rejected', async () => {
const res = await request(app)
.post('/api/v1/auth/admin/invites')
.send({ role: 'operator' });
expect(res.status).toBe(400);
expect(mockEmailSender.sendEmail).not.toHaveBeenCalled();
});
test('ttlHours: 1 still produces shareText with correct expiry wording', async () => {
const res = await request(app)
.post('/api/v1/auth/admin/invites')
.send({ email: 'a@x.com', ttlHours: 1 });
expect(res.body.shareText).toContain('expires in 1h');
});
});
@@ -0,0 +1,297 @@
/**
* Tests for DC-086: asymmetric hysteresis on the dashboard service badge.
*
* - First probe always emits (no prior state).
* - Same-status probe does NOT re-emit (dedup against repeated green).
* - One "down" then back to "up" keeps the badge green (no flicker).
* - Two consecutive "down" probes flip the badge to red.
* - One "up" after a down streak flips back to green (fast recovery).
* - History retains every raw probe even when no emit happens.
* - getCurrentStatus returns displayed status, not raw.
*/
'use strict';
const path = require('path');
const fs = require('fs');
const os = require('os');
// Use an isolated data dir so test history doesn't pollute the real one.
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'dashcaddy-hyst-'));
process.env.HEALTH_DATA_DIR = tmpDir;
process.env.HEALTH_CONFIG_FILE = path.join(tmpDir, 'health-config.json');
process.env.HEALTH_HISTORY_FILE = path.join(tmpDir, 'health-history.json');
// Module exports a singleton instance, not a class — see module.exports in
// src/monitoring/health-checker.js. The test creates fresh state by replacing
// the relevant maps on the singleton in beforeEach.
const healthCheckerSingleton = require('../src/monitoring/health-checker');
const originalDownThreshold = process.env.HEALTH_DOWN_THRESHOLD;
const originalUpThreshold = process.env.HEALTH_UP_THRESHOLD;
function restoreEnv(name, value) {
if (value === undefined) delete process.env[name];
else process.env[name] = value;
}
function makeUp(serviceId = 'svc1') {
return {
serviceId,
timestamp: new Date().toISOString(),
status: 'up',
responseTime: 50,
statusCode: 200,
message: 'Service is healthy',
details: { headers: {}, bodyLength: 12 }
};
}
function makeDown(serviceId = 'svc1') {
return {
serviceId,
timestamp: new Date().toISOString(),
status: 'down',
responseTime: 50,
statusCode: 500,
message: 'fail',
details: { headers: {}, bodyLength: 0 }
};
}
describe('DC-086: hysteresis on the dashboard badge', () => {
let hc;
let emitSpy;
beforeEach(() => {
// Reset the singleton's per-test state so each case starts clean.
healthCheckerSingleton.displayedStatus = new Map();
healthCheckerSingleton.consecutiveSinceChange = new Map();
healthCheckerSingleton.currentStatus = new Map();
healthCheckerSingleton.history = {};
healthCheckerSingleton.removeAllListeners('status-check');
emitSpy = jest.fn();
healthCheckerSingleton.on('status-check', emitSpy);
hc = healthCheckerSingleton;
});
afterEach(() => {
restoreEnv('HEALTH_DOWN_THRESHOLD', originalDownThreshold);
restoreEnv('HEALTH_UP_THRESHOLD', originalUpThreshold);
});
afterAll(() => {
fs.rmSync(tmpDir, { recursive: true, force: true });
});
test('first probe (no prior state) emits', () => {
hc.recordStatus('svc1', makeUp());
expect(emitSpy).toHaveBeenCalledTimes(1);
expect(emitSpy.mock.calls[0][0].status).toBe('up');
});
test('second probe with same status does NOT re-emit', () => {
hc.recordStatus('svc1', makeUp());
hc.recordStatus('svc1', makeUp());
expect(emitSpy).toHaveBeenCalledTimes(1);
});
test('one "down" then "up" keeps the badge green (the flicker bug)', () => {
hc.recordStatus('svc1', makeUp()); // baseline: green, emit 1
hc.recordStatus('svc1', makeDown()); // one blip — keep green, no emit
hc.recordStatus('svc1', makeUp()); // recovered — still green, no emit
expect(emitSpy).toHaveBeenCalledTimes(1);
expect(hc.displayedStatus.get('svc1').status).toBe('up');
});
test('up, down, up, down, down resets the first streak before flipping', () => {
hc.recordStatus('svc1', makeUp());
hc.recordStatus('svc1', makeDown());
hc.recordStatus('svc1', makeUp());
hc.recordStatus('svc1', makeDown());
expect(hc.displayedStatus.get('svc1').status).toBe('up');
expect(emitSpy).toHaveBeenCalledTimes(1);
hc.recordStatus('svc1', makeDown());
expect(hc.displayedStatus.get('svc1').status).toBe('down');
expect(emitSpy).toHaveBeenCalledTimes(2);
});
test('two consecutive "down" probes flip the badge to red', () => {
hc.recordStatus('svc1', makeUp()); // baseline: green
hc.recordStatus('svc1', makeDown()); // blip #1 — keep green (counter=1)
hc.recordStatus('svc1', makeDown()); // blip #2 — flip red (counter=2 >= DOWN_THRESHOLD)
expect(emitSpy).toHaveBeenCalledTimes(2);
expect(emitSpy.mock.calls[1][0].status).toBe('down');
expect(hc.displayedStatus.get('svc1').status).toBe('down');
});
test('one "up" after a down streak flips back to green (fast recovery)', () => {
hc.recordStatus('svc1', makeUp());
hc.recordStatus('svc1', makeDown());
hc.recordStatus('svc1', makeDown()); // now red
expect(hc.displayedStatus.get('svc1').status).toBe('down');
hc.recordStatus('svc1', makeUp()); // first green — flip back
expect(emitSpy).toHaveBeenCalledTimes(3);
expect(emitSpy.mock.calls[2][0].status).toBe('up');
expect(hc.displayedStatus.get('svc1').status).toBe('up');
});
test('history retains every raw probe even when no emit happens', () => {
hc.recordStatus('svc1', makeUp());
hc.recordStatus('svc1', makeDown()); // blip, no emit
hc.recordStatus('svc1', makeUp()); // recovery, no emit
expect(hc.history['svc1'].length).toBe(3);
expect(hc.history['svc1'][0].status).toBe('up');
expect(hc.history['svc1'][1].status).toBe('down');
expect(hc.history['svc1'][2].status).toBe('up');
});
test('getCurrentStatus returns the displayed status, not the raw probe', () => {
const displayedUp = makeUp();
displayedUp.timestamp = '2026-08-22T09:59:00.000Z';
displayedUp.statusCode = 200;
displayedUp.message = 'healthy';
displayedUp.details = { source: 'accepted-up' };
hc.recordStatus('svc1', displayedUp);
const latestRaw = makeDown();
latestRaw.timestamp = '2026-08-22T10:00:00.000Z';
latestRaw.responseTime = 987;
latestRaw.statusCode = 500;
latestRaw.message = 'failed probe';
latestRaw.error = 'upstream failure';
latestRaw.details = { source: 'suppressed-down' };
hc.recordStatus('svc1', latestRaw); // raw=down, displayed=up
const out = hc.getCurrentStatus();
expect(out['svc1'].status).toBe('up'); // shown to API consumers
expect(out['svc1'].timestamp).toBe(displayedUp.timestamp);
expect(out['svc1'].statusCode).toBe(200);
expect(out['svc1'].message).toBe('healthy');
expect(out['svc1'].error).toBeUndefined();
expect(out['svc1'].details).toEqual({ source: 'accepted-up' });
expect(hc.currentStatus.get('svc1')).toBe(latestRaw);
});
test('a long steady-green run produces exactly ONE emit (no per-probe spam)', () => {
for (let i = 0; i < 50; i++) hc.recordStatus('svc1', makeUp());
expect(emitSpy).toHaveBeenCalledTimes(1);
});
test('a long steady-green-then-steady-red transition: 1 emit (up), 1 emit (red)', () => {
for (let i = 0; i < 10; i++) hc.recordStatus('svc1', makeUp());
expect(emitSpy).toHaveBeenCalledTimes(1);
hc.recordStatus('svc1', makeDown());
hc.recordStatus('svc1', makeDown()); // flips to red
expect(emitSpy).toHaveBeenCalledTimes(2);
for (let i = 0; i < 10; i++) hc.recordStatus('svc1', makeDown());
expect(emitSpy).toHaveBeenCalledTimes(2); // no further broadcasts
});
test('DOWN_THRESHOLD env var is honored', () => {
process.env.HEALTH_DOWN_THRESHOLD = '3';
jest.resetModules();
const HC2Module = require('../src/monitoring/health-checker');
// Module is a singleton with DOWN_THRESHOLD captured at module load —
// resetModules gives us a fresh module-level instance with the new env.
const hc2 = HC2Module;
hc2.displayedStatus = new Map();
hc2.consecutiveSinceChange = new Map();
hc2.currentStatus = new Map();
hc2.history = {};
hc2.removeAllListeners('status-check');
const spy = jest.fn();
hc2.on('status-check', spy);
hc2.recordStatus('svc1', makeUp());
hc2.recordStatus('svc1', makeDown()); // 1
hc2.recordStatus('svc1', makeDown()); // 2 — still green (need 3)
expect(spy).toHaveBeenCalledTimes(1);
expect(hc2.displayedStatus.get('svc1').status).toBe('up');
hc2.recordStatus('svc1', makeDown()); // 3 — flip
expect(spy).toHaveBeenCalledTimes(2);
expect(hc2.displayedStatus.get('svc1').status).toBe('down');
});
test.each(['not-a-number', '0', '-2', '1.5'])('malformed DOWN_THRESHOLD %s falls back to 2', value => {
process.env.HEALTH_DOWN_THRESHOLD = value;
jest.resetModules();
const hc2 = require('../src/monitoring/health-checker');
hc2.displayedStatus = new Map();
hc2.consecutiveSinceChange = new Map();
hc2.currentStatus = new Map();
hc2.history = {};
hc2.removeAllListeners('status-check');
const spy = jest.fn();
hc2.on('status-check', spy);
hc2.recordStatus('svc1', makeUp());
hc2.recordStatus('svc1', makeDown());
expect(spy).toHaveBeenCalledTimes(1);
hc2.recordStatus('svc1', makeDown());
expect(spy).toHaveBeenCalledTimes(2);
});
test('UP_THRESHOLD env var greater than 1 is honored', () => {
process.env.HEALTH_UP_THRESHOLD = '2';
jest.resetModules();
const hc2 = require('../src/monitoring/health-checker');
hc2.displayedStatus = new Map();
hc2.consecutiveSinceChange = new Map();
hc2.currentStatus = new Map();
hc2.history = {};
hc2.removeAllListeners('status-check');
const spy = jest.fn();
hc2.on('status-check', spy);
hc2.recordStatus('svc1', makeDown());
hc2.recordStatus('svc1', makeUp());
expect(spy).toHaveBeenCalledTimes(1);
expect(hc2.displayedStatus.get('svc1').status).toBe('down');
hc2.recordStatus('svc1', makeUp());
expect(spy).toHaveBeenCalledTimes(2);
expect(hc2.displayedStatus.get('svc1').status).toBe('up');
});
test.each(['not-a-number', '0', '-2', '1.5'])('malformed UP_THRESHOLD %s falls back to 1', value => {
process.env.HEALTH_UP_THRESHOLD = value;
jest.resetModules();
const hc2 = require('../src/monitoring/health-checker');
hc2.displayedStatus = new Map();
hc2.consecutiveSinceChange = new Map();
hc2.currentStatus = new Map();
hc2.history = {};
hc2.removeAllListeners('status-check');
const spy = jest.fn();
hc2.on('status-check', spy);
hc2.recordStatus('svc1', makeDown());
hc2.recordStatus('svc1', makeUp());
expect(spy).toHaveBeenCalledTimes(2);
expect(hc2.displayedStatus.get('svc1').status).toBe('up');
});
test('removeService clears hysteresis state before the same ID is re-added', () => {
hc.config.services.svc1 = { name: 'Service 1' };
hc.recordStatus('svc1', makeUp());
hc.recordStatus('svc1', makeDown());
expect(hc.displayedStatus.has('svc1')).toBe(true);
expect(hc.consecutiveSinceChange.get('svc1')).toBe(1);
hc.consecutiveFailures.set('svc1', 3);
const timer = setTimeout(() => {}, 60_000);
hc.serviceTimers.set('svc1', timer);
hc.saveConfig = jest.fn();
hc.removeService('svc1');
expect(hc.displayedStatus.has('svc1')).toBe(false);
expect(hc.consecutiveSinceChange.has('svc1')).toBe(false);
expect(hc.currentStatus.has('svc1')).toBe(false);
expect(hc.consecutiveFailures.has('svc1')).toBe(false);
expect(hc.serviceTimers.has('svc1')).toBe(false);
hc.config.services.svc1 = { name: 'Service 1 re-added' };
const emitSpyAfterReAdd = jest.fn();
hc.on('status-check', emitSpyAfterReAdd);
hc.recordStatus('svc1', makeDown());
expect(emitSpyAfterReAdd).toHaveBeenCalledTimes(1);
expect(hc.displayedStatus.get('svc1').status).toBe('down');
expect(hc.consecutiveSinceChange.has('svc1')).toBe(false);
});
});
@@ -203,6 +203,48 @@ describe('HealthChecker', () => {
expect(result.error).toBe('ECONNREFUSED');
});
it('opens and resolves an outage incident across real checkService transitions', async () => {
healthChecker._doRequest = jest.fn()
.mockResolvedValueOnce({ healthy: true, statusCode: 200, message: 'ok', details: {} })
.mockResolvedValueOnce({ healthy: false, statusCode: 500, message: 'down', details: {} })
.mockResolvedValueOnce({ healthy: true, statusCode: 200, message: 'ok', details: {} });
const config = { url: 'http://test.local' };
await healthChecker.checkService('svc1', config);
await healthChecker.checkService('svc1', config);
expect(healthChecker.incidents).toHaveLength(1);
expect(healthChecker.incidents[0]).toMatchObject({
serviceId: 'svc1',
type: 'outage',
status: 'open'
});
await healthChecker.checkService('svc1', config);
expect(healthChecker.incidents[0].status).toBe('resolved');
expect(healthChecker.incidents[0].resolvedAt).toBeDefined();
});
it('does not resurrect state when an in-flight probe resolves after removal', async () => {
let resolveProbe;
healthChecker.config.services.svc1 = { url: 'http://test.local' };
healthChecker._doRequest = jest.fn(() => new Promise(resolve => {
resolveProbe = resolve;
}));
const pending = healthChecker.checkService('svc1', healthChecker.config.services.svc1);
healthChecker.saveConfig = jest.fn();
healthChecker.removeService('svc1');
resolveProbe({ healthy: true, statusCode: 200, message: 'late', details: {} });
await pending;
expect(healthChecker.currentStatus.has('svc1')).toBe(false);
expect(healthChecker.displayedStatus.has('svc1')).toBe(false);
expect(healthChecker.consecutiveFailures.has('svc1')).toBe(false);
expect(healthChecker.history.svc1).toBeUndefined();
expect(healthChecker.incidents).toEqual([]);
});
it('increments consecutive failures on error', async () => {
healthChecker._doRequest = jest.fn().mockRejectedValue(new Error('fail'));
+34 -37
View File
@@ -32,23 +32,6 @@ const emailSender = require('../../src/auth/providers/email-sender');
const { ValidationError, NotFoundError, ForbiddenError, PaymentRequiredError } = require('../../src/utilities/errors');
const { ok, successMessage } = require('../../src/utils/responses');
/**
* Build the URL an invitee should click. Mirrors EmailMagicLinkProvider's
* _resolvePublicUrl logic kept duplicated (not extracted) because the two
* callers have slightly different link paths and the duplication is smaller
* than the abstraction would be.
*/
function _buildInviteUrl(req, siteConfig, token) {
if (siteConfig && siteConfig.publicBaseUrl) {
return siteConfig.publicBaseUrl.replace(/\/+$/, '') +
'/api/v1/auth/invites/' + encodeURIComponent(token) + '/accept';
}
const proto = (req.headers && req.headers['x-forwarded-proto']) || (req.protocol || 'https');
const host = (req.headers && (req.headers['x-forwarded-host'] || req.headers.host))
|| (siteConfig && siteConfig.dashboardHost) || 'localhost:3001';
return `${proto}://${host}/api/v1/auth/invites/${encodeURIComponent(token)}/accept`;
}
function _requireAdmin(req, _res, next) {
if (!req.user || req.user.role !== 'admin') {
return next(new ForbiddenError('Admin role required'));
@@ -240,11 +223,21 @@ module.exports = function({ asyncHandler, errorResponse, log, session, dataDir }
});
if (!issued.ok) throw new ValidationError(issued.reason, 'email');
let deliveredVia = 'none';
const maskedEmail = email.replace(/(^.).+(@.*$)/, '$1***$2');
if (sendEmail !== false) {
// Best-effort send. If SMTP isn't configured, log to error.log (dev path).
const acceptUrl = _buildInviteUrl(req, /* siteConfig */ req.app.locals && req.app.locals.siteConfig, issued.token);
// Build the accept URL once — used both for the response and for email delivery.
const baseUrl = (req.app.locals && req.app.locals.siteConfig && req.app.locals.siteConfig.publicBaseUrl
? req.app.locals.siteConfig.publicBaseUrl.replace(/\/+$/, '')
: ((req.headers['x-forwarded-proto'] || req.protocol || 'https') + '://' +
(req.headers['x-forwarded-host'] || req.headers.host || 'localhost:3001')));
const acceptUrl = baseUrl + '/api/v1/auth/invites/' + encodeURIComponent(issued.token) + '/accept';
// DC-085: link-first delivery. Default = no email, just hand the link back.
// Operators opt INTO email by sending { sendEmail: true } (or the admin UI
// checks the "Send email" checkbox). When SMTP is unconfigured AND the
// operator did opt in, we surface the failure as `deliveredVia: 'failed'`
// but NEVER leak the raw token into the server log — the link is already
// in the response, so the operator has a UI-side fallback.
let deliveredVia = 'manual';
if (sendEmail === true) {
const ttlHoursOut = Math.round(issued.ttlMs / (60 * 60 * 1000));
const text = _buildEmailText({ acceptUrl, ttlHours: ttlHoursOut, role: issued.role });
const html = _buildEmailHtml({ acceptUrl, ttlHours: ttlHoursOut, role: issued.role });
@@ -254,33 +247,37 @@ module.exports = function({ asyncHandler, errorResponse, log, session, dataDir }
await emailSender.sendEmail(smtpConfig, issued.email, 'You\'re invited to DashCaddy', text, html);
deliveredVia = 'email';
} else {
// Dev fallback — log the raw link so operators can grab it.
log.warn && log.warn('auth-invite-dev',
'[DC-048-DEV-INVITE-LINK] email=' + issued.email +
' role=' + issued.role + ' url=' + acceptUrl);
deliveredVia = 'dev-console';
// Operator asked for email but SMTP isn't configured. Surface the
// failure cleanly; the link is still in the response so the
// operator can share it manually. Do NOT log the raw URL — it
// would duplicate what's already in the response and pollute the
// server log on every unconfigured-install invite.
log.warn && log.warn('auth-invite-send',
'invite send skipped: SMTP not configured (operator opted in)',
{ inviteId: issued.id, email: issued.email });
deliveredVia = 'failed';
}
} catch (sendErr) {
log.warn && log.warn('auth-invite-send',
'invite send failed: ' + (sendErr.message || String(sendErr)));
'invite send failed: ' + (sendErr.message || String(sendErr)),
{ inviteId: issued.id });
deliveredVia = 'failed';
}
} else {
deliveredVia = 'manual';
}
const maskedEmail = email.replace(/(^.).+(@.*$)/, '$1***$2');
const ttlHoursOut = Math.round(issued.ttlMs / (60 * 60 * 1000));
const shareText =
'Join my DashCaddy as ' + issued.role + ' — ' + acceptUrl +
' — expires in ' + ttlHoursOut + 'h.';
return ok(res, {
id: issued.id,
email: issued.email,
role: issued.role,
expiresAt: issued.expiresAt,
// The raw token is returned ONCE so the admin UI can show/copy the
// link. It is also embedded in the email when sendEmail !== false.
acceptUrl: (req.app.locals && req.app.locals.siteConfig && req.app.locals.siteConfig.publicBaseUrl
? req.app.locals.siteConfig.publicBaseUrl.replace(/\/+$/, '')
: ((req.headers['x-forwarded-proto'] || req.protocol || 'https') + '://' +
(req.headers['x-forwarded-host'] || req.headers.host || 'localhost:3001'))) +
'/api/v1/auth/invites/' + encodeURIComponent(issued.token) + '/accept',
acceptUrl,
shareText,
deliveredVia,
maskedEmail,
});
+149 -11
View File
@@ -33,17 +33,45 @@ const MAX_CHECK_INTERVAL = parseInt(process.env.HEALTH_CHECK_MAX_INTERVAL || '30
const MAX_ENTRIES_PER_SERVICE = parseInt(process.env.HEALTH_MAX_ENTRIES || '500', 10); // Cap to prevent disk explosion
const HISTORY_RETENTION_DAYS = parseInt(process.env.HEALTH_HISTORY_RETENTION || '30', 10);
// DC-086: hysteresis thresholds for badge display.
// The raw probe result can flap on a single transient blip (Caddy reload,
// container CPU steal, network hiccup, mid-flight TLS handshake). Showing
// every probe result as-is to the dashboard creates the "perpetual flicker"
// UX. Asymmetric thresholds: going red is slow (don't false-alarm), going
// green is fast (don't keep showing red after recovery).
// - DOWN_THRESHOLD = N consecutive "down" probes before the badge flips to red
// - UP_THRESHOLD = N consecutive "up" probes before the badge flips back to green
// Single probe flips to green on purpose — false-positive-green is much less
// painful than perpetual-red (operators notice red, ignore green).
function readPositiveIntEnv(name, fallback) {
const raw = process.env[name];
if (raw === undefined || raw === '') return fallback;
const value = Number(raw);
return Number.isSafeInteger(value) && value >= 1 ? value : fallback;
}
const DOWN_THRESHOLD = readPositiveIntEnv('HEALTH_DOWN_THRESHOLD', 2);
const UP_THRESHOLD = readPositiveIntEnv('HEALTH_UP_THRESHOLD', 1);
class HealthChecker extends EventEmitter {
constructor() {
super();
this.config = this.loadConfig();
this.history = this.loadHistory();
this.currentStatus = new Map();
// DC-086: the status the dashboard SHOULD display (post-hysteresis).
// Distinct from currentStatus, which is the latest raw probe result.
this.displayedStatus = new Map();
// DC-086: counter of consecutive healthy/unhealthy probes since the
// last displayed-status change. Reset to 0 whenever displayed status flips.
this.consecutiveSinceChange = new Map();
this.incidents = [];
this.checking = false;
this.checkInterval = null;
this.consecutiveFailures = new Map(); // serviceId -> failure count
this.serviceTimers = new Map(); // serviceId -> timer for per-service backoff
// Invalidate probe completions that race with removal/reconfiguration.
this.serviceGenerations = new Map(); // serviceId -> configuration generation
}
/**
@@ -116,6 +144,7 @@ class HealthChecker extends EventEmitter {
*/
async checkService(serviceId, config) {
const startTime = Date.now();
const generation = this.serviceGenerations.get(serviceId) || 0;
try {
const result = await this.performHealthCheck(config);
@@ -131,6 +160,10 @@ class HealthChecker extends EventEmitter {
details: result.details
};
if ((this.serviceGenerations.get(serviceId) || 0) !== generation) {
return status;
}
// Track consecutive failures for exponential backoff
if (result.healthy) {
this.consecutiveFailures.delete(serviceId);
@@ -138,8 +171,9 @@ class HealthChecker extends EventEmitter {
this.consecutiveFailures.set(serviceId, (this.consecutiveFailures.get(serviceId) || 0) + 1);
}
const previousStatus = this.currentStatus.get(serviceId);
this.recordStatus(serviceId, status);
this.checkForIncidents(serviceId, status, config);
this.checkForIncidents(serviceId, status, config, previousStatus);
return status;
} catch (error) {
@@ -156,8 +190,13 @@ class HealthChecker extends EventEmitter {
error: error.message
};
if ((this.serviceGenerations.get(serviceId) || 0) !== generation) {
return status;
}
const previousStatus = this.currentStatus.get(serviceId);
this.recordStatus(serviceId, status);
this.checkForIncidents(serviceId, status, config);
this.checkForIncidents(serviceId, status, config, previousStatus);
return status;
}
@@ -273,14 +312,81 @@ class HealthChecker extends EventEmitter {
return true;
}
/**
* Compute the displayed status for a service given the latest raw probe
* result. Applies asymmetric hysteresis:
* - Going DOWN: requires DOWN_THRESHOLD (default 2) consecutive "down"
* probes since the last display-state change. A single blip keeps the
* badge green.
* - Going UP: requires UP_THRESHOLD (default 1) consecutive "up" probes.
* Any single "up" after a down streak flips back to green so the badge
* doesn't linger red after the service has recovered.
*
* Returns the displayed status object (same shape as the raw status) so
* recordStatus can use it both for the displayed map and as the broadcast
* payload when the displayed status actually changes.
*/
_computeDisplayedStatus(serviceId, rawStatus) {
const currentDisplayed = this.displayedStatus.get(serviceId);
const previousStatus = currentDisplayed ? currentDisplayed.status : null;
// If no prior state, accept the raw probe as-is (first-check bootstrap).
if (!previousStatus) {
return rawStatus;
}
// Probe agrees with current displayed → no change, reset the counter so
// a brief blip doesn't accumulate against the displayed state.
if (rawStatus.status === previousStatus) {
this.consecutiveSinceChange.set(serviceId, 0);
return rawStatus;
}
// Probe disagrees with displayed. Bump the streak counter — this counts
// CONSECUTIVE probes that disagree with what's shown, regardless of
// whether the raw value itself changed between probes. That's what
// makes "down, down" flip after threshold but "down, up, down" not flip.
const prev = this.consecutiveSinceChange.get(serviceId) || 0;
const next = prev + 1;
if (rawStatus.status === 'down') {
// Going DOWN: need DOWN_THRESHOLD consecutive probes that disagree
// with the displayed "up" state.
if (previousStatus === 'up' && next < DOWN_THRESHOLD) {
this.consecutiveSinceChange.set(serviceId, next);
// Keep the last internally-consistent displayed snapshot. Mixing the
// raw failure metadata with status="up" would expose contradictory
// API data (for example statusCode=500 on an "up" service).
return currentDisplayed;
}
// Threshold met (or already down) — flip to red.
this.consecutiveSinceChange.set(serviceId, 0);
return rawStatus;
}
// rawStatus.status === 'up' (must be — the equal-to-displayed case above
// already returned). Going UP after a down streak: need UP_THRESHOLD.
if (previousStatus === 'down' && next < UP_THRESHOLD) {
this.consecutiveSinceChange.set(serviceId, next);
return currentDisplayed;
}
this.consecutiveSinceChange.set(serviceId, 0);
return rawStatus;
}
/**
* Record service status
*
* DC-086: history + consecutiveFailures are updated for EVERY probe
* (operators want full probe history for postmortems). The dashboard's
* `status-check` event is only emitted when the DISPLAYED status changes,
* so the badge stops re-rendering on every probe.
*/
recordStatus(serviceId, status) {
// Update current status
// Update current (raw) status — used by checkForIncidents and history.
this.currentStatus.set(serviceId, status);
// Add to history
// Add raw probe to history (full fidelity — operators rely on this).
if (!this.history[serviceId]) {
this.history[serviceId] = [];
}
@@ -292,8 +398,23 @@ class HealthChecker extends EventEmitter {
this.history[serviceId] = this.history[serviceId].slice(-MAX_ENTRIES_PER_SERVICE);
}
// Emit status event
this.emit('status-check', status);
// Compute the post-hysteresis displayed status; only emit when it changes.
// _computeDisplayedStatus compares the raw probe against the DISPLAYED
// status (not the previous raw status), so the "consecutive since
// change" counter doesn't depend on the order of writes here.
const displayed = this._computeDisplayedStatus(serviceId, status);
const previousDisplayed = this.displayedStatus.get(serviceId);
const displayChanged =
!previousDisplayed || previousDisplayed.status !== displayed.status;
this.displayedStatus.set(serviceId, displayed);
if (displayChanged) {
// Emit with the displayed status so the dashboard renders the same
// state the hysteresis just decided. The raw probe result is still
// in `history` and `currentStatus` for anyone who wants it.
this.emit('status-check', displayed);
}
// Save history periodically
if (Math.random() < 0.05) { // 5% chance (every ~20 checks)
@@ -304,8 +425,7 @@ class HealthChecker extends EventEmitter {
/**
* Check for incidents (downtime, slow response, etc.)
*/
checkForIncidents(serviceId, status, config) {
const previous = this.currentStatus.get(serviceId);
checkForIncidents(serviceId, status, config, previous = this.currentStatus.get(serviceId)) {
// Check for status change (up -> down or down -> up)
if (previous && previous.status !== status.status) {
@@ -445,19 +565,29 @@ class HealthChecker extends EventEmitter {
}
/**
* Get current status for all services
* Get current status for all services.
*
* DC-086: returns the DISPLAYED status (post-hysteresis), not the latest
* raw probe. A page reload should show the same badge state the live
* SSE stream is currently showing otherwise an operator who reloads
* the page after a single blip sees red even though the hysteresis kept
* the badge green for them.
*/
getCurrentStatus() {
const result = {};
for (const [serviceId, status] of this.currentStatus.entries()) {
for (const [serviceId, rawStatus] of this.currentStatus.entries()) {
const config = this.config.services[serviceId];
const uptime24h = this.calculateUptime(serviceId, 24);
const uptime7d = this.calculateUptime(serviceId, 168);
const avgResponseTime = this.calculateAverageResponseTime(serviceId, 24);
// Prefer the displayed status if we've already computed one; fall back
// to the raw probe on the very first call (before recordStatus has run).
const displayed = this.displayedStatus.get(serviceId) || rawStatus;
result[serviceId] = {
...status,
...displayed,
name: config?.name || serviceId,
uptime: {
'24h': uptime24h,
@@ -530,6 +660,7 @@ class HealthChecker extends EventEmitter {
this.config.services = {};
}
this.serviceGenerations.set(serviceId, (this.serviceGenerations.get(serviceId) || 0) + 1);
this.config.services[serviceId] = {
enabled: config.enabled !== false,
name: config.name || serviceId,
@@ -552,12 +683,19 @@ class HealthChecker extends EventEmitter {
* Remove service configuration
*/
removeService(serviceId) {
this.serviceGenerations.set(serviceId, (this.serviceGenerations.get(serviceId) || 0) + 1);
if (this.config.services) {
delete this.config.services[serviceId];
this.saveConfig();
}
this.currentStatus.delete(serviceId);
this.displayedStatus.delete(serviceId);
this.consecutiveSinceChange.delete(serviceId);
this.consecutiveFailures.delete(serviceId);
const timer = this.serviceTimers.get(serviceId);
if (timer) clearTimeout(timer);
this.serviceTimers.delete(serviceId);
delete this.history[serviceId];
}
+65 -4
View File
@@ -216,8 +216,8 @@
_el('input', { name: 'ttlHours', type: 'number', min: '1', max: '168', value: '24', style: 'padding:6px;width:80px' }),
));
form.appendChild(_el('label', { style: 'display:flex;gap:4px;align-items:center;font-size:0.85rem' },
_el('input', { name: 'sendEmail', type: 'checkbox', checked: true }),
_el('span', { text: 'Send email' }),
_el('input', { name: 'sendEmail', type: 'checkbox', checked: false }),
_el('span', { text: 'Also send via email (optional)' }),
));
form.appendChild(_el('button', { type: 'submit', class: 'btn-sm', style: 'padding:6px 12px', text: 'Issue invite' }));
container.appendChild(form);
@@ -297,16 +297,77 @@
},
});
banner.appendChild(copyBtn);
if (invite.deliveredVia === 'dev-console') {
// DC-085: pre-formatted message for one-tap paste into iMessage / WhatsApp /
// Telegram / SMS / Signal / Discord / paste-into-email. The operator can
// copy this as a sentence instead of dealing with the raw URL.
if (invite.shareText) {
const shareBlock = _el('div', { style: 'margin-top:12px' });
shareBlock.appendChild(_el('div', {
style: 'font-size:0.8rem;color:#86efac;margin-bottom:4px',
text: 'Share this message:',
}));
shareBlock.appendChild(_el('div', {
style: 'padding:8px;background:#000;border-radius:4px;color:#d1fae5;white-space:pre-wrap',
text: invite.shareText,
}));
const shareActions = _el('div', { style: 'margin-top:6px;display:flex;gap:6px;flex-wrap:wrap' });
const copyTextBtn = _el('button', {
class: 'btn-sm', style: 'padding:4px 10px',
text: 'Copy message',
onclick: async () => {
try {
await navigator.clipboard.writeText(invite.shareText);
copyTextBtn.textContent = 'Copied!';
setTimeout(() => { copyTextBtn.textContent = 'Copy message'; }, 2000);
} catch (e) {
window.errorHandler && window.errorHandler.show('Clipboard blocked: select the text manually.');
}
},
});
shareActions.appendChild(copyTextBtn);
// Native share sheet on mobile / supported browsers. Falls back silently
// (the copy buttons cover the same intent).
if (typeof navigator !== 'undefined' && typeof navigator.share === 'function') {
const nativeShareBtn = _el('button', {
class: 'btn-sm', style: 'padding:4px 10px',
text: 'Share via…',
onclick: async () => {
try {
await navigator.share({
title: 'DashCaddy invite',
text: invite.shareText,
url: invite.acceptUrl,
});
} catch (e) {
// User-cancelled throws AbortError — that's fine, just stay quiet.
if (e && e.name && e.name !== 'AbortError') {
window.errorHandler && window.errorHandler.show('Share failed: ' + e.message);
}
}
},
});
shareActions.appendChild(nativeShareBtn);
}
shareBlock.appendChild(shareActions);
banner.appendChild(shareBlock);
}
if (invite.deliveredVia === 'failed') {
banner.appendChild(_el('p', {
style: 'margin-top:8px;color:#fbbf24;font-size:0.8rem',
text: 'SMTP not configured the invite was logged to the server console (search for [DC-048-DEV-INVITE-LINK]).',
text: 'Email could not be sent (SMTP not configured). Share the link above instead — it works the same way.',
}));
} else if (invite.deliveredVia === 'email') {
banner.appendChild(_el('p', {
style: 'margin-top:8px;color:#86efac;font-size:0.8rem',
text: 'Email sent to ' + invite.email + '.',
}));
} else if (invite.deliveredVia === 'manual') {
banner.appendChild(_el('p', {
style: 'margin-top:8px;color:#86efac;font-size:0.8rem',
text: 'Share the link above via text, chat, or any messenger.',
}));
}
parent.appendChild(banner);
}