Merge dc/DC-085-link-first-invite-dc-086-flicker-fix: DC-085 link-first invite + DC-086 badge flicker hysteresis + race hardening [glm-grade=A]
This commit is contained in:
+16
@@ -401,6 +401,22 @@ Tickets DC-046 through DC-049 implement pluggable auth + email magic link. Sami
|
||||
- **result:** Shipped codex-graded A. All 49 `console.*` sites in `src/managers/update-manager.js` now route through `log.info/log.warn/log.error` from `src/utils/logging` (tag = `'update'`). Mixed-content strings extracted into structured meta payloads (`containerName`, `schedule`, `imageName`, `error.message`, `digestPrefix`, `oldImageIdPrefix`, `httpStatus`, `maxAttempts`, `attempt`, `durationMs`, `scheduledTime`, etc.) so fields are queryable instead of inlined into the message. Errors now go through `log.error(ctx, errObj)` so they land in error.log with full stack trace + context, not just stderr. 1539/1539 Jest tests pass (78/78 update-manager tests still pass). ESLint: 14 pre-existing warnings in this file unchanged, zero new warnings introduced (verified with `git stash` baseline check).
|
||||
|
||||
|
||||
### DC-086: Service-status flicker fix — asymmetric hysteresis on the badge
|
||||
- **status:** in-progress
|
||||
- **owner:** hermes
|
||||
- **details:** Dashboard service badges perpetually flip between green and red for "a few seconds at a time, never stable" (Sami's report, 2026-08-20). Root cause: `src/monitoring/health-checker.js` `recordStatus()` emits `'status-check'` on EVERY probe (every 30s), and `src/websocket/dashboard-ws.js` forwards every probe as `'status-change'` to the browser with no diff. The frontend `live-events.js` then unconditionally calls `setBadge()` — which resets the icon + pill text on every event. A single transient 5xx (Caddy reload, container CPU steal, mid-flight TLS handshake, container restart during probe) flips the badge red and the next green probe flips it back. Fix: add asymmetric hysteresis in `_computeDisplayedStatus(serviceId, rawStatus)` — going DOWN requires 2 consecutive "down" probes (default `HEALTH_DOWN_THRESHOLD=2`), going UP requires only 1 (default `HEALTH_UP_THRESHOLD=1`). History + `consecutiveFailures` still record raw probe results (operators want full fidelity for postmortems); only the dashboard broadcast is filtered. `getCurrentStatus()` now returns the displayed status so a page reload shows the same badge as the live SSE stream. Both thresholds are env-var configurable so operators can tune. New tests in `__tests__/health-checker-hysteresis.test.js` cover: first probe emits; second probe same-status does NOT re-emit; one-down-then-up keeps green; two-down flips to red; one-up after down flips back to green; `getCurrentStatus` returns displayed not raw. Effort: ~30 min. Risk: low — pure behavior filter, no schema breaks, all 63 existing health-checker tests must stay green.
|
||||
- **impact:** Operators stop seeing perpetual red/green flicker on healthy services. Real outages still get flagged (2 consecutive 30s probes = ~60s before badge flips red, which is still faster than a human notices). Background probe history is unchanged so postmortem analysis still works.
|
||||
- **prerequisite:** None.
|
||||
- **result:** _pending — ship + codex round_
|
||||
|
||||
### DC-085: Link-first invite — Discord-style "share it however you want"
|
||||
- **status:** in-progress
|
||||
- **owner:** hermes
|
||||
- **details:** Today `POST /api/v1/auth/admin/invites` defaults to sending the invite link via SMTP; if SMTP is not configured it spams the server console with `[DC-048-DEV-INVITE-LINK]` log lines. Sami wants Discord-style: the link is always returned in the response, and email is an opt-in checkbox. Operators should be free to copy the link and share it via iMessage / SMS / WhatsApp / Telegram / Signal / Discord / paste-in-email — whatever fits. (1) Flip default `sendEmail !== false` to `sendEmail === true` in `routes/auth/admin.js` so omitting the field means "no email, just hand me the link." (2) Stop logging the raw invite URL to error.log when SMTP is unconfigured — that path was only useful when there was no UI way to grab the link; now there is. (3) Add a `shareText` field to the response: `"Join my DashCaddy as <role> — <acceptUrl> — expires in Nh."` for one-tap paste into any messenger. (4) Frontend: `status/js/admin.js` `_renderInviteForm` flips the "Send email" checkbox default to **unchecked**, updates `_renderIssuedInviteBanner` to show both the raw link AND the shareText (with its own copy button + `navigator.share()` native share-sheet button where available). (5) New tests in `__tests__/admin-invites.test.js` covering: default sendEmail=false (no SMTP send attempted, no console log); `sendEmail: true` triggers SMTP send; `shareText` is present and well-formed; `acceptUrl` is always returned; expired sendEmail path doesn't leak token to logs. Effort: ~1 hr. Risk: low — pure behavior flip + UI additive change.
|
||||
- **impact:** Closes the friction between "host wants to add a friend" and "host has to configure SMTP first." Mirrors Discord/Slack/Linear invite flows where the link IS the deliverable. No new tier changes, no schema breaks.
|
||||
- **prerequisite:** DC-048 (invite store + admin route), DC-052 (Pro gate stays).
|
||||
- **result:** _pending — ship + codex round_
|
||||
|
||||
### DC-084: Remove redundant active Caddy health check from `arch.sami` site — eliminate 6 syslog spam lines/min
|
||||
- **status:** done
|
||||
- **owner:** hermes
|
||||
|
||||
@@ -0,0 +1,240 @@
|
||||
/**
|
||||
* Tests for DC-085: link-first invite (Discord-style "share it however you want").
|
||||
*
|
||||
* - default sendEmail omission = no email sent, link returned, no token in logs
|
||||
* - sendEmail:true triggers SMTP send when configured
|
||||
* - sendEmail:true + SMTP unconfigured = deliveredVia:'failed', no token leaked
|
||||
* - shareText field present and well-formed in every response
|
||||
* - acceptUrl always present (regardless of sendEmail)
|
||||
* - role + ttl validation unchanged from DC-048
|
||||
*
|
||||
* Strategy: drive the route handler directly with mock req/res, mount the admin
|
||||
* router against an isolated userStore + inviteStore + email-sender stub.
|
||||
*/
|
||||
|
||||
'use strict';
|
||||
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const os = require('os');
|
||||
const express = require('express');
|
||||
|
||||
function _tmpDir() {
|
||||
return fs.mkdtempSync(path.join(os.tmpdir(), 'dashcaddy-admin-invites-test-'));
|
||||
}
|
||||
function _cleanup(dir) {
|
||||
try { fs.rmSync(dir, { recursive: true, force: true }); } catch {}
|
||||
}
|
||||
|
||||
// Stub email-sender so we can assert "was it called?" without an SMTP server.
|
||||
// NOTE: the variable name MUST start with `mock` so Jest's hoisted `jest.mock()`
|
||||
// call is allowed to reference it (Babel guard against out-of-scope access).
|
||||
const mockEmailSender = {
|
||||
isConfigured: jest.fn(() => false),
|
||||
sendEmail: jest.fn(async () => undefined),
|
||||
};
|
||||
jest.mock('../src/auth/providers/email-sender', () => mockEmailSender);
|
||||
|
||||
describe('DC-085: link-first admin invites', () => {
|
||||
let dir, app, request;
|
||||
let logCalls; // captured { level, msg, meta } from our fake log
|
||||
|
||||
beforeEach(async () => {
|
||||
jest.clearAllMocks();
|
||||
dir = _tmpDir();
|
||||
logCalls = [];
|
||||
|
||||
// Set up email auth enable flag so userStore mounts.
|
||||
process.env.NODE_ENV = 'test';
|
||||
|
||||
const { createUserStore } = require('../src/security/user-store');
|
||||
const userStore = createUserStore({ dataDir: dir });
|
||||
|
||||
// Bootstrap the admin so we have a session-attributable user.
|
||||
await userStore.login({ email: 'admin@sami-host.me' });
|
||||
|
||||
// Build a tiny Express app with the admin router mounted, but skip the
|
||||
// global auth gate (we inject req.user directly).
|
||||
const adminRouter = require('../routes/auth/admin')({
|
||||
asyncHandler: (fn) => (req, res, next) => Promise.resolve(fn(req, res, next)).catch(next),
|
||||
errorResponse: (_res, code, msg) => ({ status: code, msg }),
|
||||
log: {
|
||||
info: (topic, msg, meta) => logCalls.push({ level: 'info', topic, msg, meta }),
|
||||
warn: (topic, msg, meta) => logCalls.push({ level: 'warn', topic, msg, meta }),
|
||||
error: (topic, msg, meta) => logCalls.push({ level: 'error', topic, msg, meta }),
|
||||
},
|
||||
session: { isSessionValid: () => true, create: () => {}, setCookie: () => {} },
|
||||
dataDir: dir,
|
||||
});
|
||||
|
||||
app = express();
|
||||
app.use(express.json());
|
||||
// Inject req.user = admin so /admin/* passes the role gate.
|
||||
app.use((req, _res, next) => {
|
||||
req.user = { id: 'admin-id', email: 'admin@sami-host.me', role: 'admin' };
|
||||
req.app.locals = req.app.locals || {};
|
||||
req.app.locals.siteConfig = {}; // no publicBaseUrl — route uses req.headers
|
||||
req.app.locals.emailConfig = null; // SMTP not configured by default
|
||||
next();
|
||||
});
|
||||
app.use('/api/v1/auth', adminRouter);
|
||||
// Error handler — last in chain.
|
||||
app.use((err, _req, res, _next) => {
|
||||
const code = (err && err.statusCode) || 500;
|
||||
res.status(code).json({
|
||||
success: false,
|
||||
error: err && err.message,
|
||||
code: err && err.code,
|
||||
});
|
||||
});
|
||||
|
||||
request = require('supertest');
|
||||
});
|
||||
|
||||
afterEach(() => _cleanup(dir));
|
||||
|
||||
test('default sendEmail (omitted) returns link and does NOT send email', async () => {
|
||||
const res = await request(app)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ email: 'friend@example.com', role: 'operator' });
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.success).toBe(true);
|
||||
expect(mockEmailSender.sendEmail).not.toHaveBeenCalled();
|
||||
expect(res.body.acceptUrl).toMatch(/\/api\/v1\/auth\/invites\/[^/]+\/accept$/);
|
||||
expect(res.body.deliveredVia).toBe('manual');
|
||||
});
|
||||
|
||||
test('default sendEmail does NOT log raw token to server log', async () => {
|
||||
const res = await request(app)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ email: 'friend@example.com', role: 'operator' });
|
||||
|
||||
const acceptUrl = res.body.acceptUrl;
|
||||
// Extract the token from the URL and verify it does NOT appear in any log call.
|
||||
const token = acceptUrl.match(/invites\/([^/]+)\/accept/)[1];
|
||||
const tokenLeaked = logCalls.some(c =>
|
||||
typeof c.msg === 'string' && c.msg.includes(token)
|
||||
);
|
||||
expect(tokenLeaked).toBe(false);
|
||||
|
||||
// Also assert no log entry mentions the URL verbatim (the old
|
||||
// `[DC-048-DEV-INVITE-LINK] url=...` spam).
|
||||
const oldSpam = logCalls.find(c =>
|
||||
typeof c.msg === 'string' && c.msg.includes('[DC-048-DEV-INVITE-LINK]')
|
||||
);
|
||||
expect(oldSpam).toBeUndefined();
|
||||
});
|
||||
|
||||
test('shareText is present and well-formed in every response', async () => {
|
||||
const res = await request(app)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ email: 'friend@example.com', role: 'operator', ttlHours: 24 });
|
||||
|
||||
expect(res.body.shareText).toBeDefined();
|
||||
expect(res.body.shareText).toContain('Join my DashCaddy');
|
||||
expect(res.body.shareText).toContain('operator');
|
||||
expect(res.body.shareText).toContain(res.body.acceptUrl);
|
||||
expect(res.body.shareText).toContain('expires in 24h');
|
||||
});
|
||||
|
||||
test('acceptUrl is always returned regardless of sendEmail', async () => {
|
||||
const r1 = await request(app)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ email: 'a@x.com', sendEmail: false });
|
||||
const r2 = await request(app)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ email: 'b@x.com' });
|
||||
expect(r1.body.acceptUrl).toBeTruthy();
|
||||
expect(r2.body.acceptUrl).toBeTruthy();
|
||||
});
|
||||
|
||||
test('sendEmail: true triggers SMTP send when configured', async () => {
|
||||
// Build a SECOND app instance where emailConfig is a real-looking object,
|
||||
// so isConfigured() returns true. The first app uses emailConfig=null.
|
||||
mockEmailSender.isConfigured.mockReturnValueOnce(true);
|
||||
mockEmailSender.sendEmail.mockResolvedValueOnce(undefined);
|
||||
const app2 = express();
|
||||
app2.use(express.json());
|
||||
app2.use((req, _res, next) => {
|
||||
req.user = { id: 'admin-id', email: 'admin@sami-host.me', role: 'admin' };
|
||||
req.app.locals = req.app.locals || {};
|
||||
req.app.locals.siteConfig = {};
|
||||
req.app.locals.emailConfig = { host: 'smtp.test', from: 'noreply@test' };
|
||||
next();
|
||||
});
|
||||
const { createUserStore } = require('../src/security/user-store');
|
||||
const userStore2 = createUserStore({ dataDir: dir });
|
||||
await userStore2.login({ email: 'admin@sami-host.me' });
|
||||
const router2 = require('../routes/auth/admin')({
|
||||
asyncHandler: (fn) => (req, res, next) => Promise.resolve(fn(req, res, next)).catch(next),
|
||||
errorResponse: (_res, code, msg) => ({ status: code, msg }),
|
||||
log: { info() {}, warn: (t, m, meta) => logCalls.push({ level: 'warn', topic: t, msg: m, meta }), error() {} },
|
||||
session: { isSessionValid: () => true, create: () => {}, setCookie: () => {} },
|
||||
dataDir: dir,
|
||||
});
|
||||
app2.use('/api/v1/auth', router2);
|
||||
|
||||
const res = await request(app2)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ email: 'friend@example.com', role: 'viewer', sendEmail: true });
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
expect(mockEmailSender.sendEmail).toHaveBeenCalledTimes(1);
|
||||
const [_cfg, to, subject, text, html] = mockEmailSender.sendEmail.mock.calls[0];
|
||||
expect(to).toBe('friend@example.com');
|
||||
expect(subject).toMatch(/invited/i);
|
||||
expect(text).toContain(res.body.acceptUrl);
|
||||
expect(html).toContain(res.body.acceptUrl);
|
||||
expect(res.body.deliveredVia).toBe('email');
|
||||
});
|
||||
|
||||
test('sendEmail: true + SMTP unconfigured returns deliveredVia:failed and does NOT leak token', async () => {
|
||||
mockEmailSender.isConfigured.mockReturnValueOnce(false);
|
||||
|
||||
const res = await request(app)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ email: 'friend@example.com', role: 'operator', sendEmail: true });
|
||||
|
||||
expect(res.status).toBe(200);
|
||||
expect(mockEmailSender.sendEmail).not.toHaveBeenCalled();
|
||||
expect(res.body.deliveredVia).toBe('failed');
|
||||
// acceptUrl + shareText still present so the operator can share manually.
|
||||
expect(res.body.acceptUrl).toBeTruthy();
|
||||
expect(res.body.shareText).toBeTruthy();
|
||||
// Token does NOT appear in any log call.
|
||||
const token = res.body.acceptUrl.match(/invites\/([^/]+)\/accept/)[1];
|
||||
const tokenLeaked = logCalls.some(c =>
|
||||
typeof c.msg === 'string' && c.msg.includes(token)
|
||||
);
|
||||
expect(tokenLeaked).toBe(false);
|
||||
});
|
||||
|
||||
test('invalid role silently defaults to operator (DC-048 behavior preserved)', async () => {
|
||||
// DC-048: the route's `(role && VALID_ROLES.has(role)) ? role : 'operator'`
|
||||
// silently substitutes default rather than throwing. This test pins that
|
||||
// behavior so a future "strict role validation" change is a deliberate
|
||||
// decision, not a silent regression.
|
||||
const res = await request(app)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ email: 'a@x.com', role: 'superuser' });
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.role).toBe('operator');
|
||||
expect(mockEmailSender.sendEmail).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('email validation: missing email still rejected', async () => {
|
||||
const res = await request(app)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ role: 'operator' });
|
||||
expect(res.status).toBe(400);
|
||||
expect(mockEmailSender.sendEmail).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
test('ttlHours: 1 still produces shareText with correct expiry wording', async () => {
|
||||
const res = await request(app)
|
||||
.post('/api/v1/auth/admin/invites')
|
||||
.send({ email: 'a@x.com', ttlHours: 1 });
|
||||
expect(res.body.shareText).toContain('expires in 1h');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,297 @@
|
||||
/**
|
||||
* Tests for DC-086: asymmetric hysteresis on the dashboard service badge.
|
||||
*
|
||||
* - First probe always emits (no prior state).
|
||||
* - Same-status probe does NOT re-emit (dedup against repeated green).
|
||||
* - One "down" then back to "up" keeps the badge green (no flicker).
|
||||
* - Two consecutive "down" probes flip the badge to red.
|
||||
* - One "up" after a down streak flips back to green (fast recovery).
|
||||
* - History retains every raw probe even when no emit happens.
|
||||
* - getCurrentStatus returns displayed status, not raw.
|
||||
*/
|
||||
|
||||
'use strict';
|
||||
|
||||
const path = require('path');
|
||||
const fs = require('fs');
|
||||
const os = require('os');
|
||||
|
||||
// Use an isolated data dir so test history doesn't pollute the real one.
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'dashcaddy-hyst-'));
|
||||
process.env.HEALTH_DATA_DIR = tmpDir;
|
||||
process.env.HEALTH_CONFIG_FILE = path.join(tmpDir, 'health-config.json');
|
||||
process.env.HEALTH_HISTORY_FILE = path.join(tmpDir, 'health-history.json');
|
||||
|
||||
// Module exports a singleton instance, not a class — see module.exports in
|
||||
// src/monitoring/health-checker.js. The test creates fresh state by replacing
|
||||
// the relevant maps on the singleton in beforeEach.
|
||||
const healthCheckerSingleton = require('../src/monitoring/health-checker');
|
||||
const originalDownThreshold = process.env.HEALTH_DOWN_THRESHOLD;
|
||||
const originalUpThreshold = process.env.HEALTH_UP_THRESHOLD;
|
||||
|
||||
function restoreEnv(name, value) {
|
||||
if (value === undefined) delete process.env[name];
|
||||
else process.env[name] = value;
|
||||
}
|
||||
|
||||
function makeUp(serviceId = 'svc1') {
|
||||
return {
|
||||
serviceId,
|
||||
timestamp: new Date().toISOString(),
|
||||
status: 'up',
|
||||
responseTime: 50,
|
||||
statusCode: 200,
|
||||
message: 'Service is healthy',
|
||||
details: { headers: {}, bodyLength: 12 }
|
||||
};
|
||||
}
|
||||
|
||||
function makeDown(serviceId = 'svc1') {
|
||||
return {
|
||||
serviceId,
|
||||
timestamp: new Date().toISOString(),
|
||||
status: 'down',
|
||||
responseTime: 50,
|
||||
statusCode: 500,
|
||||
message: 'fail',
|
||||
details: { headers: {}, bodyLength: 0 }
|
||||
};
|
||||
}
|
||||
|
||||
describe('DC-086: hysteresis on the dashboard badge', () => {
|
||||
let hc;
|
||||
let emitSpy;
|
||||
|
||||
beforeEach(() => {
|
||||
// Reset the singleton's per-test state so each case starts clean.
|
||||
healthCheckerSingleton.displayedStatus = new Map();
|
||||
healthCheckerSingleton.consecutiveSinceChange = new Map();
|
||||
healthCheckerSingleton.currentStatus = new Map();
|
||||
healthCheckerSingleton.history = {};
|
||||
healthCheckerSingleton.removeAllListeners('status-check');
|
||||
emitSpy = jest.fn();
|
||||
healthCheckerSingleton.on('status-check', emitSpy);
|
||||
hc = healthCheckerSingleton;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
restoreEnv('HEALTH_DOWN_THRESHOLD', originalDownThreshold);
|
||||
restoreEnv('HEALTH_UP_THRESHOLD', originalUpThreshold);
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
fs.rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
test('first probe (no prior state) emits', () => {
|
||||
hc.recordStatus('svc1', makeUp());
|
||||
expect(emitSpy).toHaveBeenCalledTimes(1);
|
||||
expect(emitSpy.mock.calls[0][0].status).toBe('up');
|
||||
});
|
||||
|
||||
test('second probe with same status does NOT re-emit', () => {
|
||||
hc.recordStatus('svc1', makeUp());
|
||||
hc.recordStatus('svc1', makeUp());
|
||||
expect(emitSpy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
test('one "down" then "up" keeps the badge green (the flicker bug)', () => {
|
||||
hc.recordStatus('svc1', makeUp()); // baseline: green, emit 1
|
||||
hc.recordStatus('svc1', makeDown()); // one blip — keep green, no emit
|
||||
hc.recordStatus('svc1', makeUp()); // recovered — still green, no emit
|
||||
expect(emitSpy).toHaveBeenCalledTimes(1);
|
||||
expect(hc.displayedStatus.get('svc1').status).toBe('up');
|
||||
});
|
||||
|
||||
test('up, down, up, down, down resets the first streak before flipping', () => {
|
||||
hc.recordStatus('svc1', makeUp());
|
||||
hc.recordStatus('svc1', makeDown());
|
||||
hc.recordStatus('svc1', makeUp());
|
||||
hc.recordStatus('svc1', makeDown());
|
||||
expect(hc.displayedStatus.get('svc1').status).toBe('up');
|
||||
expect(emitSpy).toHaveBeenCalledTimes(1);
|
||||
|
||||
hc.recordStatus('svc1', makeDown());
|
||||
expect(hc.displayedStatus.get('svc1').status).toBe('down');
|
||||
expect(emitSpy).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
test('two consecutive "down" probes flip the badge to red', () => {
|
||||
hc.recordStatus('svc1', makeUp()); // baseline: green
|
||||
hc.recordStatus('svc1', makeDown()); // blip #1 — keep green (counter=1)
|
||||
hc.recordStatus('svc1', makeDown()); // blip #2 — flip red (counter=2 >= DOWN_THRESHOLD)
|
||||
expect(emitSpy).toHaveBeenCalledTimes(2);
|
||||
expect(emitSpy.mock.calls[1][0].status).toBe('down');
|
||||
expect(hc.displayedStatus.get('svc1').status).toBe('down');
|
||||
});
|
||||
|
||||
test('one "up" after a down streak flips back to green (fast recovery)', () => {
|
||||
hc.recordStatus('svc1', makeUp());
|
||||
hc.recordStatus('svc1', makeDown());
|
||||
hc.recordStatus('svc1', makeDown()); // now red
|
||||
expect(hc.displayedStatus.get('svc1').status).toBe('down');
|
||||
|
||||
hc.recordStatus('svc1', makeUp()); // first green — flip back
|
||||
expect(emitSpy).toHaveBeenCalledTimes(3);
|
||||
expect(emitSpy.mock.calls[2][0].status).toBe('up');
|
||||
expect(hc.displayedStatus.get('svc1').status).toBe('up');
|
||||
});
|
||||
|
||||
test('history retains every raw probe even when no emit happens', () => {
|
||||
hc.recordStatus('svc1', makeUp());
|
||||
hc.recordStatus('svc1', makeDown()); // blip, no emit
|
||||
hc.recordStatus('svc1', makeUp()); // recovery, no emit
|
||||
expect(hc.history['svc1'].length).toBe(3);
|
||||
expect(hc.history['svc1'][0].status).toBe('up');
|
||||
expect(hc.history['svc1'][1].status).toBe('down');
|
||||
expect(hc.history['svc1'][2].status).toBe('up');
|
||||
});
|
||||
|
||||
test('getCurrentStatus returns the displayed status, not the raw probe', () => {
|
||||
const displayedUp = makeUp();
|
||||
displayedUp.timestamp = '2026-08-22T09:59:00.000Z';
|
||||
displayedUp.statusCode = 200;
|
||||
displayedUp.message = 'healthy';
|
||||
displayedUp.details = { source: 'accepted-up' };
|
||||
hc.recordStatus('svc1', displayedUp);
|
||||
const latestRaw = makeDown();
|
||||
latestRaw.timestamp = '2026-08-22T10:00:00.000Z';
|
||||
latestRaw.responseTime = 987;
|
||||
latestRaw.statusCode = 500;
|
||||
latestRaw.message = 'failed probe';
|
||||
latestRaw.error = 'upstream failure';
|
||||
latestRaw.details = { source: 'suppressed-down' };
|
||||
hc.recordStatus('svc1', latestRaw); // raw=down, displayed=up
|
||||
const out = hc.getCurrentStatus();
|
||||
expect(out['svc1'].status).toBe('up'); // shown to API consumers
|
||||
expect(out['svc1'].timestamp).toBe(displayedUp.timestamp);
|
||||
expect(out['svc1'].statusCode).toBe(200);
|
||||
expect(out['svc1'].message).toBe('healthy');
|
||||
expect(out['svc1'].error).toBeUndefined();
|
||||
expect(out['svc1'].details).toEqual({ source: 'accepted-up' });
|
||||
expect(hc.currentStatus.get('svc1')).toBe(latestRaw);
|
||||
});
|
||||
|
||||
test('a long steady-green run produces exactly ONE emit (no per-probe spam)', () => {
|
||||
for (let i = 0; i < 50; i++) hc.recordStatus('svc1', makeUp());
|
||||
expect(emitSpy).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
test('a long steady-green-then-steady-red transition: 1 emit (up), 1 emit (red)', () => {
|
||||
for (let i = 0; i < 10; i++) hc.recordStatus('svc1', makeUp());
|
||||
expect(emitSpy).toHaveBeenCalledTimes(1);
|
||||
hc.recordStatus('svc1', makeDown());
|
||||
hc.recordStatus('svc1', makeDown()); // flips to red
|
||||
expect(emitSpy).toHaveBeenCalledTimes(2);
|
||||
for (let i = 0; i < 10; i++) hc.recordStatus('svc1', makeDown());
|
||||
expect(emitSpy).toHaveBeenCalledTimes(2); // no further broadcasts
|
||||
});
|
||||
|
||||
test('DOWN_THRESHOLD env var is honored', () => {
|
||||
process.env.HEALTH_DOWN_THRESHOLD = '3';
|
||||
jest.resetModules();
|
||||
const HC2Module = require('../src/monitoring/health-checker');
|
||||
// Module is a singleton with DOWN_THRESHOLD captured at module load —
|
||||
// resetModules gives us a fresh module-level instance with the new env.
|
||||
const hc2 = HC2Module;
|
||||
hc2.displayedStatus = new Map();
|
||||
hc2.consecutiveSinceChange = new Map();
|
||||
hc2.currentStatus = new Map();
|
||||
hc2.history = {};
|
||||
hc2.removeAllListeners('status-check');
|
||||
const spy = jest.fn();
|
||||
hc2.on('status-check', spy);
|
||||
hc2.recordStatus('svc1', makeUp());
|
||||
hc2.recordStatus('svc1', makeDown()); // 1
|
||||
hc2.recordStatus('svc1', makeDown()); // 2 — still green (need 3)
|
||||
expect(spy).toHaveBeenCalledTimes(1);
|
||||
expect(hc2.displayedStatus.get('svc1').status).toBe('up');
|
||||
hc2.recordStatus('svc1', makeDown()); // 3 — flip
|
||||
expect(spy).toHaveBeenCalledTimes(2);
|
||||
expect(hc2.displayedStatus.get('svc1').status).toBe('down');
|
||||
});
|
||||
|
||||
test.each(['not-a-number', '0', '-2', '1.5'])('malformed DOWN_THRESHOLD %s falls back to 2', value => {
|
||||
process.env.HEALTH_DOWN_THRESHOLD = value;
|
||||
jest.resetModules();
|
||||
const hc2 = require('../src/monitoring/health-checker');
|
||||
hc2.displayedStatus = new Map();
|
||||
hc2.consecutiveSinceChange = new Map();
|
||||
hc2.currentStatus = new Map();
|
||||
hc2.history = {};
|
||||
hc2.removeAllListeners('status-check');
|
||||
const spy = jest.fn();
|
||||
hc2.on('status-check', spy);
|
||||
hc2.recordStatus('svc1', makeUp());
|
||||
hc2.recordStatus('svc1', makeDown());
|
||||
expect(spy).toHaveBeenCalledTimes(1);
|
||||
hc2.recordStatus('svc1', makeDown());
|
||||
expect(spy).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
test('UP_THRESHOLD env var greater than 1 is honored', () => {
|
||||
process.env.HEALTH_UP_THRESHOLD = '2';
|
||||
jest.resetModules();
|
||||
const hc2 = require('../src/monitoring/health-checker');
|
||||
hc2.displayedStatus = new Map();
|
||||
hc2.consecutiveSinceChange = new Map();
|
||||
hc2.currentStatus = new Map();
|
||||
hc2.history = {};
|
||||
hc2.removeAllListeners('status-check');
|
||||
const spy = jest.fn();
|
||||
hc2.on('status-check', spy);
|
||||
hc2.recordStatus('svc1', makeDown());
|
||||
hc2.recordStatus('svc1', makeUp());
|
||||
expect(spy).toHaveBeenCalledTimes(1);
|
||||
expect(hc2.displayedStatus.get('svc1').status).toBe('down');
|
||||
hc2.recordStatus('svc1', makeUp());
|
||||
expect(spy).toHaveBeenCalledTimes(2);
|
||||
expect(hc2.displayedStatus.get('svc1').status).toBe('up');
|
||||
});
|
||||
|
||||
test.each(['not-a-number', '0', '-2', '1.5'])('malformed UP_THRESHOLD %s falls back to 1', value => {
|
||||
process.env.HEALTH_UP_THRESHOLD = value;
|
||||
jest.resetModules();
|
||||
const hc2 = require('../src/monitoring/health-checker');
|
||||
hc2.displayedStatus = new Map();
|
||||
hc2.consecutiveSinceChange = new Map();
|
||||
hc2.currentStatus = new Map();
|
||||
hc2.history = {};
|
||||
hc2.removeAllListeners('status-check');
|
||||
const spy = jest.fn();
|
||||
hc2.on('status-check', spy);
|
||||
hc2.recordStatus('svc1', makeDown());
|
||||
hc2.recordStatus('svc1', makeUp());
|
||||
expect(spy).toHaveBeenCalledTimes(2);
|
||||
expect(hc2.displayedStatus.get('svc1').status).toBe('up');
|
||||
});
|
||||
|
||||
test('removeService clears hysteresis state before the same ID is re-added', () => {
|
||||
hc.config.services.svc1 = { name: 'Service 1' };
|
||||
hc.recordStatus('svc1', makeUp());
|
||||
hc.recordStatus('svc1', makeDown());
|
||||
expect(hc.displayedStatus.has('svc1')).toBe(true);
|
||||
expect(hc.consecutiveSinceChange.get('svc1')).toBe(1);
|
||||
hc.consecutiveFailures.set('svc1', 3);
|
||||
const timer = setTimeout(() => {}, 60_000);
|
||||
hc.serviceTimers.set('svc1', timer);
|
||||
|
||||
hc.saveConfig = jest.fn();
|
||||
hc.removeService('svc1');
|
||||
|
||||
expect(hc.displayedStatus.has('svc1')).toBe(false);
|
||||
expect(hc.consecutiveSinceChange.has('svc1')).toBe(false);
|
||||
expect(hc.currentStatus.has('svc1')).toBe(false);
|
||||
expect(hc.consecutiveFailures.has('svc1')).toBe(false);
|
||||
expect(hc.serviceTimers.has('svc1')).toBe(false);
|
||||
|
||||
hc.config.services.svc1 = { name: 'Service 1 re-added' };
|
||||
const emitSpyAfterReAdd = jest.fn();
|
||||
hc.on('status-check', emitSpyAfterReAdd);
|
||||
hc.recordStatus('svc1', makeDown());
|
||||
|
||||
expect(emitSpyAfterReAdd).toHaveBeenCalledTimes(1);
|
||||
expect(hc.displayedStatus.get('svc1').status).toBe('down');
|
||||
expect(hc.consecutiveSinceChange.has('svc1')).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -203,6 +203,48 @@ describe('HealthChecker', () => {
|
||||
expect(result.error).toBe('ECONNREFUSED');
|
||||
});
|
||||
|
||||
it('opens and resolves an outage incident across real checkService transitions', async () => {
|
||||
healthChecker._doRequest = jest.fn()
|
||||
.mockResolvedValueOnce({ healthy: true, statusCode: 200, message: 'ok', details: {} })
|
||||
.mockResolvedValueOnce({ healthy: false, statusCode: 500, message: 'down', details: {} })
|
||||
.mockResolvedValueOnce({ healthy: true, statusCode: 200, message: 'ok', details: {} });
|
||||
|
||||
const config = { url: 'http://test.local' };
|
||||
await healthChecker.checkService('svc1', config);
|
||||
await healthChecker.checkService('svc1', config);
|
||||
|
||||
expect(healthChecker.incidents).toHaveLength(1);
|
||||
expect(healthChecker.incidents[0]).toMatchObject({
|
||||
serviceId: 'svc1',
|
||||
type: 'outage',
|
||||
status: 'open'
|
||||
});
|
||||
|
||||
await healthChecker.checkService('svc1', config);
|
||||
expect(healthChecker.incidents[0].status).toBe('resolved');
|
||||
expect(healthChecker.incidents[0].resolvedAt).toBeDefined();
|
||||
});
|
||||
|
||||
it('does not resurrect state when an in-flight probe resolves after removal', async () => {
|
||||
let resolveProbe;
|
||||
healthChecker.config.services.svc1 = { url: 'http://test.local' };
|
||||
healthChecker._doRequest = jest.fn(() => new Promise(resolve => {
|
||||
resolveProbe = resolve;
|
||||
}));
|
||||
|
||||
const pending = healthChecker.checkService('svc1', healthChecker.config.services.svc1);
|
||||
healthChecker.saveConfig = jest.fn();
|
||||
healthChecker.removeService('svc1');
|
||||
resolveProbe({ healthy: true, statusCode: 200, message: 'late', details: {} });
|
||||
await pending;
|
||||
|
||||
expect(healthChecker.currentStatus.has('svc1')).toBe(false);
|
||||
expect(healthChecker.displayedStatus.has('svc1')).toBe(false);
|
||||
expect(healthChecker.consecutiveFailures.has('svc1')).toBe(false);
|
||||
expect(healthChecker.history.svc1).toBeUndefined();
|
||||
expect(healthChecker.incidents).toEqual([]);
|
||||
});
|
||||
|
||||
it('increments consecutive failures on error', async () => {
|
||||
healthChecker._doRequest = jest.fn().mockRejectedValue(new Error('fail'));
|
||||
|
||||
|
||||
@@ -32,23 +32,6 @@ const emailSender = require('../../src/auth/providers/email-sender');
|
||||
const { ValidationError, NotFoundError, ForbiddenError, PaymentRequiredError } = require('../../src/utilities/errors');
|
||||
const { ok, successMessage } = require('../../src/utils/responses');
|
||||
|
||||
/**
|
||||
* Build the URL an invitee should click. Mirrors EmailMagicLinkProvider's
|
||||
* _resolvePublicUrl logic — kept duplicated (not extracted) because the two
|
||||
* callers have slightly different link paths and the duplication is smaller
|
||||
* than the abstraction would be.
|
||||
*/
|
||||
function _buildInviteUrl(req, siteConfig, token) {
|
||||
if (siteConfig && siteConfig.publicBaseUrl) {
|
||||
return siteConfig.publicBaseUrl.replace(/\/+$/, '') +
|
||||
'/api/v1/auth/invites/' + encodeURIComponent(token) + '/accept';
|
||||
}
|
||||
const proto = (req.headers && req.headers['x-forwarded-proto']) || (req.protocol || 'https');
|
||||
const host = (req.headers && (req.headers['x-forwarded-host'] || req.headers.host))
|
||||
|| (siteConfig && siteConfig.dashboardHost) || 'localhost:3001';
|
||||
return `${proto}://${host}/api/v1/auth/invites/${encodeURIComponent(token)}/accept`;
|
||||
}
|
||||
|
||||
function _requireAdmin(req, _res, next) {
|
||||
if (!req.user || req.user.role !== 'admin') {
|
||||
return next(new ForbiddenError('Admin role required'));
|
||||
@@ -240,11 +223,21 @@ module.exports = function({ asyncHandler, errorResponse, log, session, dataDir }
|
||||
});
|
||||
if (!issued.ok) throw new ValidationError(issued.reason, 'email');
|
||||
|
||||
let deliveredVia = 'none';
|
||||
const maskedEmail = email.replace(/(^.).+(@.*$)/, '$1***$2');
|
||||
if (sendEmail !== false) {
|
||||
// Best-effort send. If SMTP isn't configured, log to error.log (dev path).
|
||||
const acceptUrl = _buildInviteUrl(req, /* siteConfig */ req.app.locals && req.app.locals.siteConfig, issued.token);
|
||||
// Build the accept URL once — used both for the response and for email delivery.
|
||||
const baseUrl = (req.app.locals && req.app.locals.siteConfig && req.app.locals.siteConfig.publicBaseUrl
|
||||
? req.app.locals.siteConfig.publicBaseUrl.replace(/\/+$/, '')
|
||||
: ((req.headers['x-forwarded-proto'] || req.protocol || 'https') + '://' +
|
||||
(req.headers['x-forwarded-host'] || req.headers.host || 'localhost:3001')));
|
||||
const acceptUrl = baseUrl + '/api/v1/auth/invites/' + encodeURIComponent(issued.token) + '/accept';
|
||||
|
||||
// DC-085: link-first delivery. Default = no email, just hand the link back.
|
||||
// Operators opt INTO email by sending { sendEmail: true } (or the admin UI
|
||||
// checks the "Send email" checkbox). When SMTP is unconfigured AND the
|
||||
// operator did opt in, we surface the failure as `deliveredVia: 'failed'`
|
||||
// but NEVER leak the raw token into the server log — the link is already
|
||||
// in the response, so the operator has a UI-side fallback.
|
||||
let deliveredVia = 'manual';
|
||||
if (sendEmail === true) {
|
||||
const ttlHoursOut = Math.round(issued.ttlMs / (60 * 60 * 1000));
|
||||
const text = _buildEmailText({ acceptUrl, ttlHours: ttlHoursOut, role: issued.role });
|
||||
const html = _buildEmailHtml({ acceptUrl, ttlHours: ttlHoursOut, role: issued.role });
|
||||
@@ -254,33 +247,37 @@ module.exports = function({ asyncHandler, errorResponse, log, session, dataDir }
|
||||
await emailSender.sendEmail(smtpConfig, issued.email, 'You\'re invited to DashCaddy', text, html);
|
||||
deliveredVia = 'email';
|
||||
} else {
|
||||
// Dev fallback — log the raw link so operators can grab it.
|
||||
log.warn && log.warn('auth-invite-dev',
|
||||
'[DC-048-DEV-INVITE-LINK] email=' + issued.email +
|
||||
' role=' + issued.role + ' url=' + acceptUrl);
|
||||
deliveredVia = 'dev-console';
|
||||
// Operator asked for email but SMTP isn't configured. Surface the
|
||||
// failure cleanly; the link is still in the response so the
|
||||
// operator can share it manually. Do NOT log the raw URL — it
|
||||
// would duplicate what's already in the response and pollute the
|
||||
// server log on every unconfigured-install invite.
|
||||
log.warn && log.warn('auth-invite-send',
|
||||
'invite send skipped: SMTP not configured (operator opted in)',
|
||||
{ inviteId: issued.id, email: issued.email });
|
||||
deliveredVia = 'failed';
|
||||
}
|
||||
} catch (sendErr) {
|
||||
log.warn && log.warn('auth-invite-send',
|
||||
'invite send failed: ' + (sendErr.message || String(sendErr)));
|
||||
'invite send failed: ' + (sendErr.message || String(sendErr)),
|
||||
{ inviteId: issued.id });
|
||||
deliveredVia = 'failed';
|
||||
}
|
||||
} else {
|
||||
deliveredVia = 'manual';
|
||||
}
|
||||
|
||||
const maskedEmail = email.replace(/(^.).+(@.*$)/, '$1***$2');
|
||||
const ttlHoursOut = Math.round(issued.ttlMs / (60 * 60 * 1000));
|
||||
const shareText =
|
||||
'Join my DashCaddy as ' + issued.role + ' — ' + acceptUrl +
|
||||
' — expires in ' + ttlHoursOut + 'h.';
|
||||
|
||||
return ok(res, {
|
||||
id: issued.id,
|
||||
email: issued.email,
|
||||
role: issued.role,
|
||||
expiresAt: issued.expiresAt,
|
||||
// The raw token is returned ONCE so the admin UI can show/copy the
|
||||
// link. It is also embedded in the email when sendEmail !== false.
|
||||
acceptUrl: (req.app.locals && req.app.locals.siteConfig && req.app.locals.siteConfig.publicBaseUrl
|
||||
? req.app.locals.siteConfig.publicBaseUrl.replace(/\/+$/, '')
|
||||
: ((req.headers['x-forwarded-proto'] || req.protocol || 'https') + '://' +
|
||||
(req.headers['x-forwarded-host'] || req.headers.host || 'localhost:3001'))) +
|
||||
'/api/v1/auth/invites/' + encodeURIComponent(issued.token) + '/accept',
|
||||
acceptUrl,
|
||||
shareText,
|
||||
deliveredVia,
|
||||
maskedEmail,
|
||||
});
|
||||
|
||||
@@ -33,17 +33,45 @@ const MAX_CHECK_INTERVAL = parseInt(process.env.HEALTH_CHECK_MAX_INTERVAL || '30
|
||||
const MAX_ENTRIES_PER_SERVICE = parseInt(process.env.HEALTH_MAX_ENTRIES || '500', 10); // Cap to prevent disk explosion
|
||||
const HISTORY_RETENTION_DAYS = parseInt(process.env.HEALTH_HISTORY_RETENTION || '30', 10);
|
||||
|
||||
// DC-086: hysteresis thresholds for badge display.
|
||||
// The raw probe result can flap on a single transient blip (Caddy reload,
|
||||
// container CPU steal, network hiccup, mid-flight TLS handshake). Showing
|
||||
// every probe result as-is to the dashboard creates the "perpetual flicker"
|
||||
// UX. Asymmetric thresholds: going red is slow (don't false-alarm), going
|
||||
// green is fast (don't keep showing red after recovery).
|
||||
// - DOWN_THRESHOLD = N consecutive "down" probes before the badge flips to red
|
||||
// - UP_THRESHOLD = N consecutive "up" probes before the badge flips back to green
|
||||
// Single probe flips to green on purpose — false-positive-green is much less
|
||||
// painful than perpetual-red (operators notice red, ignore green).
|
||||
function readPositiveIntEnv(name, fallback) {
|
||||
const raw = process.env[name];
|
||||
if (raw === undefined || raw === '') return fallback;
|
||||
const value = Number(raw);
|
||||
return Number.isSafeInteger(value) && value >= 1 ? value : fallback;
|
||||
}
|
||||
|
||||
const DOWN_THRESHOLD = readPositiveIntEnv('HEALTH_DOWN_THRESHOLD', 2);
|
||||
const UP_THRESHOLD = readPositiveIntEnv('HEALTH_UP_THRESHOLD', 1);
|
||||
|
||||
class HealthChecker extends EventEmitter {
|
||||
constructor() {
|
||||
super();
|
||||
this.config = this.loadConfig();
|
||||
this.history = this.loadHistory();
|
||||
this.currentStatus = new Map();
|
||||
// DC-086: the status the dashboard SHOULD display (post-hysteresis).
|
||||
// Distinct from currentStatus, which is the latest raw probe result.
|
||||
this.displayedStatus = new Map();
|
||||
// DC-086: counter of consecutive healthy/unhealthy probes since the
|
||||
// last displayed-status change. Reset to 0 whenever displayed status flips.
|
||||
this.consecutiveSinceChange = new Map();
|
||||
this.incidents = [];
|
||||
this.checking = false;
|
||||
this.checkInterval = null;
|
||||
this.consecutiveFailures = new Map(); // serviceId -> failure count
|
||||
this.serviceTimers = new Map(); // serviceId -> timer for per-service backoff
|
||||
// Invalidate probe completions that race with removal/reconfiguration.
|
||||
this.serviceGenerations = new Map(); // serviceId -> configuration generation
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -116,6 +144,7 @@ class HealthChecker extends EventEmitter {
|
||||
*/
|
||||
async checkService(serviceId, config) {
|
||||
const startTime = Date.now();
|
||||
const generation = this.serviceGenerations.get(serviceId) || 0;
|
||||
|
||||
try {
|
||||
const result = await this.performHealthCheck(config);
|
||||
@@ -131,6 +160,10 @@ class HealthChecker extends EventEmitter {
|
||||
details: result.details
|
||||
};
|
||||
|
||||
if ((this.serviceGenerations.get(serviceId) || 0) !== generation) {
|
||||
return status;
|
||||
}
|
||||
|
||||
// Track consecutive failures for exponential backoff
|
||||
if (result.healthy) {
|
||||
this.consecutiveFailures.delete(serviceId);
|
||||
@@ -138,8 +171,9 @@ class HealthChecker extends EventEmitter {
|
||||
this.consecutiveFailures.set(serviceId, (this.consecutiveFailures.get(serviceId) || 0) + 1);
|
||||
}
|
||||
|
||||
const previousStatus = this.currentStatus.get(serviceId);
|
||||
this.recordStatus(serviceId, status);
|
||||
this.checkForIncidents(serviceId, status, config);
|
||||
this.checkForIncidents(serviceId, status, config, previousStatus);
|
||||
|
||||
return status;
|
||||
} catch (error) {
|
||||
@@ -156,8 +190,13 @@ class HealthChecker extends EventEmitter {
|
||||
error: error.message
|
||||
};
|
||||
|
||||
if ((this.serviceGenerations.get(serviceId) || 0) !== generation) {
|
||||
return status;
|
||||
}
|
||||
|
||||
const previousStatus = this.currentStatus.get(serviceId);
|
||||
this.recordStatus(serviceId, status);
|
||||
this.checkForIncidents(serviceId, status, config);
|
||||
this.checkForIncidents(serviceId, status, config, previousStatus);
|
||||
|
||||
return status;
|
||||
}
|
||||
@@ -273,27 +312,109 @@ class HealthChecker extends EventEmitter {
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute the displayed status for a service given the latest raw probe
|
||||
* result. Applies asymmetric hysteresis:
|
||||
* - Going DOWN: requires DOWN_THRESHOLD (default 2) consecutive "down"
|
||||
* probes since the last display-state change. A single blip keeps the
|
||||
* badge green.
|
||||
* - Going UP: requires UP_THRESHOLD (default 1) consecutive "up" probes.
|
||||
* Any single "up" after a down streak flips back to green so the badge
|
||||
* doesn't linger red after the service has recovered.
|
||||
*
|
||||
* Returns the displayed status object (same shape as the raw status) so
|
||||
* recordStatus can use it both for the displayed map and as the broadcast
|
||||
* payload when the displayed status actually changes.
|
||||
*/
|
||||
_computeDisplayedStatus(serviceId, rawStatus) {
|
||||
const currentDisplayed = this.displayedStatus.get(serviceId);
|
||||
const previousStatus = currentDisplayed ? currentDisplayed.status : null;
|
||||
|
||||
// If no prior state, accept the raw probe as-is (first-check bootstrap).
|
||||
if (!previousStatus) {
|
||||
return rawStatus;
|
||||
}
|
||||
|
||||
// Probe agrees with current displayed → no change, reset the counter so
|
||||
// a brief blip doesn't accumulate against the displayed state.
|
||||
if (rawStatus.status === previousStatus) {
|
||||
this.consecutiveSinceChange.set(serviceId, 0);
|
||||
return rawStatus;
|
||||
}
|
||||
|
||||
// Probe disagrees with displayed. Bump the streak counter — this counts
|
||||
// CONSECUTIVE probes that disagree with what's shown, regardless of
|
||||
// whether the raw value itself changed between probes. That's what
|
||||
// makes "down, down" flip after threshold but "down, up, down" not flip.
|
||||
const prev = this.consecutiveSinceChange.get(serviceId) || 0;
|
||||
const next = prev + 1;
|
||||
|
||||
if (rawStatus.status === 'down') {
|
||||
// Going DOWN: need DOWN_THRESHOLD consecutive probes that disagree
|
||||
// with the displayed "up" state.
|
||||
if (previousStatus === 'up' && next < DOWN_THRESHOLD) {
|
||||
this.consecutiveSinceChange.set(serviceId, next);
|
||||
// Keep the last internally-consistent displayed snapshot. Mixing the
|
||||
// raw failure metadata with status="up" would expose contradictory
|
||||
// API data (for example statusCode=500 on an "up" service).
|
||||
return currentDisplayed;
|
||||
}
|
||||
// Threshold met (or already down) — flip to red.
|
||||
this.consecutiveSinceChange.set(serviceId, 0);
|
||||
return rawStatus;
|
||||
}
|
||||
|
||||
// rawStatus.status === 'up' (must be — the equal-to-displayed case above
|
||||
// already returned). Going UP after a down streak: need UP_THRESHOLD.
|
||||
if (previousStatus === 'down' && next < UP_THRESHOLD) {
|
||||
this.consecutiveSinceChange.set(serviceId, next);
|
||||
return currentDisplayed;
|
||||
}
|
||||
this.consecutiveSinceChange.set(serviceId, 0);
|
||||
return rawStatus;
|
||||
}
|
||||
|
||||
/**
|
||||
* Record service status
|
||||
*
|
||||
* DC-086: history + consecutiveFailures are updated for EVERY probe
|
||||
* (operators want full probe history for postmortems). The dashboard's
|
||||
* `status-check` event is only emitted when the DISPLAYED status changes,
|
||||
* so the badge stops re-rendering on every probe.
|
||||
*/
|
||||
recordStatus(serviceId, status) {
|
||||
// Update current status
|
||||
// Update current (raw) status — used by checkForIncidents and history.
|
||||
this.currentStatus.set(serviceId, status);
|
||||
|
||||
// Add to history
|
||||
// Add raw probe to history (full fidelity — operators rely on this).
|
||||
if (!this.history[serviceId]) {
|
||||
this.history[serviceId] = [];
|
||||
}
|
||||
|
||||
this.history[serviceId].push(status);
|
||||
|
||||
|
||||
// Cap entries to prevent unbounded growth (disk explosion fix)
|
||||
if (this.history[serviceId].length > MAX_ENTRIES_PER_SERVICE) {
|
||||
this.history[serviceId] = this.history[serviceId].slice(-MAX_ENTRIES_PER_SERVICE);
|
||||
}
|
||||
|
||||
// Emit status event
|
||||
this.emit('status-check', status);
|
||||
// Compute the post-hysteresis displayed status; only emit when it changes.
|
||||
// _computeDisplayedStatus compares the raw probe against the DISPLAYED
|
||||
// status (not the previous raw status), so the "consecutive since
|
||||
// change" counter doesn't depend on the order of writes here.
|
||||
const displayed = this._computeDisplayedStatus(serviceId, status);
|
||||
const previousDisplayed = this.displayedStatus.get(serviceId);
|
||||
const displayChanged =
|
||||
!previousDisplayed || previousDisplayed.status !== displayed.status;
|
||||
|
||||
this.displayedStatus.set(serviceId, displayed);
|
||||
|
||||
if (displayChanged) {
|
||||
// Emit with the displayed status so the dashboard renders the same
|
||||
// state the hysteresis just decided. The raw probe result is still
|
||||
// in `history` and `currentStatus` for anyone who wants it.
|
||||
this.emit('status-check', displayed);
|
||||
}
|
||||
|
||||
// Save history periodically
|
||||
if (Math.random() < 0.05) { // 5% chance (every ~20 checks)
|
||||
@@ -304,8 +425,7 @@ class HealthChecker extends EventEmitter {
|
||||
/**
|
||||
* Check for incidents (downtime, slow response, etc.)
|
||||
*/
|
||||
checkForIncidents(serviceId, status, config) {
|
||||
const previous = this.currentStatus.get(serviceId);
|
||||
checkForIncidents(serviceId, status, config, previous = this.currentStatus.get(serviceId)) {
|
||||
|
||||
// Check for status change (up -> down or down -> up)
|
||||
if (previous && previous.status !== status.status) {
|
||||
@@ -445,19 +565,29 @@ class HealthChecker extends EventEmitter {
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current status for all services
|
||||
* Get current status for all services.
|
||||
*
|
||||
* DC-086: returns the DISPLAYED status (post-hysteresis), not the latest
|
||||
* raw probe. A page reload should show the same badge state the live
|
||||
* SSE stream is currently showing — otherwise an operator who reloads
|
||||
* the page after a single blip sees red even though the hysteresis kept
|
||||
* the badge green for them.
|
||||
*/
|
||||
getCurrentStatus() {
|
||||
const result = {};
|
||||
|
||||
for (const [serviceId, status] of this.currentStatus.entries()) {
|
||||
|
||||
for (const [serviceId, rawStatus] of this.currentStatus.entries()) {
|
||||
const config = this.config.services[serviceId];
|
||||
const uptime24h = this.calculateUptime(serviceId, 24);
|
||||
const uptime7d = this.calculateUptime(serviceId, 168);
|
||||
const avgResponseTime = this.calculateAverageResponseTime(serviceId, 24);
|
||||
|
||||
|
||||
// Prefer the displayed status if we've already computed one; fall back
|
||||
// to the raw probe on the very first call (before recordStatus has run).
|
||||
const displayed = this.displayedStatus.get(serviceId) || rawStatus;
|
||||
|
||||
result[serviceId] = {
|
||||
...status,
|
||||
...displayed,
|
||||
name: config?.name || serviceId,
|
||||
uptime: {
|
||||
'24h': uptime24h,
|
||||
@@ -467,7 +597,7 @@ class HealthChecker extends EventEmitter {
|
||||
sla: config?.sla
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
@@ -530,6 +660,7 @@ class HealthChecker extends EventEmitter {
|
||||
this.config.services = {};
|
||||
}
|
||||
|
||||
this.serviceGenerations.set(serviceId, (this.serviceGenerations.get(serviceId) || 0) + 1);
|
||||
this.config.services[serviceId] = {
|
||||
enabled: config.enabled !== false,
|
||||
name: config.name || serviceId,
|
||||
@@ -552,12 +683,19 @@ class HealthChecker extends EventEmitter {
|
||||
* Remove service configuration
|
||||
*/
|
||||
removeService(serviceId) {
|
||||
this.serviceGenerations.set(serviceId, (this.serviceGenerations.get(serviceId) || 0) + 1);
|
||||
if (this.config.services) {
|
||||
delete this.config.services[serviceId];
|
||||
this.saveConfig();
|
||||
}
|
||||
|
||||
this.currentStatus.delete(serviceId);
|
||||
this.displayedStatus.delete(serviceId);
|
||||
this.consecutiveSinceChange.delete(serviceId);
|
||||
this.consecutiveFailures.delete(serviceId);
|
||||
const timer = this.serviceTimers.get(serviceId);
|
||||
if (timer) clearTimeout(timer);
|
||||
this.serviceTimers.delete(serviceId);
|
||||
delete this.history[serviceId];
|
||||
}
|
||||
|
||||
|
||||
+65
-4
@@ -216,8 +216,8 @@
|
||||
_el('input', { name: 'ttlHours', type: 'number', min: '1', max: '168', value: '24', style: 'padding:6px;width:80px' }),
|
||||
));
|
||||
form.appendChild(_el('label', { style: 'display:flex;gap:4px;align-items:center;font-size:0.85rem' },
|
||||
_el('input', { name: 'sendEmail', type: 'checkbox', checked: true }),
|
||||
_el('span', { text: 'Send email' }),
|
||||
_el('input', { name: 'sendEmail', type: 'checkbox', checked: false }),
|
||||
_el('span', { text: 'Also send via email (optional)' }),
|
||||
));
|
||||
form.appendChild(_el('button', { type: 'submit', class: 'btn-sm', style: 'padding:6px 12px', text: 'Issue invite' }));
|
||||
container.appendChild(form);
|
||||
@@ -297,16 +297,77 @@
|
||||
},
|
||||
});
|
||||
banner.appendChild(copyBtn);
|
||||
if (invite.deliveredVia === 'dev-console') {
|
||||
|
||||
// DC-085: pre-formatted message for one-tap paste into iMessage / WhatsApp /
|
||||
// Telegram / SMS / Signal / Discord / paste-into-email. The operator can
|
||||
// copy this as a sentence instead of dealing with the raw URL.
|
||||
if (invite.shareText) {
|
||||
const shareBlock = _el('div', { style: 'margin-top:12px' });
|
||||
shareBlock.appendChild(_el('div', {
|
||||
style: 'font-size:0.8rem;color:#86efac;margin-bottom:4px',
|
||||
text: 'Share this message:',
|
||||
}));
|
||||
shareBlock.appendChild(_el('div', {
|
||||
style: 'padding:8px;background:#000;border-radius:4px;color:#d1fae5;white-space:pre-wrap',
|
||||
text: invite.shareText,
|
||||
}));
|
||||
const shareActions = _el('div', { style: 'margin-top:6px;display:flex;gap:6px;flex-wrap:wrap' });
|
||||
const copyTextBtn = _el('button', {
|
||||
class: 'btn-sm', style: 'padding:4px 10px',
|
||||
text: 'Copy message',
|
||||
onclick: async () => {
|
||||
try {
|
||||
await navigator.clipboard.writeText(invite.shareText);
|
||||
copyTextBtn.textContent = 'Copied!';
|
||||
setTimeout(() => { copyTextBtn.textContent = 'Copy message'; }, 2000);
|
||||
} catch (e) {
|
||||
window.errorHandler && window.errorHandler.show('Clipboard blocked: select the text manually.');
|
||||
}
|
||||
},
|
||||
});
|
||||
shareActions.appendChild(copyTextBtn);
|
||||
// Native share sheet on mobile / supported browsers. Falls back silently
|
||||
// (the copy buttons cover the same intent).
|
||||
if (typeof navigator !== 'undefined' && typeof navigator.share === 'function') {
|
||||
const nativeShareBtn = _el('button', {
|
||||
class: 'btn-sm', style: 'padding:4px 10px',
|
||||
text: 'Share via…',
|
||||
onclick: async () => {
|
||||
try {
|
||||
await navigator.share({
|
||||
title: 'DashCaddy invite',
|
||||
text: invite.shareText,
|
||||
url: invite.acceptUrl,
|
||||
});
|
||||
} catch (e) {
|
||||
// User-cancelled throws AbortError — that's fine, just stay quiet.
|
||||
if (e && e.name && e.name !== 'AbortError') {
|
||||
window.errorHandler && window.errorHandler.show('Share failed: ' + e.message);
|
||||
}
|
||||
}
|
||||
},
|
||||
});
|
||||
shareActions.appendChild(nativeShareBtn);
|
||||
}
|
||||
shareBlock.appendChild(shareActions);
|
||||
banner.appendChild(shareBlock);
|
||||
}
|
||||
|
||||
if (invite.deliveredVia === 'failed') {
|
||||
banner.appendChild(_el('p', {
|
||||
style: 'margin-top:8px;color:#fbbf24;font-size:0.8rem',
|
||||
text: 'SMTP not configured — the invite was logged to the server console (search for [DC-048-DEV-INVITE-LINK]).',
|
||||
text: 'Email could not be sent (SMTP not configured). Share the link above instead — it works the same way.',
|
||||
}));
|
||||
} else if (invite.deliveredVia === 'email') {
|
||||
banner.appendChild(_el('p', {
|
||||
style: 'margin-top:8px;color:#86efac;font-size:0.8rem',
|
||||
text: 'Email sent to ' + invite.email + '.',
|
||||
}));
|
||||
} else if (invite.deliveredVia === 'manual') {
|
||||
banner.appendChild(_el('p', {
|
||||
style: 'margin-top:8px;color:#86efac;font-size:0.8rem',
|
||||
text: 'Share the link above via text, chat, or any messenger.',
|
||||
}));
|
||||
}
|
||||
parent.appendChild(banner);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user