sami7777/dashcaddy
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

[grade=B urn:ump:35dpnggtxsqrxvlqm2d2buqkbnggw3acflbccnkqpqyiiijcxx3q] fix(DC-170): MCP truth batch — 6 dead tools fixed, full-catalog drift gate

Browse Source 
Audit P0-5 + deploy_app truth (batch 5 of the 2026-10-27 sweep, AUDIT-2026-10-27.md committed here):

ROUTE ADDITIONS:
- routes/containers.js GET / — container list (dockerode listContainers);
  the router had NO list route, so dashcaddy_list_containers 404'd always
- routes/dns.js GET /records — record list via active provider with a
  list-records capability guard (501 + guidance for push-only providers
  like RFC2136); dashcaddy_list_dns called GET /dns, never existed

HANDLER FIXES (src/mcp/mcp-core.js):
- get_service resolves from GET /services by id/name (single-service GET
  never existed) with a known-ids 404
- check_health + diagnose -> /health/service/:id and /health/services
  (both previously hit nonexistent routes)
- container_action remove -> DELETE /containers/:id (was /:id/remove)
- system_health/system_metrics via apiCall error envelope (metrics text
  capped at 64KB)
- search_catalog applies category filter client-side (/catalog/search
  has no category param — constraint was silently dropped)
- generate_caddyfile defaults authService=status when auth:true (auth
  flag alone silently produced UNGATED Caddyfiles)
- deploy_app honest re-contract: registers service + generates
  Caddyfile, does NOT claim image pull/container creation/deployed:true;
  base domain read from /config (hardcoded owner .sami TLD removed);
  caddy-code failure now surfaces as caddyfileError + FAILED message
- list_fleet description gains Premium hint

VERIFIER (audit #40): EXPECTED table 12 -> 34 rows covering EVERY tool's
(method,path); NEW Layer-3 handler-derived test drives the real
createToolHandler for all 28 tools and strictly resolves every captured
call mount-aware (a vacuous first draft of this gate was caught and
fixed pre-grade); extractRoutes regex extended to apiRouter.get inline
decls; Bearer credential-passthrough test added (DC-162 coverage gap)
NEW TESTS: dns-records.routes.test.js (5 route-level cases incl. 501
capability behavior); deploy_app partial-failure + no-.sami tests

HOST RELIABILITY (same sweep, evidenced here): jest maxWorkers 50% —
default 5-worker runs repeatedly OOM-killed this host's gateway
(dmesg 2026-10-29 00:58:40; swap was 4G/4G full). Gateway protected
separately via systemd OOMScoreAdjust=-500 drop-in + /swapfile2 (host
ops, graded B urn:ump:35dpnggtxsqrxvlqm2d2buqkbnggw3acflbccnkqpqyiiijcxx3q).
DC-171 DNS provider-parity ticket (Sami, plug-in-directly) appended to
backlog.

Full suite: 173/173 suites, 3740/3740 tests. Judge B r1
(urn:ump:jsobkg2t2omzvpinel2fufk7sspoy3vubsrth3e6lat3ygolvffq) blockers
all fixed: dns.js 501 arg order (the file aliases error() message-first
— the sweep's 'reversed-arg' finding was a false positive for its 7
sites; lesson #14), deploy_app partial-failure honesty, handler-derived
verifier.
This commit is contained in:
Hermes committed 2026-10-29 01:30:42 -07:00
1 parent 94ee76e9de
commit f4f2c26de3
9 files changed
+529 -40
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

dashcaddy-api/__tests__/mcp/mcp-core-dc166-integration.test.js
+125 -9
View File
@@ -1,12 +1,13 @@
/**
* DC-166 r2 (judge round 1 blocking demand): resolve the mounted
* method+path for the DC-166 MCP tool calls — no substring greps.
* DC-166 r2: resolve the mounted method+path for MCP tool calls — no
* substring greps.
*
* SCOPE (merge-judge r1 blocker, urn:ump:2jdpii43xlasarlxl2jug64xab465xnzzbdt3bnd4fbr7y23l26q):
* this verifier covers the 11 NEW DC-166 tools (12 requests — security-summary
* fans out) — NOT the whole 28-tool catalog. The 17 pre-existing tools are
* resolved against declared routes by the follow-up MCP truth batch
* (audit 2026-10-27 #40: extend to every tool + fail on any unresolved path).
* SCOPE (DC-170 r2, audit #40 — FULL CATALOG): the EXPECTED table now covers
* EVERY tool's resolved (method, path) — the 11 DC-166 additions AND the 17
* pre-existing tools (33 rows; check_health/container_action/deploy_app/
* search_catalog contribute their distinct paths). The handler-derived test
* additionally drives the REAL createToolHandler and resolves every captured
* call, so handler drift fails even when this table goes stale.
*
* Layer 1 (composition): boots the routers whose dependency surface can be
* satisfied in-process (workflows — the one the judge flagged) and
@@ -17,6 +18,9 @@
* against the route declarations extracted from the owning route file
* (router.<method>('<path>') with the mount prefix from src/app.js) —
* an exact normalized match, so a double prefix or wrong method fails.
* Layer 3 (DC-170 handler-derived): every TOOLS entry is invoked with sample
* args against a capture fetch; every emitted (method, path) must resolve
* to a declared route under the correct mount.
*/
'use strict';
@@ -43,6 +47,31 @@ const EXPECTED = [
{ method: 'POST', path: '/api/v1/recipes/deploy', mount: '/recipes', file: 'routes/recipes/deploy.js', rel: '/deploy' },
{ method: 'POST', path: '/api/v1/workflows/disk-space-alert/run', mount: '/workflows', file: 'routes/workflows.js', rel: '/:workflowId/run' },
{ method: 'GET', path: '/api/v1/tailscale/status', mount: '/tailscale', file: 'routes/tailscale.js', rel: '/status' },
// ── DC-170: full-catalog coverage (audit #40 / merge-judge r2 commitment) —
// every remaining tool's resolved (method, path). Sample segments stand in
// for :params. ──
{ method: 'GET', path: '/api/v1/services', mount: 'BARE', file: 'routes/services.js', rel: '/services' },
{ method: 'GET', path: '/api/v1/health/service/plex', mount: 'BARE', file: 'routes/health.js', rel: '/health/service/:id' },
{ method: 'GET', path: '/api/v1/health/services', mount: 'BARE', file: 'routes/health.js', rel: '/health/services' },
{ method: 'GET', path: '/api/v1/system/health', mount: 'BARE', file: 'routes/health.js', rel: '/system/health' },
{ method: 'GET', path: '/api/v1/metrics/prometheus', mount: 'BARE', file: 'src/app.js', rel: '/metrics/prometheus' },
{ method: 'GET', path: '/api/v1/containers', mount: '/containers', file: 'routes/containers.js', rel: '/' },
{ method: 'POST', path: '/api/v1/containers/abc123/start', mount: '/containers', file: 'routes/containers.js', rel: '/:id/start' },
{ method: 'POST', path: '/api/v1/containers/abc123/stop', mount: '/containers', file: 'routes/containers.js', rel: '/:id/stop' },
{ method: 'POST', path: '/api/v1/containers/abc123/restart', mount: '/containers', file: 'routes/containers.js', rel: '/:id/restart' },
{ method: 'POST', path: '/api/v1/containers/abc123/update', mount: '/containers', file: 'routes/containers.js', rel: '/:id/update' },
{ method: 'DELETE', path: '/api/v1/containers/abc123', mount: '/containers', file: 'routes/containers.js', rel: '/:id' },
{ method: 'GET', path: '/api/v1/catalog', mount: 'BARE', file: 'routes/catalog.js', rel: '/catalog' },
{ method: 'GET', path: '/api/v1/catalog/search', mount: 'BARE', file: 'routes/catalog.js', rel: '/catalog/search' },
{ method: 'GET', path: '/api/v1/catalog/plex', mount: 'BARE', file: 'routes/catalog.js', rel: '/catalog/:appId' },
{ method: 'GET', path: '/api/v1/discover', mount: 'BARE', file: 'routes/discover.js', rel: '/discover' },
{ method: 'POST', path: '/api/v1/wizard/recommend', mount: 'BARE', file: 'routes/wizard.js', rel: '/wizard/recommend' },
{ method: 'POST', path: '/api/v1/caddycode/generate', mount: 'BARE', file: 'routes/caddycode.js', rel: '/caddycode/generate' },
{ method: 'GET', path: '/api/v1/dns/records', mount: '/dns', file: 'routes/dns.js', rel: '/records' },
{ method: 'POST', path: '/api/v1/disaster/backup', mount: 'BARE', file: 'routes/disaster-recovery.js', rel: '/disaster/backup' },
{ method: 'GET', path: '/api/v1/disaster/status', mount: 'BARE', file: 'routes/disaster-recovery.js', rel: '/disaster/status' },
{ method: 'GET', path: '/api/v1/fleet/hosts', mount: 'BARE', file: 'routes/fleet.js', rel: '/fleet/hosts' },
{ method: 'GET', path: '/api/v1/config', mount: 'BARE', file: 'routes/config/settings.js', rel: '/config', bareMount: /apiRouter\.use\(configRoutes\(ctx\)\)/ },
];
function extractRoutes(file) {
@@ -51,7 +80,9 @@ function extractRoutes(file) {
for (const raw of src.split('\n')) {
const line = raw.trim();
if (line.startsWith('//') || line.startsWith('*')) continue; // skip comments
for (const m of raw.matchAll(/router\.(get|post|put|delete|patch)\(\s*'([^']*)'/g)) {
// (?:api)? Router — also matches inline `apiRouter.get(...)` declarations
// in src/app.js (DC-170: the prometheus metrics endpoint lives there).
for (const m of raw.matchAll(/(?:api)?[Rr]outer\.(get|post|put|delete|patch)\(\s*'([^']*)'/g)) {
out.push({ method: m[1].toUpperCase(), rel: m[2] || '/' });
}
}
@@ -67,13 +98,98 @@ describe('DC-166 r2: exact (method, path) resolution — no double prefixes', ()
const appSrc = fs.readFileSync(path.join(API, 'src/app.js'), 'utf8');
for (const e of EXPECTED) {
if (e.mount === 'BARE') {
expect(appSrc).toMatch(/apiRouter\.use\(capabilitiesRoute\.buildRouter\(\)\)/);
// DC-170: BARE rows assert the actual module mount for their file so a
// silently-unmounted router can't pass the table (r1 judge blocker
// urn:ump:jsobkg2t2omzvpinel2fufk7sspoy3vubsrth3e6lat3ygolvffq asked
// for per-row BARE verification, not the old capabilities-only check).
expect(appSrc).toMatch(e.bareMount || /apiRouter\.use\(capabilitiesRoute\.buildRouter\(\)\)/);
} else {
expect(appSrc).toContain(`apiRouter.use('${e.mount}'`);
}
}
});
// DC-170 (audit #40, merge-judge r2 blocker on the static table): derive the
// (method, path) from the REAL tool handler by driving createToolHandler
// against a capture fetch, then resolve every captured call against the
// union of all declared routes. This fails if a handler drifts even when
// the hand-written table above is stale.
const HANDLER_SAMPLE_ARGS = {
dashcaddy_get_service: { serviceId: 'plex' },
dashcaddy_check_health: {},
dashcaddy_system_health: {},
dashcaddy_system_metrics: {},
dashcaddy_list_containers: {},
dashcaddy_container_action: { containerId: 'abc123', action: 'restart' },
dashcaddy_search_catalog: {},
dashcaddy_discover_services: {},
dashcaddy_deploy_app: { templateId: 'plex' },
dashcaddy_wizard_recommend: { categories: ['media-streaming'] },
dashcaddy_list_dns: {},
dashcaddy_generate_caddyfile: { domain: 'x.example.com', upstream: 'localhost:80' },
dashcaddy_diagnose: {},
dashcaddy_create_backup: {},
dashcaddy_get_backup_status: {},
dashcaddy_list_fleet: {},
dashcaddy_calendar_events: {},
dashcaddy_calendar_add_source: { name: 'x', type: 'ics', url: 'https://c.example/x.ics' },
dashcaddy_security_events: {},
dashcaddy_security_summary: {},
dashcaddy_license_status: {},
dashcaddy_capabilities: {},
dashcaddy_dns_add_record: { domain: 'w.example.com', ip: '1.2.3.4' },
dashcaddy_dns_remove_record: { domain: 'w.example.com' },
dashcaddy_deploy_recipe: { recipeId: 'media-stack' },
dashcaddy_run_workflow: { workflowId: 'disk-space-alert' },
dashcaddy_tailscale_status: {},
};
test('every tool handler call resolves against a declared route (handler-derived)', async () => {
const { TOOLS, createToolHandler } = require('../../src/mcp/mcp-core');
const captured = [];
const fetchImpl = async (url, opts = {}) => {
captured.push({ method: (opts.method || 'GET').toUpperCase(), url: String(url) });
return { ok: true, status: 200, text: async () => JSON.stringify({ success: true }) };
};
const handleTool = createToolHandler({ baseUrl: 'http://127.0.0.1:1', apiKey: 'dk_driftgate', fetchImpl });
// Mount-aware resolver: each EXPECTED row owns (mount, file) — the mount
// test already proves app.js mounts it. Build (mount, decls) index.
const mounts = [];
for (const e of EXPECTED) {
let entry = mounts.find((m) => m.mount === e.mount && m.file === e.file);
if (!entry) { entry = { mount: e.mount, file: e.file, decls: extractRoutes(e.file).map((r) => ({ ...r, rel: normalize(r.rel) })) }; mounts.push(entry); }
}
const segMatchH = (decl, segs) => {
const parts = (decl || '').split('/').filter(Boolean);
if (parts.length !== segs.length) return false;
return parts.every((p, i) => p.startsWith(':') || p === segs[i]);
};
const resolveCall = (method, pathname) => {
for (const m of mounts) {
const prefix = `/api/v1${m.mount === 'BARE' ? '' : m.mount}`;
if (!pathname.startsWith(prefix)) continue;
const segs = pathname.slice(prefix.length).split('/').filter(Boolean);
const hit = m.decls.find((r) => r.method === method && segMatchH(r.rel, segs));
if (hit) return { mount: m.mount, file: m.file, rel: hit.rel };
}
return null;
};
for (const tool of TOOLS) {
captured.length = 0;
await handleTool(tool.name, HANDLER_SAMPLE_ARGS[tool.name] || {});
expect(captured.length).toBeGreaterThan(0); // every tool must hit the API
for (const c of captured) {
const u = new URL(c.url);
expect(u.pathname.startsWith('/api/v1/')).toBe(true);
const resolved = resolveCall(c.method, u.pathname);
// STRICT (r2 judge blocker on vacuous `hit || fallback`): any tool call
// resolving to no declared route must FAIL here.
expect(resolved || null).toEqual(expect.anything());
if (!resolved) throw new Error(`UNRESOLVED tool call: ${tool.name} -> ${c.method} ${u.pathname}`);
}
}
});
test('every MCP tool path resolves exactly to a declared route (method+pattern)', () => {
const segMatch = (decl, segs) => {
const parts = decl.split('/').filter(Boolean);
dashcaddy-api/__tests__/mcp/mcp-core-dc166.test.js
+54
View File
@@ -190,10 +190,64 @@ describe('DC-166 handler routing (captured fetchImpl)', () => {
expect(calls[0].headers['x-api-key']).toBe('dk_' + 'test_key');
});
test('credential passthrough preserved for JWT callers (Bearer), not just x-api-key', async () => {
// Merge-judge r2 polish (urn:ump:5clu4udrbwuwxn74tqwxtukfjnrwkrbiykp6uh66ipipzxr32xsq):
// the DC-162 contract preserves the CALLER'S credential TYPE — exercise the
// Bearer path, which previously had zero coverage.
const { calls, handleTool } = harness({ headers: { Authorization: 'Bearer test_jwt' } });
await handleTool('dashcaddy_dns_add_record', { domain: 'w.example', ip: '1.1.1.1' });
expect(calls[0].headers['Authorization']).toBe('Bearer ' + 'test_jwt');
expect(calls[0].headers['x-api-key']).toBeUndefined();
});
test('unknown tool still rejected after catalog extension', async () => {
const { handleTool } = harness();
const r = await handleTool('dashcaddy_delete_everything', {});
expect(r.error).toBe(true);
expect(r.message).toMatch(/Unknown tool/);
});
test('deploy_app reports partial failure when Caddyfile generation fails (DC-170)', async () => {
// r2 judge blocker (urn:ump:jsobkg2t2omzvpinel2fufk7sspoy3vubsrth3e6lat3ygolvffq):
// a caddy-code failure was swallowed — registered:true was returned while
// the message claimed the route was generated.
const calls = [];
const fetchImpl = async (url, opts = {}) => {
calls.push(String(url));
if (String(url).includes('/caddycode/generate')) {
return { ok: false, status: 500, text: async () => JSON.stringify({ error: 'caddy templating exploded' }) };
}
return { ok: true, status: 200, text: async () => JSON.stringify({
success: true, dashboardHost: 'status.example.net',
apps: [{ id: 'plex', name: 'Plex', category: 'media', image: 'plex/plex', ports: [32400] }],
}) };
};
const handleTool = createToolHandler({ baseUrl: 'http://127.0.0.1:1', apiKey: 'dk_x', fetchImpl });
const r = await handleTool('dashcaddy_deploy_app', { templateId: 'plex' });
expect(calls.filter((c) => c.includes('/config'))).toHaveLength(1); // base domain from /config, not hardcoded .sami
expect(r.registered).toBe(true);
expect(r.caddyfile).toBeNull();
expect(r.caddyfileError).toMatch(/caddy templating exploded/);
expect(r.message).toMatch(/FAILED/);
expect(r.url).toBe('https://plex.example.net');
});
test('deploy_app derives the base domain from /config (no hardcoded .sami)', async () => {
const calls = [];
const fetchImpl = async (url, opts = {}) => {
calls.push({ url: String(url), body: opts.body ? JSON.parse(opts.body) : undefined });
return { ok: true, status: 200, text: async () => JSON.stringify({
success: true, dashboardHost: 'status.example.net',
apps: [{ id: 'plex', name: 'Plex', category: 'media', image: 'plex/plex', ports: [32400] }],
caddyfile: 'generated-route',
}) };
};
const handleTool = createToolHandler({ baseUrl: 'http://127.0.0.1:1', apiKey: 'dk_x', fetchImpl });
const r = await handleTool('dashcaddy_deploy_app', { templateId: 'plex', subdomain: 'plex' });
expect(r.registered).toBe(true);
expect(r.url).toBe('https://plex.example.net');
const svcCall = calls.find((c) => c.url.includes('/services'));
expect(svcCall.body.domain).toBe('plex.example.net');
expect(JSON.stringify(svcCall.body)).not.toMatch(/\.sami/);
});
});