dashcaddy/BUILD-PIPELINE-FIX.md

Build Pipeline Fix — Complete Source in Tarballs

Date: 2026-07-01 Bug: Every published release tarball at get.dashcaddy.net/release/ was missing dashcaddy-api/src/ — the directory holding ~80% of the application code (app.js, all managers, monitoring, docker, security, utilities modules). Hosts had to run a post-deploy patches script after every update to fix 23+ broken require('./src/...') paths.

---

What was broken

/opt/dashcaddy-release/build-release.sh (the script triggered by the Gitea webhook on push to main) assembled the tarball using these copy commands:

cp -f dashcaddy-api/*.js "$staging/dashcaddy-api/"          # root-level only
cp -rf dashcaddy-api/routes/* "$staging/dashcaddy-api/routes/"
cp -f dashcaddy-api/package.json ...                       # misc root files

It never copied dashcaddy-api/src/, even though server.js does:

const { createApp } = require('./src/app');
const authManager    = require('./src/managers/auth-manager');
const selfUpdater    = require('./src/docker/self-updater');
const healthChecker  = require('./src/monitoring/health-checker');
// ...and 20+ more require('./src/...') calls

Result: every published tarball was missing 60+ source files. The post-deploy script dashcaddy-post-deploy-patches.sh existed only to paper over this gap.

The shipped tarball filename pattern (dashcaddy-${version}.tar.gz), the webroot path (/var/www/get.dashcaddy.net/release/), and existing version.json field names were preserved — only an additive fix.

---

What changed

1. build-release.sh — tarball assembly (lines 45–63)

Added three copy blocks after the existing API files section:

# Application source (this is the bulk of the code: app.js, managers, monitoring, etc.)
if [ -d "dashcaddy-api/src" ]; then
  cp -rf dashcaddy-api/src "$staging/dashcaddy-api/"
else
  log "FATAL: dashcaddy-api/src/ not found in repo — refusing to build incomplete tarball"
  exit 1
fi

# Optional app assets / scripts if they exist
[ -d "dashcaddy-api/assets" ] && cp -rf dashcaddy-api/assets "$staging/dashcaddy-api/"
[ -d "dashcaddy-api/scripts" ] && cp -rf dashcaddy-api/scripts "$staging/dashcaddy-api/"

Also simplified the routes copy from cp -rf dashcaddy-api/routes/* to cp -rf dashcaddy-api/routes — the previous form silently dropped dotfiles/hidden routes and would fail entirely on an empty directory under set -e.

2. build-release.sh — verification step (lines 83–88)

After the tarball is built, a self-check refuses to publish if src/ isn't in it:

if ! tar tzf "$tarball" | grep -q "^dashcaddy/dashcaddy-api/src/"; then
  log "FATAL: tarball is missing dashcaddy-api/src/ — refusing to publish"
  exit 1
fi
log "Tarball contains src/: OK"

This makes the missing-src bug structurally impossible to recur.

3. build-release.sh — src_sha256 field (lines 95–99, 108)

Added computation of a deterministic SHA-256 over the src/ directory contents (files in sorted order, hashed with sha256sum, then the resulting block rehashed):

src_sha256=$(cd "$BUILD_DIR/repo" && find dashcaddy-api/src -type f | sort | xargs sha256sum | sha256sum | cut -d' ' -f1)

This is written into version.json as a new src_sha256 field alongside the existing sha256 (tarball hash). The self-updater at dashcaddy-api/src/docker/self-updater.js can now compare its locally-extracted src/ hash to the remote src_sha256 and detect drift between tarball-level metadata and actual source contents.

version.json schema after the change:

{
  "version": "1.14.6",
  "commit": "abc1234",
  "date": "2026-07-01T08:45:52Z",
  "sha256": "<tarball sha256>",
  "src_sha256": "<deterministic src/ sha256>",
  "changelog": "...",
  "breaking": false,
  "tarball": "dashcaddy-1.14.6.tar.gz"
}

src_sha256 is additive only — no existing field was renamed or removed.

4. Idempotency & safety

• set -euo pipefail preserved.
• All new copies are guarded ([ -d ... ] for optional dirs; explicit if [ -d ... ] for src/ with a fatal exit).
• Tarball filename pattern (dashcaddy-${version}.tar.gz) unchanged.
• Webroot path (/var/www/get.dashcaddy.net/release/) unchanged.
• Mirror rsync step unchanged — destination server will receive the new (complete) tarballs automatically.

---

How to verify locally

The script can be smoke-tested without contacting Gitea or the mirror:

# 1. Snapshot the repo into a scratch dir (avoid touching /opt/dashcaddy)
mkdir -p /tmp/verify/repo
tar --exclude='.git' --exclude='updates' --exclude='backups' \
    -C /opt/dashcaddy -cf - . | tar -C /tmp/verify/repo -xf -

# 2. Replicate the assembly from build-release.sh against the snapshot
cd /tmp/verify/repo
mkdir -p /tmp/verify/dashcaddy/dashcaddy-api/routes /tmp/verify/dashcaddy/status /tmp/verify/dashcaddy/scripts
STG=/tmp/verify/dashcaddy
cp -f dashcaddy-api/*.js "$STG/dashcaddy-api/" 2>/dev/null || true
cp -rf dashcaddy-api/routes "$STG/dashcaddy-api/"
cp -f dashcaddy-api/package.json "$STG/dashcaddy-api/"
cp -f dashcaddy-api/package-lock.json "$STG/dashcaddy-api/" 2>/dev/null || true
cp -f dashcaddy-api/Dockerfile "$STG/dashcaddy-api/"
cp -f dashcaddy-api/openapi.yaml "$STG/dashcaddy-api/" 2>/dev/null || true
[ -d dashcaddy-api/src ] && cp -rf dashcaddy-api/src "$STG/dashcaddy-api/"
[ -d dashcaddy-api/assets ] && cp -rf dashcaddy-api/assets "$STG/dashcaddy-api/"
[ -d dashcaddy-api/scripts ] && cp -rf dashcaddy-api/scripts "$STG/dashcaddy-api/"
# ... status/ + scripts/ as in build-release.sh ...

# 3. Build the tarball and run the verification step
cd /tmp/verify
tar czf test.tar.gz dashcaddy/
tar tzf test.tar.gz | grep -q "^dashcaddy/dashcaddy-api/src/" && echo "src/ present: OK"

# 4. Confirm src_sha256 is deterministic
find dashcaddy-api/src -type f | sort | xargs sha256sum | sha256sum

Expected output:

• src/ present: OK
• src_sha256 identical across two runs (no timestamps or non-deterministic ordering).

The local dry-run on 2026-07-01 produced an 18 MB tarball with 74 src/ entries (was 0 before), and verified that all of src/app.js, src/docker/self-updater.js, src/managers/auth-manager.js, src/managers/resource-monitor.js, src/monitoring/health-checker.js, src/utilities/startup-validator.js, and src/utils/http.js are present.

---

Migration note for existing installations

Hosts already running the old (src-less) release format will need to pick up one of the new tarballs to get the complete source tree:

• Option A (recommended): trigger a normal update from get.dashcaddy.net/release/latest.tar.gz. Because the new tarball includes src/, no post-deploy patching is needed — server.js will resolve every require('./src/...') directly. The post-deploy-patches.sh script remains in place and is still safe to run (it's a no-op on a complete tree).
• Option B (no network): leave the host on its current release. The post-deploy-patches.sh script continues to function as before — it patches the broken require() paths after every update. Nothing changes for offline hosts.

There is no database migration, no config-file change, and no restart ordering change required. The next tarball published after this commit will simply contain the missing src/ directory.

---

Files modified

Path	Change
/opt/dashcaddy-release/build-release.sh	Added src/, assets/, scripts/ copies + verification step + src_sha256 field
/opt/dashcaddy/BUILD-PIPELINE-FIX.md	This document

Files NOT modified (and why)

• dashcaddy-api/src/docker/self-updater.js — src_sha256 is now published in version.json, but the self-updater doesn't need a code change to receive it. Adding the comparison logic in the updater is a separate, optional task that should be done when ready to consume the new field.
• dashcaddy-post-deploy-patches.sh — kept as a safety net; now a no-op for fresh installs but still useful for legacy hosts.
• Any version.json already on disk at /var/www/get.dashcaddy.net/release/ — overwritten automatically on the next release build.