# Build Pipeline Fix — Complete Source in Tarballs

**Date:** 2026-07-01
**Bug:** Every published release tarball at `get.dashcaddy.net/release/` was missing `dashcaddy-api/src/` — the directory holding ~80% of the application code (app.js, all managers, monitoring, docker, security, utilities modules). Hosts had to run a post-deploy patches script after every update to fix 23+ broken `require('./src/...')` paths.

---

## What was broken

`/opt/dashcaddy-release/build-release.sh` (the script triggered by the Gitea webhook on push to `main`) assembled the tarball using these copy commands:

```bash
cp -f dashcaddy-api/*.js "$staging/dashcaddy-api/"          # root-level only
cp -rf dashcaddy-api/routes/* "$staging/dashcaddy-api/routes/"
cp -f dashcaddy-api/package.json ...                       # misc root files
```

It never copied `dashcaddy-api/src/`, even though `server.js` does:

```js
const { createApp } = require('./src/app');
const authManager    = require('./src/managers/auth-manager');
const selfUpdater    = require('./src/docker/self-updater');
const healthChecker  = require('./src/monitoring/health-checker');
// ...and 20+ more require('./src/...') calls
```

**Result:** every published tarball was missing 60+ source files. The post-deploy script `dashcaddy-post-deploy-patches.sh` existed only to paper over this gap.

The shipped tarball filename pattern (`dashcaddy-${version}.tar.gz`), the webroot path (`/var/www/get.dashcaddy.net/release/`), and existing `version.json` field names were preserved — only an additive fix.

---

## What changed

### 1. `build-release.sh` — tarball assembly (lines 45–63)

Added three copy blocks after the existing API files section:

```bash
# Application source (this is the bulk of the code: app.js, managers, monitoring, etc.)
if [ -d "dashcaddy-api/src" ]; then
  cp -rf dashcaddy-api/src "$staging/dashcaddy-api/"
else
  log "FATAL: dashcaddy-api/src/ not found in repo — refusing to build incomplete tarball"
  exit 1
fi

# Optional app assets / scripts if they exist
[ -d "dashcaddy-api/assets" ] && cp -rf dashcaddy-api/assets "$staging/dashcaddy-api/"
[ -d "dashcaddy-api/scripts" ] && cp -rf dashcaddy-api/scripts "$staging/dashcaddy-api/"
```

Also simplified the routes copy from `cp -rf dashcaddy-api/routes/*` to `cp -rf dashcaddy-api/routes` — the previous form silently dropped dotfiles/hidden routes and would fail entirely on an empty directory under `set -e`.

### 2. `build-release.sh` — verification step (lines 83–88)

After the tarball is built, a self-check refuses to publish if `src/` isn't in it:

```bash
if ! tar tzf "$tarball" | grep -q "^dashcaddy/dashcaddy-api/src/"; then
  log "FATAL: tarball is missing dashcaddy-api/src/ — refusing to publish"
  exit 1
fi
log "Tarball contains src/: OK"
```

This makes the missing-src bug structurally impossible to recur.

### 3. `build-release.sh` — `src_sha256` field (lines 95–99, 108)

Added computation of a deterministic SHA-256 over the `src/` directory contents (files in sorted order, hashed with sha256sum, then the resulting block rehashed):

```bash
src_sha256=$(cd "$BUILD_DIR/repo" && find dashcaddy-api/src -type f | sort | xargs sha256sum | sha256sum | cut -d' ' -f1)
```

This is written into `version.json` as a new `src_sha256` field alongside the existing `sha256` (tarball hash). The self-updater at `dashcaddy-api/src/docker/self-updater.js` can now compare its locally-extracted `src/` hash to the remote `src_sha256` and detect drift between tarball-level metadata and actual source contents.

`version.json` schema after the change:

```json
{
  "version": "1.14.6",
  "commit": "abc1234",
  "date": "2026-07-01T08:45:52Z",
  "sha256": "<tarball sha256>",
  "src_sha256": "<deterministic src/ sha256>",
  "changelog": "...",
  "breaking": false,
  "tarball": "dashcaddy-1.14.6.tar.gz"
}
```

`src_sha256` is **additive only** — no existing field was renamed or removed.

### 4. Idempotency & safety

- `set -euo pipefail` preserved.
- All new copies are guarded (`[ -d ... ]` for optional dirs; explicit `if [ -d ... ]` for `src/` with a fatal exit).
- Tarball filename pattern (`dashcaddy-${version}.tar.gz`) unchanged.
- Webroot path (`/var/www/get.dashcaddy.net/release/`) unchanged.
- Mirror rsync step unchanged — destination server will receive the new (complete) tarballs automatically.

---

## How to verify locally

The script can be smoke-tested without contacting Gitea or the mirror:

```bash
# 1. Snapshot the repo into a scratch dir (avoid touching /opt/dashcaddy)
mkdir -p /tmp/verify/repo
tar --exclude='.git' --exclude='updates' --exclude='backups' \
    -C /opt/dashcaddy -cf - . | tar -C /tmp/verify/repo -xf -

# 2. Replicate the assembly from build-release.sh against the snapshot
cd /tmp/verify/repo
mkdir -p /tmp/verify/dashcaddy/dashcaddy-api/routes /tmp/verify/dashcaddy/status /tmp/verify/dashcaddy/scripts
STG=/tmp/verify/dashcaddy
cp -f dashcaddy-api/*.js "$STG/dashcaddy-api/" 2>/dev/null || true
cp -rf dashcaddy-api/routes "$STG/dashcaddy-api/"
cp -f dashcaddy-api/package.json "$STG/dashcaddy-api/"
cp -f dashcaddy-api/package-lock.json "$STG/dashcaddy-api/" 2>/dev/null || true
cp -f dashcaddy-api/Dockerfile "$STG/dashcaddy-api/"
cp -f dashcaddy-api/openapi.yaml "$STG/dashcaddy-api/" 2>/dev/null || true
[ -d dashcaddy-api/src ] && cp -rf dashcaddy-api/src "$STG/dashcaddy-api/"
[ -d dashcaddy-api/assets ] && cp -rf dashcaddy-api/assets "$STG/dashcaddy-api/"
[ -d dashcaddy-api/scripts ] && cp -rf dashcaddy-api/scripts "$STG/dashcaddy-api/"
# ... status/ + scripts/ as in build-release.sh ...

# 3. Build the tarball and run the verification step
cd /tmp/verify
tar czf test.tar.gz dashcaddy/
tar tzf test.tar.gz | grep -q "^dashcaddy/dashcaddy-api/src/" && echo "src/ present: OK"

# 4. Confirm src_sha256 is deterministic
find dashcaddy-api/src -type f | sort | xargs sha256sum | sha256sum
```

Expected output:

- `src/ present: OK`
- `src_sha256` identical across two runs (no timestamps or non-deterministic ordering).

The local dry-run on 2026-07-01 produced an 18 MB tarball with **74 `src/` entries** (was 0 before), and verified that all of `src/app.js`, `src/docker/self-updater.js`, `src/managers/auth-manager.js`, `src/managers/resource-monitor.js`, `src/monitoring/health-checker.js`, `src/utilities/startup-validator.js`, and `src/utils/http.js` are present.

---

## Migration note for existing installations

Hosts already running the old (src-less) release format will need to pick up one of the new tarballs to get the complete source tree:

- **Option A (recommended):** trigger a normal update from `get.dashcaddy.net/release/latest.tar.gz`. Because the new tarball includes `src/`, no post-deploy patching is needed — `server.js` will resolve every `require('./src/...')` directly. The post-deploy-patches.sh script remains in place and is still safe to run (it's a no-op on a complete tree).
- **Option B (no network):** leave the host on its current release. The post-deploy-patches.sh script continues to function as before — it patches the broken `require()` paths after every update. Nothing changes for offline hosts.

There is no database migration, no config-file change, and no restart ordering change required. The next tarball published after this commit will simply contain the missing `src/` directory.

---

## Files modified

| Path | Change |
|---|---|
| `/opt/dashcaddy-release/build-release.sh` | Added `src/`, `assets/`, `scripts/` copies + verification step + `src_sha256` field |
| `/opt/dashcaddy/BUILD-PIPELINE-FIX.md` | This document |

## Files NOT modified (and why)

- `dashcaddy-api/src/docker/self-updater.js` — `src_sha256` is now published in `version.json`, but the self-updater doesn't need a code change to *receive* it. Adding the comparison logic in the updater is a separate, optional task that should be done when ready to consume the new field.
- `dashcaddy-post-deploy-patches.sh` — kept as a safety net; now a no-op for fresh installs but still useful for legacy hosts.
- Any `version.json` already on disk at `/var/www/get.dashcaddy.net/release/` — overwritten automatically on the next release build.