Skip to content

Dashboard Released Image Rollback

The dashboard image is pinned by tag and digest in apps/dashboard/docker-compose.yaml. Renovate opens PRs to bump the pin when a new ghcr.io/fro-bot/dashboard release is published; merging triggers the Deploy Dashboard workflow, which pulls the new digest and ships it to the droplet.

This runbook covers rolling back to a prior image digest when a release is bad. For the deploy flow and anti-patterns, see apps/dashboard/AGENTS.md.

If the release being rolled back changed the operator contract, roll back together with the gateway: see Rollback and Rollback is not pin reversal. Never roll back only the dashboard across a contract change.


The compose pin is the single source of truth for which image version runs on the droplet. Rolling back means reverting the compose pin commit to restore the prior digest, then deploying. There is no separate override file or runtime flag — the compose file is the contract.

A recent prior repository@digest may already be cached on the droplet and can speed rollback if the registry is unavailable, but the reviewed Compose pin plus the normal deploy remains the source of truth and authorization. Cache presence alone is not permission to change runtime state, and the next deploy attempt may prune the cached image.


  • A newly deployed image causes errors, crashes, or health-check failures on dashboard.fro.bot.
  • https://dashboard.fro.bot/api/healthz returns non-200 after a Renovate bump deploy.
  • infra dashboard status shows the dashboard container unhealthy or restarting.
  • infra dashboard logs dashboard reveals a regression introduced by the new release.

Find the commit that last changed the image pin in apps/dashboard/docker-compose.yaml:

Terminal window
git log --oneline -- apps/dashboard/docker-compose.yaml

Identify the commit immediately before the bad bump. Inspect it to confirm the prior digest:

Terminal window
git show <prior-commit>:apps/dashboard/docker-compose.yaml | grep 'image:.*dashboard'

Note the full image reference, e.g.: ghcr.io/fro-bot/dashboard:2026.06.14@sha256:abc123…

Create a revert commit targeting the bad bump:

Terminal window
git revert <bad-bump-commit> --no-edit

Verify the compose file now contains the prior digest:

Terminal window
grep 'image:.*dashboard' apps/dashboard/docker-compose.yaml

Confirm it matches the digest from Step 1.

Terminal window
git push origin HEAD

Open a PR against main. Title it clearly, e.g.: revert: roll back dashboard image to <prior-tag> (regression in <bad-tag>)

Merge the PR. The Deploy Dashboard workflow triggers automatically on merge to main.

Approve the dashboard environment gate in the GitHub Actions UI. The deploy pulls the reverted digest and ships it to the droplet through the single locked SSH transaction. The transaction validates the host, records baseline evidence, always prunes unused images, requires 6 GiB free after prune, verifies the exact staged image set from cache before pulling when possible; otherwise it pulls and verifies the complete set (or accepts a complete exact cache fallback after a failed pull), requires 6 GiB free again, publishes active files with Compose last, verifies dashboard digest and health, and only then unlocks. The remote transaction has a fixed 900-second deadline and the caller watchdog has a 960-second deadline; timeout failures report their safe code and last stage. Advisory probes and audit write-back happen after unlock.

Terminal window
bunx @marcusrbrown/infra dashboard status

Confirm both dashboard and caddy services are healthy.

Probe the public endpoint:

Terminal window
curl -sf https://dashboard.fro.bot/api/healthz

Expect HTTP 200. If Caddy ACME renewal is in progress, retry after 60 seconds.

Check logs for errors:

Terminal window
bunx @marcusrbrown/infra dashboard logs dashboard --tail 50

If the bad release is the first time the dashboard was deployed (no prior digest in git history), there is no compose pin to revert to. Options:

  1. Pin a specific prior tag manually. Browse ghcr.io/fro-bot/dashboard releases (GitHub Packages UI or gh api /orgs/fro-bot/packages/container/dashboard/versions) to find a known-good release. Update the image: line in apps/dashboard/docker-compose.yaml with the tag and digest of that release, commit, PR, merge, and deploy.

  2. Wait for a fixed upstream release. If the upstream fro-bot/dashboard repo is actively fixing the regression, wait for a new release and let Renovate open the bump PR normally.

If the compose file has been modified since the bad bump (e.g., a config tweak landed on top of it), git revert <bad-bump-commit> --no-edit will stop with a merge conflict. Two options:

Option A — staged revert, resolve manually:

Terminal window
git revert <bad-bump-commit> --no-commit

This stages the revert but leaves conflicts in the working tree. Open apps/dashboard/docker-compose.yaml, resolve the conflict markers so the image: line contains the prior digest (from Step 1), then:

Terminal window
git add apps/dashboard/docker-compose.yaml
git revert --continue

Option B — direct edit (faster during an active incident):

If resolving the conflict is slower than the situation allows, edit the image: line directly:

Terminal window
# Set the image line to the known-good tag@digest identified in Step 1
$EDITOR apps/dashboard/docker-compose.yaml

Commit with a clear message:

Terminal window
git add apps/dashboard/docker-compose.yaml
git commit -m "fix: roll back dashboard image to <prior-tag> (regression in <bad-tag>)"

Then continue from Step 3. This is acceptable when it is clearer or faster than resolving the revert conflict — the compose file is the contract; how the commit is authored is secondary.

If a newer release has already shipped and is known-good, prefer bumping forward to that release rather than reverting backward. Update the compose pin to the newer digest, commit, PR, merge, and deploy. This avoids re-introducing the regression window.

Deployment stops before or during rollback

Section titled “Deployment stops before or during rollback”

Resolve the reported condition before rerunning. Every failure reports a stable lowercase-hyphen code and the last completed stage; remote stderr is not surfaced. The deterministic failure classes are lock contention, prune failure, post-prune low headroom, acquisition/cache mismatch, post-acquisition low headroom, unsafe path, payload malformed, transaction timeout, and active publication/runtime failure. A prune failure always stops the deployment; docker image prune -af may let Docker reclaim image data, but the deployment never directly deletes containerd storage or files. It never prunes containers or volumes or uses Compose teardown for recovery.

Stopped containers pin their images. If obsolete stopped containers keep free space below 6 GiB, inspect them and remove only the specifically obsolete containers manually, then rerun. There is no automatic override flag. After a successful rollback, the superseded image is left locally as one temporary rollback generation because pruning occurs only before acquisition; a later deploy attempt may prune it. The deploy does not perform automatic rollback if publication or runtime convergence fails; inspect the reported stage and actual state before retrying.