Skip to content

Architecture

System shape and invariants for this monorepo. For where things live, see STRUCTURE.md. For per-app operational detail (deploy procedures, runbooks, secret contracts) see each apps/<name>/AGENTS.md and packages/cli/AGENTS.md. For the CLI surface and command help, see packages/cli/AGENTS.md.

This is a Bun-workspace monorepo that deploys and manages personal infrastructure: KeeWeb (box.heatvision.co), a CLIProxyAPI Claude proxy (cliproxy.fro.bot), the Fro Bot Discord gateway (gateway.fro.bot), Umami analytics (metrics.fro.bot), a Fro Bot monitoring dashboard (dashboard.fro.bot), an OIDC credential broker (broker.fro.bot), and a WireGuard VPN egress box on AWS Lightsail (eu-west-1). Each deployable lives under apps/; each is self-contained (its own Docker Compose stack or build, a TypeScript deploy script, and a provisioning script). DigitalOcean droplets host the Docker apps; KeeWeb deploys to a Mail-in-a-Box server over SSH/rsync; the VPN box runs native wg-quick@wg0 + systemd on AWS Lightsail, provisioned via @aws-sdk/client-lightsail. apps/agent is the one non-deployable: a private, operator-run AWS provisioner for fro-bot/agent durable S3 session storage (account-level OIDC provider, dedicated bucket, per-repo IAM role/policy) — it has no deploy script, Docker Compose stack, or deploy workflow; bun run provision:agent is its only entry point.

packages/cli is the unified operator surface — a goke CLI (@marcusrbrown/infra) with one command group per app plus a unified status dashboard. packages/shared holds cross-app provisioning helpers and CLIProxyAPI management-API primitives (consumed by apps/cliproxy, apps/broker, and packages/cli). The same CLI exposes a read-only subset of its commands over an MCP bridge so coding agents can query deployment state. GitHub Actions runs the deploy pipeline (one gated workflow per app behind a per-app GitHub Environment) and automation (Fro Bot review, Renovate, releases via Changesets).

Role → path. Reference symbols and files; no line numbers (they rot).

Role Path
CLI entry point (goke) packages/cli/src/cli.ts (registerKeewebCommands, registerAgentCommands, registerCliproxyCommands, registerBrokerCommands, registerGatewayCommands, registerUmamiCommands, registerDashboardCommands, registerVpnCommands, registerStatus, registerMcp)
Per-app CLI command groups packages/cli/src/commands/<app>/ (each <action>.ts + barrel index.ts)
Unified status dashboard packages/cli/src/commands/status.ts
MCP bridge + allowlist packages/cli/src/commands/mcp.ts (MCP_ALLOWLIST, registerMcp)
MCP-capturable action context packages/cli/src/lib/action-ctx.ts (ActionCtx)
Executable conventions enforcement packages/cli/src/conventions.test.ts
Fro Bot daily report reconciler packages/cli/scripts/reconcile-autoheal-reports.ts (reconcileAutohealReports, readReconcilerEnv, summaryLine) + colocated packages/cli/scripts/reconcile-autoheal-reports.test.ts — repo-local workflow code, not a CLI command, MCP tool, package export, or published file
GHCR untagged-version pruner packages/cli/scripts/prune-untagged-packages.ts (pruneUntaggedPackages, readPrunerEnv, summaryLine, nextLink, TARGET_PACKAGES) + colocated packages/cli/scripts/prune-untagged-packages.test.ts, driven only by .github/workflows/prune-packages.yaml — repo-local workflow code, same off-surface rules as the reconciler
Deploy-gate Discord notifier packages/cli/scripts/deploy-gate-notify.ts + colocated packages/cli/scripts/deploy-gate-notify.test.ts, invoked only by the notify job of each .github/workflows/deploy-<app>.yaml — repo-local, zero-dependency workflow code, same off-surface rules as the reconciler
CLI bundle build packages/cli/scripts/build.ts (externalizes public runtime deps, inlines the private @marcusrbrown/infra-shared, copies known_hosts as a file asset)
App deploy scripts apps/<name>/src/deploy.ts (main/deploy) — except keeweb
KeeWeb build + deploy apps/keeweb/src/build.ts, apps/keeweb/deploy.sh
Droplet provisioning apps/<name>/server/provision-droplet.ts (cliproxy, gateway, umami, dashboard, broker)
Dashboard deploy script apps/dashboard/src/deploy.ts (deploy, validateEnv, buildEnvFileContents, assertRunningImageDigest)
Dashboard provisioning apps/dashboard/server/provision-droplet.ts (main, validateRequiredEnv, checkDropletExistence, getDashboardSshFingerprint, establishSshAccess)
Dashboard host validators apps/dashboard/src/host.ts (validateDashboardHost), packages/cli/src/commands/dashboard/host.ts (validateDashboardHost)
Dashboard CLI command group packages/cli/src/commands/dashboard/ (status.ts, deploy.ts, logs.ts, index.ts → registerDashboardCommands)
Dashboard image pin apps/dashboard/docker-compose.yaml (pins ghcr.io/fro-bot/dashboard by tag and digest; Renovate bumps the pin on new releases)
VPN provisioning (Lightsail) apps/vpn/server/provision.ts — @aws-sdk/client-lightsail; resolves blueprint/bundle live, set-exact firewall, imports Ed25519 key, installs WireGuard, pins IP host key
VPN peer model packages/cli/src/commands/vpn/peers.ts (readPeers, writePeers, parsePeersJson, renderServerConfig, Peer), exported as @marcusrbrown/infra/vpn/peers and imported by apps/vpn/src/deploy.ts
Shared SSH/SCP/DO helpers packages/shared/server/droplet-helpers.ts (ssh, scp, waitForSsh, getSshFingerprint, pinHostKeys, materializeIdentityFile)
Shared CLIProxyAPI management-API helpers packages/shared/cliproxy/management.ts (managementHeaders, requestJson, HTTP_TIMEOUT_MS, parseManagementKeyList, toStringArray, readOAuthModelAliasFromConfig, applyOAuthModelAlias) — consumed by apps/cliproxy/src/deploy.ts, apps/broker/src/mint.ts, and every packages/cli/src/commands/cliproxy/*.ts action
Gateway upstream daemon pin apps/gateway/upstream.json
Broker OIDC verification apps/broker/src/oidc.ts (verifyOidcToken, assertNotReplayed)
Broker trust policy apps/broker/src/policy.ts (BROKER_TRUST_POLICY, evaluateClaims)
Broker mint/revoke client apps/broker/src/mint.ts (mintKey, revokeKey, listApiKeys, single-flight lock)
Broker sweeper apps/broker/src/sweeper.ts (sweepExpired, reconcile, startupReconcile, startSweeper)
Broker CLI command group packages/cli/src/commands/broker/ (status.ts, deploy.ts, logs.ts, index.ts → registerBrokerCommands)
Agent AWS provisioner (operator-run, no deploy) apps/agent/server/provision.ts (IAM + S3 convergence, readback verification, handoff manifest, --teardown)
Agent action key layout apps/agent/src/key-layout.ts (S3 session/coordination-lock paths the Action uses, independent of the Action version)
Agent CLI command group packages/cli/src/commands/agent/ (index.ts → registerAgentCommands → registerAgentSetup (agent setup, wraps shared setup-core) + registerAgentStorageCommand (agent storage, agent storage teardown))
Agent workflow verifier packages/cli/src/commands/agent/workflow-verify.ts (verifyWorkflow — validates fro-bot.yaml job-split, fro-bot-storage environment gate, and reachability before wiring storage vars)
Per-app host validators apps/<name>/src/host.ts and packages/cli/src/commands/<app>/host.ts
Deploy pipeline .github/workflows/deploy.yaml (router) + deploy-<app>.yaml
Release pipeline .github/workflows/release.yaml (Changesets version/publish + npm-registry publish verification), .github/workflows/release-alert.yaml (files/updates a GitHub issue when Release fails)
Pinned SSH host keys .github/known_hosts

Operator / agent → deploy or status:

operator (CLI) or agent (MCP)
→ packages/cli/src/cli.ts (goke parse)
→ packages/cli/src/commands/<app>/<action>.ts (goke action, receives ctx)
→ packages/shared/server/droplet-helpers.ts (SSH/SCP) or app deploy.ts
→ droplet: docker compose / rsync over SSH
→ result captured via ctx (MCP) or printed (terminal)

MCP bridge: registerMcp exposes only the MCP_ALLOWLIST set — gateway status, cliproxy status, cliproxy models, keeweb status, umami status, dashboard status, vpn status, broker status, and unified status. All mutating commands (keys, config, deploy, backup, logs, client management) are source-gated out of MCP. Allowlisted actions thread ctx (packages/cli/src/lib/action-ctx.ts) so captured output reaches the agent; global console/process.stdout bypass capture and must not be used in MCP-exposed bodies.

Deploy pipeline: a push to main triggers .github/workflows/deploy.yaml, which runs dorny/paths-filter (predicate-quantifier: every) and routes to the matching deploy-<app>.yaml. Each app deploy waits at its per-app GitHub Environment approval gate before touching the droplet.

Fro Bot daily reconciliation: the fro-bot-storage job of .github/workflows/fro-bot.yaml owns schedule and main-branch workflow_dispatch runs. One pre-agent classification drives prompt selection, skip-cache, and the reconciler guard: a schedule run, or a workflow_dispatch with an omitted or exactly empty prompt, is daily-equivalent; every non-empty prompt (including whitespace-only) is custom. Daily-equivalent runs freeze the UTC date and run ID into the trusted daily prompt before the agent, skip session-cache restore while retaining the S3 backup, and run the reconciler only after a successful agent step. Custom dispatches keep normal cache restore and never reconcile. Workflow-level concurrency runs before steps, so it carries an equivalent pre-run github/inputs predicate; conventions tests in packages/cli/src/conventions.test.ts lock the step-level classifier and that predicate to the same fixture matrix.

The agent owns report prose, create/update, and untrusted-collision visibility; the post-agent reconciler owns issue identity and state. It authenticates fro-bot by numeric actor ID, classifies managed/adoptable candidates from exact title/date plus the first-line managed marker, adopts the autoheal-report label, dedupes on a stable supersession-comment marker, closes superseded trusted reports, and proves final state by fresh exhaustive discovery. It reads and writes by numeric issue ID/number, excludes pull-request-shaped records, fully paginates, requires a bounded readback per postcondition, caps managed/adoptable candidates, and fails closed on ambiguity instead of guessing. Every terminal path emits one body-free machine-readable summary; tokens, report/comment bodies, secrets, and private cross-project text are never emitted. The reconciler receives the existing FRO_BOT_PAT only through its step-local environment, and its step blanks the temporary AWS credentials the preceding OIDC step exports.

Enforceable rules. Many are gated by packages/cli/src/conventions.test.ts, ESLint, or review; mirror the (enforced) anti-patterns in the root AGENTS.md.

  1. apps/ are deployable units; packages/ are reusable libraries. packages/ never imports from apps/. apps/agent is the sole non-deployable exception — a private operator-run provisioner with no deploy script or deploy workflow.
  2. Exactly two approved Bash scripts: apps/keeweb/deploy.sh and apps/umami/retention.sh. apps/umami/retention.sh is the narrow host-native systemd/Docker exception; packages/cli/src/conventions.test.ts enforces the allowlist. Every other script is TypeScript run via bun run.
  3. Never pass secret bytes via argv. Secret material is piped through SSH stdin (writeRemoteFile pattern); --body <value> patterns are banned.
  4. Host validation before SSH. Every SSH-spawning command validates the host (host.ts, rejecting --prefixed and invalid values) before constructing the SSH command.
  5. GitHub Actions are SHA-pinned with a trailing # vX.Y.Z comment, or # name@X.Y.Z for a scoped release tag; workflow files use .yaml (not .yml).
  6. No as any / @ts-ignore / @ts-expect-error — fix the types. No TS files excluded from type checking.
  7. Tracked files never contain secret values. Secrets are injected at build/deploy time from environment or GitHub Environment secrets.
  8. Deploy workflows use dorny/paths-filter with predicate-quantifier: every so ! negations work and docs/tests do not trigger deploys.
  9. Never use ssh-keyscan in CI — host keys are pinned in .github/known_hosts; provisioning scripts may use it locally via pinHostKeys.
  10. MCP exposes read commands only (MCP_ALLOWLIST); mutating commands are source-gated out.
  11. Reusable-workflow permissions must match caller grants. A callee job cannot request a GITHUB_TOKEN scope its caller job does not grant; job-level permissions: replaces the workflow-level block, so callers must restate contents: read alongside added scopes or GitHub rejects the run at startup with zero jobs created.
  12. Workflow-internal scripts stay outside the published surface. packages/cli/scripts/reconcile-autoheal-reports.ts, packages/cli/scripts/prune-untagged-packages.ts, and packages/cli/scripts/deploy-gate-notify.ts are repo-local operational workflow code — not CLI commands, MCP tools, package exports, or published package files (the published files list is dist and src/commands/vpn/peers.ts). Each is invoked only by its owning workflow: the daily-equivalent branch of .github/workflows/fro-bot.yaml, the manual dispatch of .github/workflows/prune-packages.yaml, and the notify jobs of the .github/workflows/deploy-<app>.yaml workflows respectively.
  13. Autoheal reconciliation is identity-anchored and fail-closed. The reconciler mutates only a report matching the exact title/date, numeric fro-bot actor, and first-line managed marker, and only when exactly one current-date report carries the exact current run marker. It excludes pull-request-shaped records, fully paginates, re-reads every mutation target by numeric ID immediately before writing, and aborts rather than mutating on any ambiguity, candidate-cap overflow, incomplete pagination, or failed readback. It never creates or rewrites report prose.
  14. Never read a coarse exit code or HTTP status as proof of a specific remote state. A probe that cannot confidently distinguish its target states must throw, not fall back to the branch that looks safe. remoteGitExists (apps/gateway/src/deploy.ts) and remoteFileExists (apps/cliproxy/src/deploy.ts) emit an explicit EXISTS/MISSING sentinel over SSH and require both exit 0 and a recognized sentinel, rather than mapping any non-zero exit to “absent” (which risked an unnecessary clone or overwriting config.yaml’s runtime API keys); readRemoteChecksum throws on a non-zero exit instead of reading a failed read as an empty checksum. removeStaleGatewayNet requires Docker’s literal “No such network” stderr marker before treating a network as absent. apps/umami/src/deploy.ts’s admin-login probe parses the real HTTP status via curl’s -w '%{http_code}' trailer (401 = already rotated, anything else fails closed) instead of trusting --fail-with-body’s exit code. apps/vpn/src/deploy.ts’s wipe-guard throws on a non-zero wg show wg0 dump exit or empty output instead of reading either as “zero live peers.” apps/broker/src/sweeper.ts’s reconcile takes an opt-in throwOnListFailure so startupReconcile’s bounded retry actually retries a failed listApiKeys call instead of the exception being swallowed and read as success.
  15. Destructive registry automation is dry-run by default, manually dispatched, and gated per target. packages/cli/scripts/prune-untagged-packages.ts deletes GHCR versions only for the fixed TARGET_PACKAGES set, only when passed --apply (the prune-packages.yaml apply input defaults to false), and only after every gate passes for that package: provably complete pagination (MAX_PAGES), at least one tagged version, untagged candidates under CANDIDATE_CAP, no tagged manifest that is itself an index, and no untagged digest referenced as a child manifest of any tagged version. Any gate failure, unrecognized response shape, or unexpected error aborts that package’s run with a coded, body-free summary and a non-zero exit — it never degrades an inconclusive probe into “safe to delete.”
  • Secret materialization. Deploy scripts write secrets to droplet files over SSH stdin (never argv), and CI loads GitHub Environment secrets into the deploy step. Local runs load the repo-root .env (Bun loads .env from CWD only — run via root provision:<app> / deploy:<app> wrappers).
  • Deploy gating. Each app has a GitHub Environment (keeweb, cliproxy, gateway, umami, vpn, dashboard, broker) with a required reviewer and a main-only branch policy. Merging a deploy-triggering change holds at the approval gate. Each deploy-<app>.yaml runs an ungated notify job before the gated job; it executes packages/cli/scripts/deploy-gate-notify.ts to post one Discord message, reading the DEPLOY_GATE_DISCORD_WEBHOOK repository secret that deploy.yaml passes explicitly to each callee (the job runs only on main). A notify failure never fails or skips the deploy: the gated job depends on notify but ignores its result, and the script always exits 0 with a warning. All seven deploy workflows hold their deploy-<app>-<ref> concurrency group at workflow level (cancel-in-progress: false), so a queued run posts only when it is next in line for its own gate; the deploy.yaml router has no top-level concurrency. Operator setup and rotation: docs/runbooks/deploy-gate-notifications.md.
  • Host-key pinning. .github/known_hosts pins both domain (unhashed) and IP (hashed) entries; CI connects with strict host-key checking against only that file.
  • Upstream pinning + verify-at-tag. The gateway daemon source is pinned in apps/gateway/upstream.json; bumping it requires diffing the daemon’s required-secret contract against the compose wiring at the new tag before deploy. This is independent of the Fro Bot review Action SHA pinned in .github/workflows/fro-bot.yaml (see the “Two independent fro-bot/agent version surfaces” gotcha in docs/solutions/best-practices/off-droplet-docker-image-build-gateway-deploy-2026-06-04.md). The dashboard image is pinned by tag and digest directly in apps/dashboard/docker-compose.yaml; Renovate tracks ghcr.io/fro-bot/dashboard and opens PRs to bump the pin when a new release is published. The deploy pulls the compose-pinned digest — no CI image build step. The former ghcr.io/marcusrbrown/infra-dashboard image is retired and can be deleted manually from the GitHub Container Registry once no references remain.
  • Executable conventions. packages/cli/src/conventions.test.ts enforces several invariants above (SHA-pinned actions, .yaml extension, no stray Bash scripts, no ssh-keyscan in workflows, no bundledDependencies, reusable-workflow permission parity between deploy.yaml router jobs and the deploy-<app>.yaml callees they uses) as part of the test suite.
  • Releases. Changesets version @marcusrbrown/infra; only packages/cli/src/ user-facing changes warrant a changeset. Private workspace packages never enter a release: .changeset/config.json sets privatePackages: {tag: false, version: false} with an empty ignore, renovate-changesets.yaml’s exclude-patterns must cover every private package directory, and packages/cli/src/conventions.test.ts asserts @marcusrbrown/infra is the only non-private package and that no changeset mixes private and publishable names. release.yaml verifies every published package resolves on the npm registry (up to 13 attempts with exponential backoff from 3s to a 30s cap, ~285s total) before the run completes, and fails closed if it does not; release-alert.yaml triggers on workflow_run completion of the Release workflow and opens or updates a single release-publish-failure-labeled issue when it fails, so a broken publish can’t go unnoticed (see docs/solutions/integration-issues/npm-publish-stacked-auth-failures-2026-08-23.md).

Integration-level patterns; for the mechanical file layout see STRUCTURE.md.

  • New deployable app → create apps/<name>/ mirroring the closest existing app (apps/cliproxy/ for a Docker-Compose droplet app, apps/vpn/ for a native-systemd AWS Lightsail app): Compose config or deploy script, src/deploy.ts, server/provision.ts (using packages/shared/server/droplet-helpers.ts for SSH helpers, or @aws-sdk/client-lightsail for Lightsail), src/host.ts. New provisioners use provision.ts; the existing DigitalOcean apps keep their provision-droplet.ts filenames. Add it to package.json workspaces, add provision:<name> / deploy:<name> root scripts, add a CLI command group under packages/cli/src/commands/<name>/, add a gated deploy-<name>.yaml workflow wired into deploy.yaml’s paths-filter, create the <name> GitHub Environment, and add apps/<name>/AGENTS.md.
  • New operator-only provisioning tool (no deploy step) → mirror apps/agent/: private: true package, server/provision.ts entry point, no src/deploy.ts, no src/host.ts, no deploy workflow, no GitHub Environment. Add only provision:<name> to root package.json scripts and, if operator interaction is needed, a CLI command group under packages/cli/src/commands/<name>/.
  • New CLI command group → add register<Name>Commands and a commands/<name>/ directory with per-action files and a barrel index.ts; register it in packages/cli/src/cli.ts. Expose a command over MCP only by adding it to MCP_ALLOWLIST, and only if it is read-only.
  • New shared provisioning helper → add it to packages/shared/server/droplet-helpers.ts with a colocated test; consume it from each app’s provisioning script (provision-droplet.ts for cliproxy/gateway/umami, provision.ts for vpn). For CLIProxyAPI management-API primitives (auth headers, JSON request helpers, config parsing), add to packages/shared/cliproxy/management.ts instead — it is the shared source of truth for apps/cliproxy, apps/broker, and packages/cli/src/commands/cliproxy/.
  • New deploy workflow → copy an existing deploy-<app>.yaml, keep the per-app Environment gate and paths-filter negations, and wire it into deploy.yaml.