Architecture
System shape and invariants for this monorepo. For where things live, see STRUCTURE.md. For per-app operational detail (deploy procedures, runbooks, secret contracts) see each apps/<name>/AGENTS.md and packages/cli/AGENTS.md. For the CLI surface and command help, see packages/cli/AGENTS.md.
Bird’s Eye Overview
Section titled “Bird’s Eye Overview”This is a Bun-workspace monorepo that deploys and manages personal infrastructure: KeeWeb (box.heatvision.co), a CLIProxyAPI Claude proxy (cliproxy.fro.bot), the Fro Bot Discord gateway (gateway.fro.bot), Umami analytics (metrics.fro.bot), a Fro Bot monitoring dashboard (dashboard.fro.bot), an OIDC credential broker (broker.fro.bot), and a WireGuard VPN egress box on AWS Lightsail (eu-west-1). Each deployable lives under apps/; each is self-contained (its own Docker Compose stack or build, a TypeScript deploy script, and a provisioning script). DigitalOcean droplets host the Docker apps; KeeWeb deploys to a Mail-in-a-Box server over SSH/rsync; the VPN box runs native wg-quick@wg0 + systemd on AWS Lightsail, provisioned via @aws-sdk/client-lightsail. apps/agent is the one non-deployable: a private, operator-run AWS provisioner for fro-bot/agent durable S3 session storage (account-level OIDC provider, dedicated bucket, per-repo IAM role/policy) — it has no deploy script, Docker Compose stack, or deploy workflow; bun run provision:agent is its only entry point.
packages/cli is the unified operator surface — a goke CLI (@marcusrbrown/infra) with one command group per app plus a unified status dashboard. packages/shared holds cross-app provisioning helpers and CLIProxyAPI management-API primitives (consumed by apps/cliproxy, apps/broker, and packages/cli). The same CLI exposes a read-only subset of its commands over an MCP bridge so coding agents can query deployment state. GitHub Actions runs the deploy pipeline (one gated workflow per app behind a per-app GitHub Environment) and automation (Fro Bot review, Renovate, releases via Changesets).
Codemap
Section titled “Codemap”Role → path. Reference symbols and files; no line numbers (they rot).
| Role | Path |
|---|---|
| CLI entry point (goke) | packages/cli/src/cli.ts (registerKeewebCommands, registerAgentCommands, registerCliproxyCommands, registerBrokerCommands, registerGatewayCommands, registerUmamiCommands, registerDashboardCommands, registerVpnCommands, registerStatus, registerMcp) |
| Per-app CLI command groups | packages/cli/src/commands/<app>/ (each <action>.ts + barrel index.ts) |
| Unified status dashboard | packages/cli/src/commands/status.ts |
| MCP bridge + allowlist | packages/cli/src/commands/mcp.ts (MCP_ALLOWLIST, registerMcp) |
| MCP-capturable action context | packages/cli/src/lib/action-ctx.ts (ActionCtx) |
| Executable conventions enforcement | packages/cli/src/conventions.test.ts |
| Fro Bot daily report reconciler | packages/cli/scripts/reconcile-autoheal-reports.ts (reconcileAutohealReports, readReconcilerEnv, summaryLine) + colocated packages/cli/scripts/reconcile-autoheal-reports.test.ts — repo-local workflow code, not a CLI command, MCP tool, package export, or published file |
| GHCR untagged-version pruner | packages/cli/scripts/prune-untagged-packages.ts (pruneUntaggedPackages, readPrunerEnv, summaryLine, nextLink, TARGET_PACKAGES) + colocated packages/cli/scripts/prune-untagged-packages.test.ts, driven only by .github/workflows/prune-packages.yaml — repo-local workflow code, same off-surface rules as the reconciler |
| Deploy-gate Discord notifier | packages/cli/scripts/deploy-gate-notify.ts + colocated packages/cli/scripts/deploy-gate-notify.test.ts, invoked only by the notify job of each .github/workflows/deploy-<app>.yaml — repo-local, zero-dependency workflow code, same off-surface rules as the reconciler |
| CLI bundle build | packages/cli/scripts/build.ts (externalizes public runtime deps, inlines the private @marcusrbrown/infra-shared, copies known_hosts as a file asset) |
| App deploy scripts | apps/<name>/src/deploy.ts (main/deploy) — except keeweb |
| KeeWeb build + deploy | apps/keeweb/src/build.ts, apps/keeweb/deploy.sh |
| Droplet provisioning | apps/<name>/server/provision-droplet.ts (cliproxy, gateway, umami, dashboard, broker) |
| Dashboard deploy script | apps/dashboard/src/deploy.ts (deploy, validateEnv, buildEnvFileContents, assertRunningImageDigest) |
| Dashboard provisioning | apps/dashboard/server/provision-droplet.ts (main, validateRequiredEnv, checkDropletExistence, getDashboardSshFingerprint, establishSshAccess) |
| Dashboard host validators | apps/dashboard/src/host.ts (validateDashboardHost), packages/cli/src/commands/dashboard/host.ts (validateDashboardHost) |
| Dashboard CLI command group | packages/cli/src/commands/dashboard/ (status.ts, deploy.ts, logs.ts, index.ts → registerDashboardCommands) |
| Dashboard image pin | apps/dashboard/docker-compose.yaml (pins ghcr.io/fro-bot/dashboard by tag and digest; Renovate bumps the pin on new releases) |
| VPN provisioning (Lightsail) | apps/vpn/server/provision.ts — @aws-sdk/client-lightsail; resolves blueprint/bundle live, set-exact firewall, imports Ed25519 key, installs WireGuard, pins IP host key |
| VPN peer model | packages/cli/src/commands/vpn/peers.ts (readPeers, writePeers, parsePeersJson, renderServerConfig, Peer), exported as @marcusrbrown/infra/vpn/peers and imported by apps/vpn/src/deploy.ts |
| Shared SSH/SCP/DO helpers | packages/shared/server/droplet-helpers.ts (ssh, scp, waitForSsh, getSshFingerprint, pinHostKeys, materializeIdentityFile) |
| Shared CLIProxyAPI management-API helpers | packages/shared/cliproxy/management.ts (managementHeaders, requestJson, HTTP_TIMEOUT_MS, parseManagementKeyList, toStringArray, readOAuthModelAliasFromConfig, applyOAuthModelAlias) — consumed by apps/cliproxy/src/deploy.ts, apps/broker/src/mint.ts, and every packages/cli/src/commands/cliproxy/*.ts action |
| Gateway upstream daemon pin | apps/gateway/upstream.json |
| Broker OIDC verification | apps/broker/src/oidc.ts (verifyOidcToken, assertNotReplayed) |
| Broker trust policy | apps/broker/src/policy.ts (BROKER_TRUST_POLICY, evaluateClaims) |
| Broker mint/revoke client | apps/broker/src/mint.ts (mintKey, revokeKey, listApiKeys, single-flight lock) |
| Broker sweeper | apps/broker/src/sweeper.ts (sweepExpired, reconcile, startupReconcile, startSweeper) |
| Broker CLI command group | packages/cli/src/commands/broker/ (status.ts, deploy.ts, logs.ts, index.ts → registerBrokerCommands) |
| Agent AWS provisioner (operator-run, no deploy) | apps/agent/server/provision.ts (IAM + S3 convergence, readback verification, handoff manifest, --teardown) |
| Agent action key layout | apps/agent/src/key-layout.ts (S3 session/coordination-lock paths the Action uses, independent of the Action version) |
| Agent CLI command group | packages/cli/src/commands/agent/ (index.ts → registerAgentCommands → registerAgentSetup (agent setup, wraps shared setup-core) + registerAgentStorageCommand (agent storage, agent storage teardown)) |
| Agent workflow verifier | packages/cli/src/commands/agent/workflow-verify.ts (verifyWorkflow — validates fro-bot.yaml job-split, fro-bot-storage environment gate, and reachability before wiring storage vars) |
| Per-app host validators | apps/<name>/src/host.ts and packages/cli/src/commands/<app>/host.ts |
| Deploy pipeline | .github/workflows/deploy.yaml (router) + deploy-<app>.yaml |
| Release pipeline | .github/workflows/release.yaml (Changesets version/publish + npm-registry publish verification), .github/workflows/release-alert.yaml (files/updates a GitHub issue when Release fails) |
| Pinned SSH host keys | .github/known_hosts |
Data Flow
Section titled “Data Flow”Operator / agent → deploy or status:
operator (CLI) or agent (MCP) → packages/cli/src/cli.ts (goke parse) → packages/cli/src/commands/<app>/<action>.ts (goke action, receives ctx) → packages/shared/server/droplet-helpers.ts (SSH/SCP) or app deploy.ts → droplet: docker compose / rsync over SSH → result captured via ctx (MCP) or printed (terminal)MCP bridge: registerMcp exposes only the MCP_ALLOWLIST set — gateway status, cliproxy status, cliproxy models, keeweb status, umami status, dashboard status, vpn status, broker status, and unified status. All mutating commands (keys, config, deploy, backup, logs, client management) are source-gated out of MCP. Allowlisted actions thread ctx (packages/cli/src/lib/action-ctx.ts) so captured output reaches the agent; global console/process.stdout bypass capture and must not be used in MCP-exposed bodies.
Deploy pipeline: a push to main triggers .github/workflows/deploy.yaml, which runs dorny/paths-filter (predicate-quantifier: every) and routes to the matching deploy-<app>.yaml. Each app deploy waits at its per-app GitHub Environment approval gate before touching the droplet.
Fro Bot daily reconciliation: the fro-bot-storage job of .github/workflows/fro-bot.yaml owns schedule and main-branch workflow_dispatch runs. One pre-agent classification drives prompt selection, skip-cache, and the reconciler guard: a schedule run, or a workflow_dispatch with an omitted or exactly empty prompt, is daily-equivalent; every non-empty prompt (including whitespace-only) is custom. Daily-equivalent runs freeze the UTC date and run ID into the trusted daily prompt before the agent, skip session-cache restore while retaining the S3 backup, and run the reconciler only after a successful agent step. Custom dispatches keep normal cache restore and never reconcile. Workflow-level concurrency runs before steps, so it carries an equivalent pre-run github/inputs predicate; conventions tests in packages/cli/src/conventions.test.ts lock the step-level classifier and that predicate to the same fixture matrix.
The agent owns report prose, create/update, and untrusted-collision visibility; the post-agent reconciler owns issue identity and state. It authenticates fro-bot by numeric actor ID, classifies managed/adoptable candidates from exact title/date plus the first-line managed marker, adopts the autoheal-report label, dedupes on a stable supersession-comment marker, closes superseded trusted reports, and proves final state by fresh exhaustive discovery. It reads and writes by numeric issue ID/number, excludes pull-request-shaped records, fully paginates, requires a bounded readback per postcondition, caps managed/adoptable candidates, and fails closed on ambiguity instead of guessing. Every terminal path emits one body-free machine-readable summary; tokens, report/comment bodies, secrets, and private cross-project text are never emitted. The reconciler receives the existing FRO_BOT_PAT only through its step-local environment, and its step blanks the temporary AWS credentials the preceding OIDC step exports.
Invariants
Section titled “Invariants”Enforceable rules. Many are gated by packages/cli/src/conventions.test.ts, ESLint, or review; mirror the (enforced) anti-patterns in the root AGENTS.md.
apps/are deployable units;packages/are reusable libraries.packages/never imports fromapps/.apps/agentis the sole non-deployable exception — a private operator-run provisioner with no deploy script or deploy workflow.- Exactly two approved Bash scripts:
apps/keeweb/deploy.shandapps/umami/retention.sh.apps/umami/retention.shis the narrow host-native systemd/Docker exception;packages/cli/src/conventions.test.tsenforces the allowlist. Every other script is TypeScript run viabun run. - Never pass secret bytes via argv. Secret material is piped through SSH stdin (
writeRemoteFilepattern);--body <value>patterns are banned. - Host validation before SSH. Every SSH-spawning command validates the host (
host.ts, rejecting--prefixed and invalid values) before constructing the SSH command. - GitHub Actions are SHA-pinned with a trailing
# vX.Y.Zcomment, or# name@X.Y.Zfor a scoped release tag; workflow files use.yaml(not.yml). - No
as any/@ts-ignore/@ts-expect-error— fix the types. No TS files excluded from type checking. - Tracked files never contain secret values. Secrets are injected at build/deploy time from environment or GitHub Environment secrets.
- Deploy workflows use
dorny/paths-filterwithpredicate-quantifier: everyso!negations work and docs/tests do not trigger deploys. - Never use
ssh-keyscanin CI — host keys are pinned in.github/known_hosts; provisioning scripts may use it locally viapinHostKeys. - MCP exposes read commands only (
MCP_ALLOWLIST); mutating commands are source-gated out. - Reusable-workflow permissions must match caller grants. A callee job cannot request a
GITHUB_TOKENscope its caller job does not grant; job-levelpermissions:replaces the workflow-level block, so callers must restatecontents: readalongside added scopes or GitHub rejects the run at startup with zero jobs created. - Workflow-internal scripts stay outside the published surface.
packages/cli/scripts/reconcile-autoheal-reports.ts,packages/cli/scripts/prune-untagged-packages.ts, andpackages/cli/scripts/deploy-gate-notify.tsare repo-local operational workflow code — not CLI commands, MCP tools, package exports, or published package files (the publishedfileslist isdistandsrc/commands/vpn/peers.ts). Each is invoked only by its owning workflow: the daily-equivalent branch of.github/workflows/fro-bot.yaml, the manual dispatch of.github/workflows/prune-packages.yaml, and thenotifyjobs of the.github/workflows/deploy-<app>.yamlworkflows respectively. - Autoheal reconciliation is identity-anchored and fail-closed. The reconciler mutates only a report matching the exact title/date, numeric
fro-botactor, and first-line managed marker, and only when exactly one current-date report carries the exact current run marker. It excludes pull-request-shaped records, fully paginates, re-reads every mutation target by numeric ID immediately before writing, and aborts rather than mutating on any ambiguity, candidate-cap overflow, incomplete pagination, or failed readback. It never creates or rewrites report prose. - Never read a coarse exit code or HTTP status as proof of a specific remote state. A probe that cannot confidently distinguish its target states must throw, not fall back to the branch that looks safe.
remoteGitExists(apps/gateway/src/deploy.ts) andremoteFileExists(apps/cliproxy/src/deploy.ts) emit an explicitEXISTS/MISSINGsentinel over SSH and require both exit 0 and a recognized sentinel, rather than mapping any non-zero exit to “absent” (which risked an unnecessary clone or overwritingconfig.yaml’s runtime API keys);readRemoteChecksumthrows on a non-zero exit instead of reading a failed read as an empty checksum.removeStaleGatewayNetrequires Docker’s literal “No such network” stderr marker before treating a network as absent.apps/umami/src/deploy.ts’s admin-login probe parses the real HTTP status via curl’s-w '%{http_code}'trailer (401 = already rotated, anything else fails closed) instead of trusting--fail-with-body’s exit code.apps/vpn/src/deploy.ts’s wipe-guard throws on a non-zerowg show wg0 dumpexit or empty output instead of reading either as “zero live peers.”apps/broker/src/sweeper.ts’sreconciletakes an opt-inthrowOnListFailuresostartupReconcile’s bounded retry actually retries a failedlistApiKeyscall instead of the exception being swallowed and read as success. - Destructive registry automation is dry-run by default, manually dispatched, and gated per target.
packages/cli/scripts/prune-untagged-packages.tsdeletes GHCR versions only for the fixedTARGET_PACKAGESset, only when passed--apply(theprune-packages.yamlapplyinput defaults to false), and only after every gate passes for that package: provably complete pagination (MAX_PAGES), at least one tagged version, untagged candidates underCANDIDATE_CAP, no tagged manifest that is itself an index, and no untagged digest referenced as a child manifest of any tagged version. Any gate failure, unrecognized response shape, or unexpected error aborts that package’s run with a coded, body-free summary and a non-zero exit — it never degrades an inconclusive probe into “safe to delete.”
Cross-Cutting Concerns
Section titled “Cross-Cutting Concerns”- Secret materialization. Deploy scripts write secrets to droplet files over SSH stdin (never argv), and CI loads GitHub Environment secrets into the deploy step. Local runs load the repo-root
.env(Bun loads.envfrom CWD only — run via rootprovision:<app>/deploy:<app>wrappers). - Deploy gating. Each app has a GitHub Environment (
keeweb,cliproxy,gateway,umami,vpn,dashboard,broker) with a required reviewer and a main-only branch policy. Merging a deploy-triggering change holds at the approval gate. Eachdeploy-<app>.yamlruns an ungatednotifyjob before the gated job; it executespackages/cli/scripts/deploy-gate-notify.tsto post one Discord message, reading theDEPLOY_GATE_DISCORD_WEBHOOKrepository secret thatdeploy.yamlpasses explicitly to each callee (the job runs only onmain). A notify failure never fails or skips the deploy: the gated job depends onnotifybut ignores its result, and the script always exits 0 with a warning. All seven deploy workflows hold theirdeploy-<app>-<ref>concurrency group at workflow level (cancel-in-progress: false), so a queued run posts only when it is next in line for its own gate; thedeploy.yamlrouter has no top-level concurrency. Operator setup and rotation:docs/runbooks/deploy-gate-notifications.md. - Host-key pinning.
.github/known_hostspins both domain (unhashed) and IP (hashed) entries; CI connects with strict host-key checking against only that file. - Upstream pinning + verify-at-tag. The gateway daemon source is pinned in
apps/gateway/upstream.json; bumping it requires diffing the daemon’s required-secret contract against the compose wiring at the new tag before deploy. This is independent of the Fro Bot review Action SHA pinned in.github/workflows/fro-bot.yaml(see the “Two independentfro-bot/agentversion surfaces” gotcha indocs/solutions/best-practices/off-droplet-docker-image-build-gateway-deploy-2026-06-04.md). The dashboard image is pinned by tag and digest directly inapps/dashboard/docker-compose.yaml; Renovate tracksghcr.io/fro-bot/dashboardand opens PRs to bump the pin when a new release is published. The deploy pulls the compose-pinned digest — no CI image build step. The formerghcr.io/marcusrbrown/infra-dashboardimage is retired and can be deleted manually from the GitHub Container Registry once no references remain. - Executable conventions.
packages/cli/src/conventions.test.tsenforces several invariants above (SHA-pinned actions,.yamlextension, no stray Bash scripts, nossh-keyscanin workflows, nobundledDependencies, reusable-workflow permission parity betweendeploy.yamlrouter jobs and thedeploy-<app>.yamlcallees theyuses) as part of the test suite. - Releases. Changesets version
@marcusrbrown/infra; onlypackages/cli/src/user-facing changes warrant a changeset. Private workspace packages never enter a release:.changeset/config.jsonsetsprivatePackages: {tag: false, version: false}with an emptyignore,renovate-changesets.yaml’sexclude-patternsmust cover every private package directory, andpackages/cli/src/conventions.test.tsasserts@marcusrbrown/infrais the only non-private package and that no changeset mixes private and publishable names.release.yamlverifies every published package resolves on the npm registry (up to 13 attempts with exponential backoff from 3s to a 30s cap, ~285s total) before the run completes, and fails closed if it does not;release-alert.yamltriggers onworkflow_runcompletion of the Release workflow and opens or updates a singlerelease-publish-failure-labeled issue when it fails, so a broken publish can’t go unnoticed (seedocs/solutions/integration-issues/npm-publish-stacked-auth-failures-2026-08-23.md).
Where to Add New Code
Section titled “Where to Add New Code”Integration-level patterns; for the mechanical file layout see STRUCTURE.md.
- New deployable app → create
apps/<name>/mirroring the closest existing app (apps/cliproxy/for a Docker-Compose droplet app,apps/vpn/for a native-systemd AWS Lightsail app): Compose config or deploy script,src/deploy.ts,server/provision.ts(usingpackages/shared/server/droplet-helpers.tsfor SSH helpers, or@aws-sdk/client-lightsailfor Lightsail),src/host.ts. New provisioners useprovision.ts; the existing DigitalOcean apps keep theirprovision-droplet.tsfilenames. Add it topackage.jsonworkspaces, addprovision:<name>/deploy:<name>root scripts, add a CLI command group underpackages/cli/src/commands/<name>/, add a gateddeploy-<name>.yamlworkflow wired intodeploy.yaml’s paths-filter, create the<name>GitHub Environment, and addapps/<name>/AGENTS.md. - New operator-only provisioning tool (no deploy step) → mirror
apps/agent/:private: truepackage,server/provision.tsentry point, nosrc/deploy.ts, nosrc/host.ts, no deploy workflow, no GitHub Environment. Add onlyprovision:<name>to rootpackage.jsonscripts and, if operator interaction is needed, a CLI command group underpackages/cli/src/commands/<name>/. - New CLI command group → add
register<Name>Commandsand acommands/<name>/directory with per-action files and a barrelindex.ts; register it inpackages/cli/src/cli.ts. Expose a command over MCP only by adding it toMCP_ALLOWLIST, and only if it is read-only. - New shared provisioning helper → add it to
packages/shared/server/droplet-helpers.tswith a colocated test; consume it from each app’s provisioning script (provision-droplet.tsfor cliproxy/gateway/umami,provision.tsfor vpn). For CLIProxyAPI management-API primitives (auth headers, JSON request helpers, config parsing), add topackages/shared/cliproxy/management.tsinstead — it is the shared source of truth forapps/cliproxy,apps/broker, andpackages/cli/src/commands/cliproxy/. - New deploy workflow → copy an existing
deploy-<app>.yaml, keep the per-app Environment gate andpaths-filternegations, and wire it intodeploy.yaml.