Operator Surface Acceptance (Automated)
How an agent runs the operator surface acceptance checks with agent-browser. Run it after every paired gateway + dashboard operator-contract deploy, then post the results. operator-surface-acceptance.md is the full manual checklist; this runbook references its check numbers and does not repeat what each check expects. <expected> means the contract version reported by /operator/health. After a gateway release deploy, also run the gateway release gates.
Browser session
Section titled “Browser session”Every agent-browser command uses the same session:
agent-browser --session infra-operator-acceptance --restore --restore-save auto <command>- Session state lives in
~/.agent-browser/sessions/, outside the repo. Put no credentials in.envor in tracked files. --allowed-domainscannot be combined with--restore. Do not pass it. If the error keeps appearing after you drop the flag, the session daemon has cached it: runagent-browser --session infra-operator-acceptance close, then delete~/.agent-browser/infra-operator-acceptance.configand.target. The saved sign-in state insessions/is kept.- Never type credentials. Never print cookies, CSRF tokens, session IDs, or push endpoints. Keep them inside
evaland report only non-secret fields. - If the session lands on a GitHub login page, stop and ask the operator to sign in once in a headed window (see setup below).
One-time setup
Section titled “One-time setup”- Run
agent-browser --session infra-operator-acceptance --restore --restore-save auto --headed open https://dashboard.fro.bot/operator/auth/github/start. - The operator signs in to GitHub in that window. The browser returns to the dashboard signed in.
- Close the headed browser. Later runs reuse the saved state headless.
Session lifetime
Section titled “Session lifetime”The dashboard session lasts seven days. When it lapses, or after logout, opening /operator/auth/github/start signs back in without interaction as long as the GitHub cookie is still valid. When the GitHub cookie has also lapsed, the page lands on a GitHub login: stop and ask the operator.
Tool notes
Section titled “Tool notes”evaldoes not allow top-levelawait. Wrap code in an async IIFE:(async () => { … })(). Pipe multi-line code witheval --stdin.- Pass absolute paths to
screenshot. A relative path resolves against the browser daemon’s working directory, not your shell’s. - Refs (
@eN) change after a reload, and again whenever the run list refreshes, which can happen within seconds. Re-runsnapshot -ibefore using one, and act on it at once. - A run card expands and collapses on click, and a list refresh can reset it. Re-snapshot to confirm the card state before acting.
- Select the repository in the Launch panel before filling the prompt.
- Every command opens the browser if it is closed. Never poll or retry with
evalin a loop; each iteration relaunches Chrome. - Close the session when the pass ends.
Launch budget
Section titled “Launch budget”The gateway allows 3 launches per minute and 10 per hour per operator, in fixed windows. A refused launch shows “Too many launch requests” in the Launch panel (429 rate limited) and creates no run. A refusal can last until the hourly window ends, so retries do not help.
Run the checks that launch nothing first, then spend the nine launches below, at least 20 seconds apart. The limit is keyed to the operator, so manual launches and earlier passes in the same hour count too. The gateway release gates share the window and spend 5 launches (G1.10, G2b, G3.1, G5.1, G5.3). Start the second of the two sets at least 60 minutes after the first launch of the earlier set, or after a gateway restart, which clears the counters (G3.4 restarts the gateway). The nine launches plus one in reserve fit only in a fresh window. Launch against fro-bot/agent. One launch stays in reserve; do not retry a launch that created a run.
| Launch | Prompt | Covers |
|---|---|---|
| L1 | Inert prompt: Reply with only the word OK. <img src=x onerror=alert(1)> <b>bold tag</b> **markdown bold** [link](https://example.com) |
5, 6, 8, 16 step 1 |
| L2 | 8,000-unit prompt | 16 step 2 |
| L3 | Approval trigger, press “Once” | 9 steps 1, 2 |
| L4 | Approval trigger, press “Always” then “Confirm always” | 9 step 5 |
| L5 | Approval trigger, press “Reject” | 9 step 3; also shows a new run prompts again after “Always” |
| L6 | Question trigger, choose an option and “Submit answers” | 10 steps 1, 2 |
| L7 | Question trigger, press “Skip” | 10 step 3 |
| L8 | Question trigger, then cancel while the question is open | 7, 10 step 5 |
| L9 | Background-subagent drain trigger | 17 steps 1–5 |
- The approval, question and drain trigger prompts are in check 9, check 10 and check 17.
- Generate the 8,000-unit and 8,001-unit prompt files with the one-liner in check 16, then fill the field from the file:
fill @eN "$(cat /tmp/prompt-8000.txt)". - While an approval (L3–L5) or question (L6, L7) is pending, collapse the card and wait about 20 seconds for the list refresh to cover the collapsed-badge steps (9 step 4, 10 step 7). Expand it again before pressing a button.
- In L1, expand the card while the run is live for check 8. After it finishes, confirm the card has no Cancel control (check 6). For check 16 step 1, hook
window.alertwithevalbefore expanding, then confirm that the<img>and<b>text is literal,<strong>wraps “markdown bold”, the link shows its host, and no alert fired. - Cancel asks for confirmation: press “Cancel run”, then “Confirm cancel”.
L9 wait reason
Section titled “L9 wait reason”Run L9 after L8 has finished, and launch nothing else until L9 ends. The first reason can appear within about 15–20 seconds of launch, so start the observer at launch. awaiting-background-task lasts roughly 40–45 seconds and is the only reason that tells a background run from a plain one, because every run ends with about 5 seconds of confirming-completion. A stretch with no reason between the subagent finishing and confirmation starting is normal. Steps 1–3 must fit inside the awaiting-background-task window: have every command ready before you launch, and do not spend the window on setup.
Timing procedure:
-
Commands in one session run one at a time. An
evalthat waits blocks every later command, so record frames with a non-blocking observer: right after pressing Launch, run oneevalthat finds the new run, starts a stream reader that stores eachstatusframe with a timestamp onwindow.__obs.framesand eachoutputframe onwindow.__obs.output, and returns at once. The run can take a few seconds to appear, so the snippet retries for up to 10 seconds. DefinelaunchedAtinline as the time just before the press, and take it from the shell clock truncated to whole seconds. Readwindow.__obslater with a singleeval. Do not reload before step 5, because a reload discards the observer.;(async () => {const since = Date.parse(launchedAt)let runfor (let i = 0; i < 10 && !run; i++) {const {runs} = await fetch("/operator/runs", {credentials: "same-origin"}).then(res => res.json())run = runs.find(r => Date.parse(r.createdAt) >= since)if (!run) await new Promise(resolve => setTimeout(resolve, 1000))}if (!run) return nullconst obs = (window.__obs = {runId: run.runId, frames: [], output: [], end: null})void (async () => {const res = await fetch(`/operator/runs/${run.runId}/stream`, {credentials: "same-origin",headers: {accept: "text/event-stream"},})const reader = res.body.getReader()const decoder = new TextDecoder()let buf = ""for (;;) {const {value, done} = await reader.read()if (done) breakbuf += decoder.decode(value, {stream: true})const frames = buf.split("\n\n")buf = frames.pop()for (const frame of frames) {const event = (frame.match(/^event: (.*)$/m) || [])[1]if (event !== "status" && event !== "output") continueconst data = JSON.parse((frame.match(/^data: (.*)$/m) || [])[1] || "null")const at = new Date().toISOString()if (event === "output")obs.output.push({at, seq: data?.seq, final: data?.final === true, text: data?.text ?? ""})else obs.frames.push({at, status: data?.status, waitReason: data?.waitReason ?? null})}}obs.end = "closed"})().catch(() => {obs.end = "error"})return run.runId})() -
A new run’s card opens expanded. Confirm that in a snapshot and leave it expanded.
-
Right after the observer returns, use short
wait --textcalls instead of polling:wait --text "Waiting on a background task"times out after about 25 seconds, so repeat it once, until about 60 seconds after launch. The “Confirming it has finished” line shows for only about 5 seconds near the end; the observer’s frames are the record for it. -
Do not wrap agent-browser commands in
timeout. Killing the wrapper can kill the browser daemon, which discards the page and the observer. -
Take timings from the observer’s frame timestamps and the run’s
createdAt. Do not use theupdatedAtin/operator/runs: it does not change when the reason changes.
-
Screenshot and snapshot the reason line under the repo name as soon as a
wait --textcall sees it. If no line shows within about 45 seconds, look forbackground dispatch observedinbunx @marcusrbrown/infra gateway logs gateway; the line carries the session ID, not the run ID, so match it to the launch by order. Without that line, record check 17 asnot triggeredwith the run ID and skip to step 5. With that line but no label yet, keep waiting another 30 seconds before recording a failure. -
Collapse the card, wait about 20 seconds for the list refresh, and screenshot it. While it is collapsed, read the run’s current status once from page context. Define
runIdinline. The gateway replays the latest status to a new stream, so the firststatusframe is the current one:;(async () => {const ctl = new AbortController()const timer = setTimeout(() => ctl.abort(), 10000)try {const res = await fetch(`/operator/runs/${runId}/stream`, {credentials: "same-origin",signal: ctl.signal,headers: {accept: "text/event-stream"},})const reader = res.body.getReader()const decoder = new TextDecoder()let buf = ""for (;;) {const {value, done} = await reader.read()if (done) breakbuf += decoder.decode(value, {stream: true})for (const frame of buf.split("\n\n").slice(0, -1)) {if ((frame.match(/^event: (.*)$/m) || [])[1] !== "status") continueconst data = JSON.parse((frame.match(/^data: (.*)$/m) || [])[1] || "null")ctl.abort()return {status: data?.status, waitReason: data?.waitReason ?? null}}}} catch {} finally {clearTimeout(timer)}return {status: null, waitReason: null}})()Record
statusandwaitReason. Expectrunningwithawaiting-background-task;nullmeans the quiet stretch has begun, so record that and move on. -
Expand the card again and
wait --textfor the current reason line. Screenshot it. If the quiet stretch has begun, no line shows; record that instead of a failure. -
Not automatable (see Not automatable). Record
not automated. -
Wait for the terminal status. Confirm the card is expanded (its text includes “Raw text”) and the line is gone, reload, and confirm the card is expanded again and the line stays gone.
Cancel safety
Section titled “Cancel safety”Cancel only a run whose prompt writes nothing: L8, and the sleep 120 runs C1, C1-D and C2 of Gate 2. A cancelled session can keep writing to the shared checkout after the cancel returns. Leftover files make every later launch on that repo fail with “Checkout refused: uncommitted changes” until an operator cleans the checkout. L8 asks a question and the sleep 120 prompt only waits, so neither writes anything. Never cancel any other run: not one whose prompt creates or edits files, and not a Gate 5 run, whose prompt runs chmod and whose marker could still change after the cancel returns.
Checks that launch nothing
Section titled “Checks that launch nothing”Run these first, in this order.
-
Check 1, health.
curl -s https://dashboard.fro.bot/operator/health. Record the JSON and use itscontractVersionas<expected>. Then create the pass’s evidence directory, once, and keep its path for the whole pass (see Afterwards, step 4):EVID=/tmp/operator-acceptance-<expected>-$(date -u +%Y%m%dT%H%M%SZ); mkdir -p "$EVID". -
Check 2, sign-in. Open
https://dashboard.fro.bot/. Confirm the operator UI shows and/operator/sessionreturns 200 with the operator login. Do not run the OAuth start flow when the session is valid. -
Check 4, contract handshake. Pick a run ID from the
/operator/runsresponse. Read the first SSE event from page context and stop:;(async () => {const ctl = new AbortController()const res = await fetch(`/operator/runs/${runId}/stream`, {credentials: "same-origin",signal: ctl.signal,headers: {accept: "text/event-stream"},})const reader = res.body.getReader()const decoder = new TextDecoder()let buf = ""while (!buf.includes("\n\n")) {const {value, done} = await reader.read()if (done) breakbuf += decoder.decode(value, {stream: true})}ctl.abort()const frame = buf.split("\n\n")[0]const data = JSON.parse((frame.match(/^data: (.*)$/m) || [])[1] || "null")return {status: res.status,event: (frame.match(/^event: (.*)$/m) || [])[1],contractVersion: data?.contractVersion,}})()Define
runIdinline. Record the event name andcontractVersion; do not echo the full frame. -
Launch lists bound repos and Recent Runs is non-empty (check 5 step 4). Open the Launch panel and record the repo count. Close the panel without launching.
-
Check 11, expired output. Use a run that finished before the latest gateway restart. Load
https://dashboard.fro.bot/#<runId>, reload, and wait for the card to open: the run list renders after the reload, so a read right after it finds no card. Without a reload, the hash change does not open the card.Terminal window agent-browser --session infra-operator-acceptance --restore --restore-save auto wait '[data-run-id="<runId>"][data-expanded="true"]'agent-browser --session infra-operator-acceptance --restore --restore-save auto wait --text 'Output no longer available.'Then read the card text. Each wait ends at the tool’s default timeout and fails when it expires. A card that never opens, or a card that sits on “Connecting…” or shows no status, is
fail. Before recording that, confirm the run is still among the newest 100 that/operator/runslists. -
Check 12, step 3. Read the
pushfield of the/operator/sessionresponse and record it. The delivery steps are not automatable. -
Check 13, simulated. Load
https://dashboard.fro.bot/#<runId>for a run in the list, reload, wait for the card as in check 11 (wait '[data-run-id="<runId>"][data-expanded="true"]'), and confirm that run opens and the “That run isn’t in your recent runs.” notice is absent. If the wait expires, read the notice text before recording anything: a notice meansfail. Record a pass assimulated. -
Check 14, session durability. Read
expiresAtfrom/operator/session. The sign-in time isexpiresAtminus seven days. The latest gateway restart is the gateway container’sState.StartedAt, which Docker updates on every restart. Read it with a read-onlydocker inspect(run this with the helpers block sourced, as for the gates):Terminal window gw "docker inspect --format '{{.State.StartedAt}}' fro-bot-gateway-1"The result is meaningful only when the sign-in time precedes that time; then a 200 now proves the session survived. When the pass restarts the gateway (G3.4), read both again after the restart and judge against the new time. When sign-in follows the restart, record
not demonstrablewith theexpiresAtvalue. A 401 when sign-in precedes the restart is a failure. -
Check 16, step 3. Read the run-record count for
$OWNER_REPOin the state bucket first, using the S3 part of the helpers block:Terminal window before="$(runcount "$OWNER_REPO") $(runkeys "$OWNER_REPO" 1)"; echo "$before"Open the Launch panel with
$OWNER_REPOselected and fill the 8,001-unit prompt. The UI evidence is a Launch button that is disabled and the inline “Prompt is too long” status: read both from a freshsnapshot -i. The browser tool refuses to click a disabled button, so do not try./operator/runscannot show that no run was created: it lists the newest 100 runs only (MAX_RUNS_PER_LISTINGin the gateway’spackages/gateway/src/web/operator/runs-route.ts), so a new run would push out the oldest and the length would stay the same. Read the state-bucket count again and compare:Terminal window after="$(runcount "$OWNER_REPO") $(runkeys "$OWNER_REPO" 1)"[ "$before" = "$after" ] && echo "NO RUN CREATED" || echo "RUN RECORDS CHANGED: $before / $after"This check launches nothing. If the button is enabled and a launch creates a run, record check 16 step 3 as
fail(an enabled button is a regression), record the run ID, count it against the launch window, and let the run finish: its prompt asks only for the wordOK, and the runs named in Cancel safety are the only ones the pass cancels.
Not automatable
Section titled “Not automatable”Record these as skip or not automated:
- Check 3: there is one operator and no second account.
skip. - Check 10 step 4 (Discord settlement) and step 6 (question push).
not automated. - Check 12 delivery steps.
not automated. - Check 13 notification taps. The deep-link simulation above covers only that
/#<runId>opens the run. - Check 16 step 4 has no safe trigger.
skip. - Check 17 step 4 (reconnect). Browser offline mode does not drop an open SSE stream, so the card never reconnects. The behaviour is covered by fro-bot/dashboard
web/src/operator/wait-reason-label.test.ts(a reconnecting connection clears the label, and a replayed running frame shows it again).not automated. - Check 17 step 6: “Finishing its reply” and “Agent still working” have no safe trigger. Record them only when they occur.
Logout
Section titled “Logout”Run check 15 last, after every launch check and after the gateway release gates when they run in the same sitting. Run the snippets from the manual runbook as page-context eval code that keeps csrfToken in a variable and returns only status codes. /operator/session returns 401 afterwards. Logout deactivates the operator’s push subscriptions on every device, so skip check 15 in a pass that is followed by a push check, and record it as not run.
Then open https://dashboard.fro.bot/operator/auth/github/start so the saved session is signed in for the next run. If that page lands on a GitHub login, stop and ask the operator.
Afterwards
Section titled “Afterwards”-
Collect every run ID the pass created from the
/operator/runsresponse. Compare the list with the one read before the first launch. -
Optionally confirm that no workspace checkout is dirty. This is read-only. Run it through
wsx(see helpers) against the checkout inside the workspace container. The checkouts live on thefro-bot_workspace-reposvolume, mounted at/workspace/repos, and the gateway lowercases the owner and repo to form a checkout’s path,/workspace/repos/<owner>/<repo>.$OWNER_REPOis already lowercase forfro-bot/agent.HOMEis set because an exec session’sHOMEis not readable by uid 10001, and git warns about that on every call:Terminal window wsx <<EOFHOME=/home/opencode git --no-optional-locks -c safe.directory='*' -C /workspace/repos/$OWNER_REPO status --porcelainEOFEmpty output means clean. A
fatal:line means the path does not exist, which is not a clean checkout: check the spelling of$OWNER_REPO. Any other output means the next launch on that repo fails with “Checkout refused: uncommitted changes”; report it to the operator. The gateway log may not show the refusal, so the checkout status is the reliable signal.Clean up only with the operator’s approval, one path at a time. Before removing a file, confirm three things: it is the only dirty path, nothing holds it open (
fuser/lsof), and its size and mtime are stable. Remove it as root, because the workspace UID 10001 has no host account andsudo -u '#10001'fails. Then read backgit status --porcelainand run oneReply with only the word OK.launch on that repo. That launch uses the reserve launch. -
Confirm the
fro-bot/.githubdatabranch survived a promotion. Run it only when aMerge Data Branchpromotion ran during the pass or the operator dispatched one for this purpose; otherwise recordnot demonstrated. Both commands are read-only and need only theghlogin the gateway gates already use. Read the branch tip before the promotion starts (at the start of the pass, alongside the run list in step 1, or just before the dispatch) and again after it finishes:Terminal window gh api repos/fro-bot/.github/git/ref/heads/data --jq .object.shaThen compare the two, old tip first:
Terminal window gh api repos/fro-bot/.github/compare/<old-sha>...<new-sha> --jq .statusPass:
ahead, which means the old tip is an ancestor of the new one.identicalmeans the branch did not move, so the check is not demonstrated: record that. Failure:diverged,behindor a404, which meansdatawas replaced or reset; report it to the operator (see check 5 step 4 for the reseed). Record both short SHAs and the status with the results. -
Save evidence under
$EVID, the per-pass directory/tmp/operator-acceptance-<expected>-<UTC date-time>/made at check 1. A fixed<expected>directory is reused by every pass at the same contract version and the passes mix; the date-time suffix keeps them apart. Save a screenshot per check namedNN-name.png, and secret-free JSON for health, thereadyframe,/operator/runs, and/operator/session(login,expiresAt, andpushonly). -
Before recording a result, read the saved JSON back from disk. A subagent’s summary is not evidence.
-
Record results as Record results describes. Add a result column to the per-check list with one of
pass,fail,skip,not automated,simulated, ornot run, and cite the run ID for checks 5–11, 13 and 17.
Gateway release gates (agent-run)
Section titled “Gateway release gates (agent-run)”How an agent runs the gateway release gates after a Fro Bot gateway deploy moves to a new upstream ref (apps/gateway/upstream.json). The gates cover stale-credential removal on OpenCode V2, /fro-bot cancel, ownership persistence across a gateway restart, approval attribution, and that a rejected command does not run. The agent drives the droplet over SSH, reads run state from S3, launches and cancels runs from the dashboard session, and reads Discord threads over REST.
Each check is one of two kinds:
- Automated. Runs on a dashboard-launched run and needs no Discord action. Every release runs these: P0–P2, G1, G2b.1, G3 and G5.
- Discord pass. Needs a Discord-started run or a human Discord action. The agent cannot act in Discord as the operator: the gateway drops messages written by bots, and Discord accepts slash commands and button clicks only from users. These checks form one Discord pass, run in one sitting, and only when it is triggered. Its operator steps are marked Operator action.
A release is a new upstream ref. A redeploy at the same ref runs no gates: its new image digests are covered by the P0 checks of the next release. A ref that was deployed without a gate record still owes its gates: a ref with no Live acceptance: agent <REF> comment on the release tracker was never gated, whatever has been redeployed since. That pass runs on the digest pair running when it is made, and the P0 row records that pair.
- G1, G2b.1, G3 and G5 use the dashboard session from Browser session. Sign in first. When both passes run in one sitting, finish these gates before Logout.
- The automated gates spend 5 launches (G1.10, G2b.1, G3.1, G5.1, G5.3) from the launch budget. The gateway gates and the browser pass share one per-operator window of 10 launches per hour. Start the second set at least 60 minutes after the first launch of the earlier set, or after a gateway restart, which clears the counters (G3.4 restarts the gateway).
- Every command below is a pattern: copy it and fill the placeholders. Add no script files to the repo.
Inputs
Section titled “Inputs”| Variable | Value |
|---|---|
REPO |
Absolute path of the repo checkout. Run from it so Bun finds the root .env. |
WORK |
mktemp -d for this pass. Holds the SSH key and the SSH control socket. |
OUT |
Evidence directory for this pass: /tmp/gateway-acceptance-$REF-<UTC date-time>. |
REF |
Upstream ref under test, as pinned in apps/gateway/upstream.json. |
OCV |
OpenCode V2 version the deploying build-images job built (--build-arg OPENCODE_VERSION). |
TAG |
ACCEPT-$REF. Prefix for every plant, probe and marker. |
OWNER_REPO |
<owner>/<repo> of the test repo the gates launch against. |
OP_LOGIN |
Operator’s GitHub login. |
CHANNEL |
Discord pass only. ID of the Discord channel bound to $OWNER_REPO. Operator supplies it. |
OP_ID |
Discord pass only. Operator’s Discord user ID. Operator supplies it. |
TRIGGER_ROLE |
Discord pass only. Trigger role ID (G2a). Operator supplies it. |
Secrets come from the repo-root .env through envval below: GATEWAY_HOST, GATEWAY_SSH_KEY, S3_BUCKET, S3_REGION, optional S3_ENDPOINT, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and, for the Discord pass only, DISCORD_TOKEN and DISCORD_GUILD_ID. The AWS_* keys can list and get in the state bucket. Never print, log, or save a value from .env.
- Run the patterns under
bash.zshdoes not word-split unquoted$(…). Tool calls may not share shell state: repeat the helpers block at the head of each call, with the sameWORKandOUT(the block prints both). The first call makesOUT; its date-time suffix keeps a pass at a ref that was gated before from mixing its files with the earlier pass’s. - Droplet commands run as
rootover SSH with the pinned host key. Many short calls trip SSH rate limits, so every call goes through one multiplexed connection (gw). - On the droplet host, do not run
rm,mv,cp,mknod,chmod, orchown. - Do not redirect any remote command to
/dev/*, including2>/dev/null. Use2>&1. - Do not run
docker compose up,down, orpull. - Restart only
workspace(G1) andgateway(G3), and only afteridle_checksucceeds. - A restart has no fixed downtime.
wait_healthychecks the container’s health every 5 seconds for 6 minutes and printshealth: not healthy after 6 minif it never reportshealthy. A restart that is not healthy by then does not pass its step. - Container-side cleanup is allowed only for the gate’s own plants, through
$DC exec -T -u 10001 workspace. - Gateway and workspace logs are unredacted. Filter with
grepand save only the matched lines. - A secret in argv is a leak: the SSH key lives in a mode 600 file, and the Discord token goes to
curlon stdin. - End of pass:
gw 'stat -c "%F %t,%T %a" /dev/null'must printcharacter special file 1,3 666. Anything else: stop and report it to the operator. Do not repair it.
Helpers
Section titled “Helpers”REPO=<repo-root>; cd "$REPO"WORK=${WORK:-$(mktemp -d)}OUT=${OUT:-/tmp/gateway-acceptance-${REF:?set REF}-$(date -u +%Y%m%dT%H%M%SZ)}; mkdir -p "$OUT"; echo "WORK=$WORK OUT=$OUT"envval() { (cd "$REPO" && NAME=$1 bun -e 'process.stdout.write(process.env[process.env.NAME] ?? "")'); }
GATEWAY_HOST=$(envval GATEWAY_HOST)[[ $GATEWAY_HOST =~ ^[A-Za-z0-9][A-Za-z0-9.-]*$ ]] || { echo "invalid GATEWAY_HOST"; false; } # a '-' prefix would be read as an ssh option[ -s "$WORK/key" ] || (umask 077; { envval GATEWAY_SSH_KEY; echo; } > "$WORK/key")chmod 600 "$WORK/key"
gw() { ssh -i "$WORK/key" -o BatchMode=yes -o IdentitiesOnly=yes -o StrictHostKeyChecking=yes \ -o UserKnownHostsFile="$REPO/.github/known_hosts" \ -o ControlMaster=auto -o ControlPersist=600 -o ControlPath="$WORK/cm" "root@$GATEWAY_HOST" "$@"}DC='docker compose --project-directory /opt/gateway/deploy'wsx() { gw "$DC exec -T -u 10001 workspace sh -s"; } # container shell as uid 10001; the script comes on stdinglog() { gw "docker logs --timestamps --since $1 fro-bot-gateway-1 2>&1"; }
export AWS_ACCESS_KEY_ID=$(envval AWS_ACCESS_KEY_ID) AWS_SECRET_ACCESS_KEY=$(envval AWS_SECRET_ACCESS_KEY) AWS_DEFAULT_REGION=$(envval S3_REGION)S3_BUCKET=$(envval S3_BUCKET); S3_ENDPOINT=$(envval S3_ENDPOINT); [ -z "$S3_ENDPOINT" ] || export AWS_ENDPOINT_URL_S3=$S3_ENDPOINTBASE=fro-bot-state/discord-gateways3cat() { aws s3 cp "s3://$S3_BUCKET/$1" -; }etag() { aws s3api head-object --bucket "$S3_BUCKET" --key "$1" --query ETag --output text; }runkeys() { aws s3api list-objects-v2 --bucket "$S3_BUCKET" --prefix "$BASE/$1/runs/" --query "sort_by(Contents,&LastModified)[-${2:-3}:].Key" --output text | tr '\t' '\n'; }runcount() { aws s3api list-objects-v2 --bucket "$S3_BUCKET" --prefix "$BASE/$1/runs/" --query 'length(Contents || `[]`)' --output json; } # json: the query runs once over every page; text prints one count per page
dget() { printf 'header = "Authorization: Bot %s"\n' "$(envval DISCORD_TOKEN)" | curl -sS -K - "https://discord.com/api/v10$1"; }thread() { dget "/channels/$1/messages?limit=50" | jq -r 'reverse[] | [.timestamp[11:23], (.edited_timestamp // "-"), "mentions=\(.mentions|length)", "roles=\(.mention_roles|length)", "everyone=\(.mention_everyone)", (.content|tojson), ([.embeds[]|{title,description}]|tojson)] | @tsv'; }
wait_healthy() { # $1 = container name for i in $(seq 72); do [ "$(gw "docker inspect --format '{{.State.Health.Status}}' $1")" = healthy ] && { echo "health: healthy $(date -u +%FT%TZ)"; return 0; } sleep 5 done echo "health: not healthy after 6 min"; return 1}
wait_terminal() { # $1 = run key, $2 = tries, $3 = seconds between tries, $4 = jq filter printed on each read (default {phase}) local i rec phase last= f=${4:-'{phase}'} for i in $(seq "$2"); do rec=$(s3cat "$1") || rec= phase=$(printf '%s' "$rec" | jq -r '.phase // empty') || phase= if [ -z "$phase" ]; then echo "$(date -u +%FT%TZ) unreadable, still waiting"; else last=$phase; printf '%s\n' "$rec" | jq -c "$f"; fi case $phase in COMPLETED|FAILED|CANCELLED) return 0 ;; esac # the run state machine's three terminal phases sleep "$3" done echo "not terminal after $2 tries (last phase read: ${last:-none})"; return 1}
readyz() { wsx <<'EOF'curl -sS -w '\nhttp=%{http_code}\n' http://127.0.0.1:9100/readyzEOF}
idle_check() { # succeeds only when every run record of every bound repo is terminal; any listing, read or parse error fails local active=0 bad=0 repo key phase b bkeys rkeys repos bkeys=$(aws s3api list-objects-v2 --bucket "$S3_BUCKET" --prefix "$BASE/_/_/bindings/by-channel/" --query 'Contents[].Key' --output text) \ || { echo "idle-check: binding listing failed"; return 1; } [ -n "$bkeys" ] && [ "$bkeys" != None ] || { echo "idle-check: no bindings listed"; return 1; } repos=$(for b in $bkeys; do s3cat "$b" | jq -er '"\(.owner)/\(.repo)"' || echo "ERR:$b"; done | sort -u) case $repos in *ERR:*) echo "idle-check: unreadable binding"; return 1 ;; esac for repo in $repos; do rkeys=$(aws s3api list-objects-v2 --bucket "$S3_BUCKET" --prefix "$BASE/$repo/runs/" --query 'Contents[].Key' --output text) \ || { echo "idle-check: run listing failed for $repo"; return 1; } [ "$rkeys" = None ] && continue for key in $rkeys; do phase=$(s3cat "$key" | jq -er .phase) || { echo "unreadable: $key"; bad=$((bad+1)); continue; } case $phase in COMPLETED|FAILED|CANCELLED) ;; *) active=$((active+1)); echo "active: $key $phase" ;; esac done done echo "idle-check active=$active unreadable=$bad $(date -u +%FT%TZ)"; [ "$active" -eq 0 ] && [ "$bad" -eq 0 ]}- Channel bindings are
$BASE/_/_/bindings/by-channel/<channelId>.jsonwith{owner, repo}. Run records are$BASE/<owner>/<repo>/runs/<runId>.jsonwithrun_id,surface,thread_id,phase,started_at,last_heartbeat, anddetails.{rootSessionId, ownedSessionIds, cancelledBy}. idle_checkreads every run record of every bound repo; the CLI paginates the listings. Run it immediately before each restart. Any phase other thanCOMPLETED,FAILEDorCANCELLEDcounts as active, and a listing, read or parse failure fails the check. A bound repo with no run records lists asNoneand counts as zero runs.runkeys(newest N records) is for finding the run under test. It is not an idle check.runcountcounts every run record of one repo.wait_terminalwaits until the record readsCOMPLETED,FAILEDorCANCELLED, the run state machine’s terminal phases, and returns 0. Any other phase keeps it waiting, and so does an empty or unreadable read (a faileds3cat, an empty body, bad JSON): a transient read error is never a verdict. After the last try it printsnot terminal after <n> trieswith the last phase it could read, ornone, and returns 1. Each gate says what to record then. It waits; it does not judge the phase. Read the phase afterwards, becauseFAILEDandCANCELLEDalso return 0.- The first message in a run thread carries one mention. Every other message must carry none.
When the Discord pass runs
Section titled “When the Discord pass runs”Decide this once per release, before P0, and record the deciding item. The last pass is recorded on the Discord pass: line of the Results comment in the release tracker (currently fro-bot/.github#3512). If the tracker closes, the operator carries the newest Discord pass: line into the first results comment of the successor tracker, and TRACKER below changes.
The pass runs when any of these holds, checked in this order. Record the first one that holds:
never passed: noDiscord pass:line yields an anchor.trigger check failed: a query errors; the anchor is not an ancestor of$REF; or the file list of the compare is empty, has 300 or more files, or lacks thepatchof a pin file.stale (<n> refs):<n>is 3 or more.- A checklist item.
none: nothing holds. The pass does not run.
1. Anchor. The anchor is the ref of the newest PASS. A FAIL or INCOMPLETE pass, and a not run line, leave the anchor where it was, so the next release diffs from the last PASS. The tracker has more than 200 comments: without --paginate, gh api returns the first page only. Only comments from the owner, members and collaborators count.
set -o pipefailTRACKER=repos/fro-bot/.github/issues/3512LAST=$(gh api --paginate "$TRACKER/comments?per_page=100" --jq '.[] | select(.author_association | IN("OWNER","MEMBER","COLLABORATOR")) | .body | split("\r?\n"; null)[] | select(test("^Discord pass: (PASS on|[A-Z]+ on|not run)"))' | tail -n 1) \ || { echo "trigger check failed: tracker query"; false; }ANCHOR=$(printf '%s\n' "$LAST" | sed -nE 's/^Discord pass: (PASS on|[A-Z]+ on [^;]*; last passed on|not run[^;]*; last passed on) (v[0-9]+\.[0-9]+\.[0-9]+).*/\2/p')echo "anchor: ${ANCHOR:-never passed}"2. Staleness. Count the fro-bot/agent releases after the anchor, up to and including $REF, in semver order. Undeployed tags count too. Non-semver tags (the *-harness.* builds) are filtered out. An anchor or $REF missing from the list is an error.
count_refs() { # $1 = anchor, $2 = REF gh api --paginate repos/fro-bot/agent/releases --jq '.[] | select(.draft | not) | .tag_name | select(test("^v[0-9]+\\.[0-9]+\\.[0-9]+$"))' \ | jq -Rn --arg a "$1" --arg r "$2" '[inputs | {t: ., k: (ltrimstr("v") | split(".") | map(tonumber))}] | sort_by(.k) | map(.t) | (index($a)) as $i | (index($r)) as $j | if $i == null or $j == null or $j < $i then error("anchor or ref not in release list") else $j - $i end'}count_refs "$ANCHOR" "$REF"3. Diff. Read the compare from the anchor to $REF. The compare endpoint lists changed files on its first page only, capped at 300 for the whole comparison, so --paginate adds none. A list of 300 means it may be truncated: that is trigger check failed. The first call must print ahead. diverged means the anchor is not an ancestor, and identical means a same-ref redeploy: run no gates. Then check the file list against the checklist. The patch of a pin file can be absent for a large diff; that is unreadable and requires the pass.
gh api "repos/fro-bot/agent/compare/$ANCHOR...$REF" --jq '"\(.status) ahead_by=\(.ahead_by) commits=\(.total_commits)"'gh api "repos/fro-bot/agent/compare/$ANCHOR...$REF" --jq '[.files[] | {filename, previous_filename, status, patch}]' > "$WORK/compare.json"jq 'length | if . == 0 or . >= 300 then "trigger check failed: \(.) files" else "files: \(.)" end' "$WORK/compare.json"
# globs: prints each matching pathjq -r --arg re '^(packages/gateway/src/(program\.ts|config\.ts|(discord|bindings|execute|approvals|operator-contract|web/operator)/.*)|packages/runtime/src/(coordination|agent)/.*)$' \ '.[] | select([.filename, (.previous_filename // empty)] | any(test($re))) | .filename' "$WORK/compare.json"
# pin fields: prints each changed pin line, or "unreadable"jq -r ' def pin(f; re): .[] | select(.filename == f) | if .patch == null then "unreadable: \(f)" else (.patch | split("\n")[] | select(test(re)) | "\(f): \(.[0:60])") end; pin("packages/gateway/package.json"; "^[-+]\\s*\"discord\\.js\"\\s*:"), pin("packages/runtime/package.json"; "^[-+]\\s*\"@opencode-ai/sdk\"\\s*:"), pin("package.json"; "^[-+]\\s*\"@opencode-ai/sdk\"\\s*:"), pin("packages/runtime/src/shared/opencode-v2-pin.json"; "^[-+]\\s*\"version\"\\s*:"), pin("bun.lock"; "^[-+]\\s*\"(discord\\.js|@opencode-ai/sdk)\"\\s*:")' "$WORK/compare.json"4. Record. The deciding item goes into the Results Discord pass: line. When no item holds, the Discord checks are reported as not run with the ref of the anchor. When an item holds and the pass is not run in this sitting, the checks are reported as not run too, and the line says the pass is due.
Discord pass checklist
Section titled “Discord pass checklist”A match on any line requires the pass. Test files under these paths count.
| Item | Match in the compare file list |
|---|---|
| Mention and interaction intake | packages/gateway/src/program.ts |
| Gateway config (trigger role, binder allowlist) | packages/gateway/src/config.ts |
| Discord handling | packages/gateway/src/discord/**, packages/gateway/src/bindings/** |
| Shared run, cancel, approval and question code | packages/gateway/src/{execute,approvals,web/operator}/**, packages/gateway/src/operator-contract/** |
| Runtime run state and agent code | packages/runtime/src/{coordination,agent}/** |
| discord.js | the discord.js version line in packages/gateway/package.json, or in bun.lock |
| OpenCode | the @opencode-ai/sdk version line in packages/runtime/package.json, root package.json or bun.lock; the version line in packages/runtime/src/shared/opencode-v2-pin.json |
Other edits to bun.lock, package.json and the Dockerfile do not match. Name the matching path or pin line as the trigger.
Dashboard launches
Section titled “Dashboard launches”G1.10, G2b.1, G3.1, G5.1 and G5.3 each launch one run from the dashboard session. Each launch follows the Launch budget spacing. The prompt omits the @fro-bot mention.
-
Take
launchedAtfrom the shell clock, truncated to whole seconds:LAUNCHED_AT=$(date -u +%FT%TZ). -
Launch guard. In the Launch panel, select
$OWNER_REPO, then take a freshsnapshot -iand read the selected repository. Abort the gate, launch nothing, and recordblockedunless it equals$OWNER_REPO. Take the refs for the repo picker, the prompt box and Launch from that same snapshot and act on them at once: the run list refreshes within seconds and renumbers refs, so a ref from an older snapshot can point at the wrong element. After any delay, re-snapshot and repeat the guard. -
Fill the prompt and press Launch. For G1.10, start the observer in the next command, with
launchedAtdefined inline. A refusal (“Too many launch requests”) created no run: recordblockedand wait for the window. -
Confirm the record is under
$OWNER_REPO. Expectsurfacewebandstarted_atat or after$LAUNCHED_AT. No such record means the run landed elsewhere: stop the gates and report it. Never retry a launch that created a run.Terminal window RUNKEY=$(runkeys "$OWNER_REPO" 1)s3cat "$RUNKEY" | jq -c '{run_id, surface, phase, started_at}'
Preconditions
Section titled “Preconditions”| ID | Check | Pass |
|---|---|---|
| P0 | Running image digests equal the digests the deploying build-images job pushed, and no gateway deploy is in flight. |
Both digests match. |
| P1 | The workspace runs OpenCode V2 at $OCV. |
WORKSPACE_IMAGE_OPENCODE_VERSION and /readyz opencodeVersion both equal $OCV. |
| P2 | The box is idle before every restart. | idle_check prints active=0 and exits 0. |
P0. A gateway deploy approved during acceptance recreates the containers between gates. Even when the upstream ref stays the same, the new images have different digests: any change under the gateway path filter triggers a rebuild, and Alpine packages float. So a pass is valid only for one digest pair.
-
List gateway deploys that are waiting, queued or running. Each list must be
[]. If one is not, tell the operator not to approve any gateway deploy gate until acceptance ends, and note the run IDs.Terminal window for wf in deploy.yaml deploy-gateway.yaml; do for st in waiting queued in_progress; doecho "$wf $st"; gh run list --workflow "$wf" --status "$st" --json databaseId,displayTitle,createdAtdone; done -
Find the run that deployed the ref: the newest
deploy.yamlordeploy-gateway.yamlrun whose gateway deploy job concludedsuccess. The run’s own conclusion does not matter: a run that failed on another job (the VPN deploy, for example) still deployed the gateway. Do not filter the list by--status success, which hides such a run. The job is nameddeploy-gateway / deploy-gatewayin adeploy.yamlrun anddeploy-gatewayin adeploy-gateway.yamlrun; match it by that ending, not by “gateway” anywhere in the name, because thebuild-images,scan-imagesandnotifyjobs of adeploy.yamlrun carry thedeploy-gateway /prefix and can succeed when the deploy did not. A run whose gateway deploy job wasskippeddeployed nothing.Terminal window for wf in deploy.yaml deploy-gateway.yaml; dogh run list --workflow "$wf" --limit 20 --json databaseId,createdAt --jq '.[] | [.createdAt, .databaseId] | @tsv'done | sort -r | while read -r at id; doc=$(gh run view "$id" --json jobs --jq '[.jobs[] | select(.name | test("deploy-gateway$")) | .conclusion] | first // "none"')echo "$id $at gateway-deploy=$c"[ "$c" = success ] && breakdoneThe last line printed is the deploying run,
<runId>below. If no line endsgateway-deploy=success, no deploy of the gateway is on record in the last 20 runs of each workflow: stop and report. -
Read the pushed digests. Expect one
infra-gatewaypush and oneinfra-workspacepush. Each push prints two lines, the push and then the same line with a duration (… 2.6s done): read the digest from the first, which ends@sha256:<digest>.Terminal window gh run view <runId> --log | grep 'pushing manifest for' -
Read the running digests and compare them with step 3.
Terminal window gw sh -s <<'EOF'for s in gateway workspace; doecho "$s $(docker inspect --format '{{json .RepoDigests}}' "$(docker inspect --format '{{.Image}}' "$(docker compose --project-directory /opt/gateway/deploy ps -q $s)")")"doneEOF
Repeat step 4 before every gate and compare it with the digests recorded for the pass. A change means a deploy landed: match the new digests against the build-images output of the run that deployed them, and re-run any gate whose result spans the change.
P1.
gw "$DC exec -T workspace printenv WORKSPACE_IMAGE_OPENCODE_VERSION"readyzExpect $OCV, then http=200 with {"ready":true,"opencode":"ready","opencodeVersion":"<OCV>"}.
P2. Run idle_check immediately before each restart. Do not restart when it exits non-zero. Wait and re-check.
Gate 1: stale credential removal on V2
Section titled “Gate 1: stale credential removal on V2”On V2 the entrypoint removes a stale auth.json at /home/opencode/.local/share/opencode/auth.json, including a symlink, without following the link. Provider credentials still come from the root-only secret. A docker compose exec session has HOME=/var/lib/workspace-agent/home, so always use the absolute path. Every G1 check is automated.
| ID | Check | Pass |
|---|---|---|
| G1.1 | Exec identity | id -u prints 10001. |
| G1.2 | Plant a regular file with a fake key | Listing shows -rw------- owned by opencode. |
| G1.3 | Restart workspace, wait healthy |
health: healthy. |
| G1.4 | File removed | ABSENT and NOT-SYMLINK. |
| G1.5 | Entrypoint log since the restart | opencode config: model=set, provider-overlay=merged-into-config-file; auth: present (…) (the text in parentheses differs between agent releases); opencode-server-v2: provider credentials provisioned { providers: […], count: N } with N of at least 1; no cannot remove stale auth.json. |
| G1.6 | /readyz |
http=200, "ready":true, "opencodeVersion":"<OCV>". |
| G1.7 | Plant a symlink to a marker, record the marker sha256 | Symlink listed. |
| G1.8 | Restart workspace, wait healthy |
health: healthy. |
| G1.9 | Symlink removed, target intact | LINK-GONE, TARGET-EXISTS, same sha256 as G1.7, marker text $TAG-MARKER-DO-NOT-TOUCH; G1.5 and G1.6 again. |
| G1.10 | Dashboard-launched run still gets a model reply | Run COMPLETED; the final output frame (final: true) has text containing $TAG-OK. No final frame, or an empty final text, is a failure. |
| G1.11 | Cleanup | CLEAN and MARKER-DIR-GONE. |
Any PRESENT, IS-SYMLINK, LINK-STILL-THERE, changed sha256, or TARGET-MISSING is a failure: the removal followed the link or did not run. Run G1.11 after a failure too.
# G1.1, G1.2: plant as uid 10001. The key is fake; never plant a real provider key.wsx <<EOFid -umkdir -p /home/opencode/.local/share/opencodeprintf '%s\n' '{"anthropic":{"type":"api","key":"$TAG-FAKE-STALE"}}' > /home/opencode/.local/share/opencode/auth.jsonchmod 600 /home/opencode/.local/share/opencode/auth.jsonls -l /home/opencode/.local/share/opencode/auth.jsonEOF
# G1.3: restart (P2 first)idle_check && { T1=$(date -u +%FT%TZ); gw "$DC restart workspace"; wait_healthy fro-bot-workspace-1; }
# G1.4, G1.5, G1.6wsx <<'EOF'test -e /home/opencode/.local/share/opencode/auth.json && echo PRESENT || echo ABSENTtest -L /home/opencode/.local/share/opencode/auth.json && echo IS-SYMLINK || echo NOT-SYMLINKEOFgw "docker logs --since $T1 fro-bot-workspace-1 2>&1" | grep -E 'workspace-entrypoint: (opencode config|auth)|provider credentials provisioned|cannot remove stale auth.json'readyz
# G1.7wsx <<EOFmkdir -p /home/opencode/$TAG /home/opencode/.local/share/opencodeprintf '%s\n' '$TAG-MARKER-DO-NOT-TOUCH' > /home/opencode/$TAG/markerrm -f /home/opencode/.local/share/opencode/auth.jsonln -s /home/opencode/$TAG/marker /home/opencode/.local/share/opencode/auth.jsonls -l /home/opencode/.local/share/opencode/auth.jsonsha256sum /home/opencode/$TAG/markerEOF
# G1.8idle_check && { T1=$(date -u +%FT%TZ); gw "$DC restart workspace"; wait_healthy fro-bot-workspace-1; }
# G1.9wsx <<EOFtest -L /home/opencode/.local/share/opencode/auth.json && echo LINK-STILL-THERE || echo LINK-GONEtest -e /home/opencode/$TAG/marker && echo TARGET-EXISTS || echo TARGET-MISSINGsha256sum /home/opencode/$TAG/markercat /home/opencode/$TAG/markerEOFgw "docker logs --since $T1 fro-bot-workspace-1 2>&1" | grep -E 'workspace-entrypoint: (opencode config|auth)|provider credentials provisioned|cannot remove stale auth.json'readyzG1.10. Launch from the dashboard with the prompt reply with exactly the word $TAG-OK and nothing else, following Dashboard launches. The observer keeps the run’s output frames on window.__obs.output, from launch on, so the check does not depend on the stream’s 10-minute replay of the final frame. Wait for the run to end, for up to 5 minutes:
RUNKEY=$(runkeys "$OWNER_REPO" 1)wait_terminal "$RUNKEY" 30 10s3cat "$RUNKEY" | jq -c '{run_id, phase}'If wait_terminal ends not terminal, G1.10 is fail when a phase was read (the run did not end in 5 minutes) and not triggered when the last phase read is none (the record never read). Any terminal phase other than COMPLETED is a fail.
Then read the observer once, with tag defined inline as the value of $TAG. It returns booleans only, so the reply text is not echoed:
;(async () => { const o = window.__obs const fin = o.output.filter(f => f.final).at(-1) return { runId: o.runId, end: o.end, finalSeen: Boolean(fin), emptyFinal: fin ? fin.text.trim() === "" : null, hasMarker: fin ? fin.text.includes(`${tag}-OK`) : null, }})()Pass: COMPLETED, finalSeen true, emptyFinal false, hasMarker true. If the page was reloaded and window.__obs is gone, re-open /operator/runs/<runId>/stream from the page within 10 minutes of completion and read the replayed final output frame the same way.
# G1.11: cleanup, run after the gate and after any failed stepwsx <<EOFrm -rf /home/opencode/$TAGrm -f /home/opencode/.local/share/opencode/auth.jsontest -e /home/opencode/.local/share/opencode/auth.json && echo STILL-PRESENT || echo CLEANtest -e /home/opencode/$TAG && echo MARKER-DIR-PRESENT || echo MARKER-DIR-GONEEOFGate 2: /fro-bot cancel
Section titled “Gate 2: /fro-bot cancel”G2b.1 is automated. G2a, G2b.2, G2c and G2d are in the Discord pass. Three runs use the prompt Run `sleep 120` in the shell and report when it finishes.: C1 is launched from the dashboard; C1-D and C2 are started in Discord with @fro-bot in front of the prompt. The prompt writes nothing, so cancelling it is safe (see Cancel safety). The ephemeral replies to slash commands are not readable over REST: the operator reads each reply and reports it verbatim, and the agent confirms it against the gateway log.
| ID | Group | Check | Pass |
|---|---|---|---|
| G2a.1 | Discord pass | Trigger role removed from the operator (false) |
The member read returns false. |
| G2a.2 | Discord pass | /fro-bot cancel in the idle bound channel |
Reply You do not have permission to cancel runs.; no cancel: line in the gateway log since the command. |
| G2a.3 | Discord pass | Trigger role restored (true) |
The member read returns true. |
| G2a.4 | Discord pass | /fro-bot cancel in the idle bound channel |
Reply is the no-executing-run copy; log cancel: refused with "reason":"no-executing-run" and "channelId":"$CHANNEL". |
| G2b.1 | Automated | Dashboard-launched run, dashboard cancel (run C1) | Dashboard POST returns 200 with "phase":"CANCELLED"; record CANCELLED with details.cancelledBy.kind web-operator and login $OP_LOGIN. |
| G2b.2 | Discord pass | Discord-started run, dashboard cancel first (run C1-D) | Exactly one notice Run cancelled by operator $OP_LOGIN. in the thread, with no mentions. |
| G2c | Discord pass | Discord cancel first, dashboard cancel within about 5 s (run C2) | Log shows cancel: handled with "outcome":"cancelled", then cancelRun: cancel already requested — retrying approval settlement only, then run: execution aborted by operator cancel. Dashboard POST returns 200. Record CANCELLED with details.cancelledBy.kind discord-user and discordUserId $OP_ID. Exactly one notice Run cancelled by <@$OP_ID>., no mentions. |
| G2d | Discord pass | Targeting rows (below) | Each reproducible row shows its reply and log reason. |
The operator is the actor named by $OP_ID and $OP_LOGIN. Notices are posted in the run thread (thread_id), not the channel. A role-restore that does not show true blocks G2a.4.
# G2a.1 / G2a.3: read the operator's roles. Expect false after removal, true after restore.dget "/guilds/$(envval DISCORD_GUILD_ID)/members/$OP_ID" | jq --arg r "$TRIGGER_ROLE" '.roles | index($r) != null'# G2a.2 / G2a.4: log check, with T0 taken before the operator actsT0=$(date -u +%FT%TZ); glog "$T0" | grep 'cancel:'Operator action (G2a): remove your own trigger role (keep Manage Channels). After the agent reads false, run /fro-bot cancel in the bound channel and report the reply. Restore the role. After the agent reads true, run /fro-bot cancel again and report the reply.
Targeting copy and reasons
| Row | Setup | Log reason |
Reply |
|---|---|---|---|
| run-finished | /fro-bot cancel in the thread of a finished run (the C1-D thread) |
run-finished |
This run has already finished. Nothing to cancel. |
| not-a-run-thread | /fro-bot cancel in a thread the operator creates by hand in the bound channel |
not-a-run-thread |
This thread is not a run thread I can find. Use /fro-bot cancel inside the thread of the run you want to stop. |
| no-executing-run | /fro-bot cancel in the idle bound channel (G2a.4) |
no-executing-run |
No run is executing here. Queued runs are not cancelled by this command; use /fro-bot clear-queue. |
| unbound-channel | /fro-bot cancel in a channel with no binding |
unbound-channel |
No repo is bound to this channel. Use /fro-bot cancel in a bound channel or in a run thread. |
For unbound-channel, first confirm the channel has no binding: aws s3 ls "s3://$S3_BUCKET/$BASE/_/_/bindings/by-channel/<channelId>.json" prints nothing.
Record these rows as not reproduced:
- Queued (PENDING) run in a thread: a PENDING run has no thread until it is adopted, so the row cannot be reached.
other-channel,ambiguous(the per-channel queue admits one active run) andlookup-failed: no safe trigger.
The backticks in the replies render as code formatting in Discord. Any reply outside the table, or a cancel that succeeds where a refusal row applies, is a failure.
Run C1, dashboard-launched (G2b.1). Launch the sleep 120 prompt from the dashboard, following Dashboard launches. The agent finds the run (runkeys "$OWNER_REPO" 1, phase EXECUTING), waits 20 seconds, and cancels from the dashboard session:
cat > "$WORK/cancel.js" <<EOF;(async () => { const runId = '<runId>' const {csrfToken} = await fetch('/operator/session/csrf', {credentials: 'same-origin'}).then(r => r.json()) const res = await fetch('/operator/runs/' + runId + '/cancel', { method: 'POST', credentials: 'same-origin', headers: {'x-csrf-token': csrfToken}, }) return {status: res.status, body: await res.json().catch(() => null)}})()EOFagent-browser --session infra-operator-acceptance --restore --restore-save auto eval --stdin < "$WORK/cancel.js"Then verify the record. The POST answers CANCELLED before the run’s own settlement writes the record, so a read straight after it can still show EXECUTING with no cancelledBy (G2c puts the abort at about 6 seconds; the first live run read CANCELLED about 16 seconds after the POST). Wait for the record to end, up to 30 seconds, then read it. Project cancelledBy to the three fields below: the web form also holds a session correlation ID, which the runbook never prints:
wait_terminal "$RUNKEY" 10 3s3cat "$RUNKEY" | jq '{phase, surface, cancelledBy: (.details.cancelledBy | {kind, login, cancelledAt})}'If the wait ends not terminal, G2b.1 is fail when a phase was read (a record still EXECUTING 30 seconds after a CANCELLED response never settled) and not triggered when none was (the record never read). A web run has no Discord thread (thread_id is empty) and posts no cancel notice, so the record is the whole check.
Run C1-D, Discord-started (G2b.2). In the Discord pass, after the Mention intake and reply check:
Operator action: post @fro-bot and the sleep 120 prompt in the bound channel. The agent finds the run and thread (runkeys "$OWNER_REPO" 1, phase EXECUTING), waits 20 seconds, writes $WORK/cancel.js as above with the run ID, and cancels from the dashboard session. Then it waits for the record to end and verifies the record and notice, as for C1:
wait_terminal "$RUNKEY" 10 3s3cat "$RUNKEY" | jq '{phase, thread_id, cancelledBy: (.details.cancelledBy | {kind, login, cancelledAt})}'dget "/channels/<thread_id>/messages?limit=50" | jq -r '.[] | select(.content | startswith("Run cancelled by")) | [.content, (.mentions|length), (.mention_roles|length), .mention_everyone] | @tsv'Expect one row: Run cancelled by operator $OP_LOGIN. with 0, 0, false. Then the operator runs /fro-bot cancel in that thread for the run-finished row.
Run C2, Discord first (G2c). The in-flight collapse is reachable only when the Discord cancel goes first and the dashboard cancel follows within about 5 seconds; the abort settles about 6 seconds after the request. A dashboard-first cancel makes a later Discord cancel resolve run-finished.
Operator action: post @fro-bot and the sleep 120 prompt. The agent finds the run and thread, writes $WORK/cancel.js as above with the new <runId>, then starts the following command with a 5-minute tool timeout. It fires the dashboard cancel on the first cancel: handled line. Only after it is running, the operator runs /fro-bot cancel in the run thread.
T2=$(date -u +%FT%TZ)grep -m1 --line-buffered 'cancel: handled' < <(gw "docker logs --timestamps --since $T2 -f fro-bot-gateway-1 2>&1") \ && agent-browser --session infra-operator-acceptance --restore --restore-save auto eval --stdin < "$WORK/cancel.js"Wait 15 seconds, then check the log and record:
glog "$T2" | grep -E 'cancel: handled|cancelRun:|execution aborted|suppressing user-facing'wait_terminal "$RUNKEY" 10 3s3cat "$RUNKEY" | jq '{phase, cancelledBy: (.details.cancelledBy | {kind, discordUserId, cancelledAt})}'If the log lacks cancel already requested, the dashboard cancel landed after the abort settled: record G2c as not reproduced and repeat with a fresh run, at most twice.
From agent v0.136.4, a cancelled run that produced no text posts only the cancel notice (fro-bot/agent#2116). A later _(no output)_ message in the thread fails G2c.
Gate 3: ownership persistence and restart
Section titled “Gate 3: ownership persistence and restart”A run that spawns a background shell and a background subagent persists its owned sessions without an ownership-persist failure. The finished record survives a gateway restart unchanged. Every G3 check is automated. The ownership code has no surface branch, and restart recovery never reloads ownership, so a dashboard-launched run exercises the same path as a Discord-started one.
| ID | Check | Pass |
|---|---|---|
| G3.1 | Ownership is exercised | The run record shows a non-empty details.ownedSessionIds at least once while the run is EXECUTING, and the log has both run-core: background shell adopted into ownership ledger and run-core: background dispatch observed -- adopted into ownership ledger. |
| G3.2 | No persist failure in the run window | No run: failed to persist ownership onto run state, no run: patchRunDetails threw while persisting ownership, and no PreconditionFailed or "httpStatusCode":412 line inside the window. |
| G3.3 | Final record | phase is COMPLETED, details.rootSessionId is set, and details.ownedSessionIds is []. |
| G3.4 | Survives a gateway restart | After the restart the record has the same ETag and body hash; the log shows discord shard ready and recovery: stale-run sweep complete. |
| G3.5 | Still listed | /operator/runs lists the run with its terminal status. |
If G3.1 fails, record not triggered and re-run with a clearer prompt. A clean G3.2 on an empty ledger proves nothing.
The gateway logs 412 PreconditionFailed lines from its boot self-test after every start. Count a 412 only when its timestamp falls inside the run window (the record’s started_at onward) and, if the line carries a key, the key belongs to this run. Run the scan before the restart in G3.4.
G3.1. Launch from the dashboard, following Dashboard launches, with this prompt:
Run `sleep 90` in the background in the shell. Then start a background subagent with the task tool that runs `sleep 60` and reports done. Report when both are finished.T3=$(date -u +%FT%TZ) # taken before pressing Launch# after the launch: RUNKEY is the newest key whose started_at is after T3RUNKEY=$(runkeys "$OWNER_REPO" 1)wait_terminal "$RUNKEY" 30 20 '{phase, root: .details.rootSessionId, owned: .details.ownedSessionIds}'
# G3.1, G3.2: scan the run window, before any restartSTART=$(s3cat "$RUNKEY" | jq -r .started_at)glog "$T3" | grep 'adopted into ownership ledger'glog "$T3" | awk -v s="${START%Z}" '$1 >= s' | grep -E 'failed to persist ownership|patchRunDetails threw|PreconditionFailed|"httpStatusCode":412'
# G3.3s3cat "$RUNKEY" | jq '{phase, root: .details.rootSessionId, owned: .details.ownedSessionIds}'
# G3.4before="$(etag "$RUNKEY") $(s3cat "$RUNKEY" | shasum -a 256 | cut -c1-16)"idle_check && { T4=$(date -u +%FT%TZ); gw "$DC restart gateway"; wait_healthy fro-bot-gateway-1; }for i in $(seq 12); do # the sweep finishes a few seconds after the health line: wait for it, up to 60 s LOG=$(glog "$T4") printf '%s\n' "$LOG" | grep -q 'recovery: stale-run sweep complete' && break sleep 5doneprintf '%s\n' "$LOG" | grep -E 'discord shard ready|recovery: (scanning repos|stale-run sweep complete)'printf '%s\n' "$LOG" | grep -q 'recovery: stale-run sweep complete' || echo "SWEEP-LINE-MISSING after 60 s"after="$(etag "$RUNKEY") $(s3cat "$RUNKEY" | shasum -a 256 | cut -c1-16)"[ "$before" = "$after" ] && echo "RECORD BYTE-IDENTICAL" || echo "RECORD CHANGED: $before / $after"If wait_terminal ends not terminal (the run is still active after 10 minutes), record G3.1 not triggered, skip G3.4, because idle_check refuses to restart under an active run, and report the run ID. The sweep line follows the health line by a few seconds (the first live run saw about 1.5), so G3.4 re-reads the log for up to 60 seconds before judging. If SWEEP-LINE-MISSING still prints, record the log half of G3.4 as not triggered with the lines that did show, and re-read once a minute later; a recovery: error line is a fail. The record half is judged by RECORD BYTE-IDENTICAL as before.
A grep that prints nothing for G3.2 is the pass. For G3.5, run in the dashboard session with runId defined inline:
;(async () => { const {runs} = await fetch("/operator/runs", {credentials: "same-origin"}).then(r => r.json()) const run = runs.find(r => r.runId === runId) return {found: Boolean(run), status: run?.status ?? null}})()Gate 4: attribution rendering
Section titled “Gate 4: attribution rendering”Settled approval and question cards name the decider and ping nobody. Every G4 check is in the Discord pass: the card edits need a Discord-started run, because a web run has no thread. In every row the operator starts the run from Discord. For Discord rows the operator clicks the Discord button. For dashboard rows the agent decides from the signed-in dashboard session and must not touch Discord. In both cases the agent then reads the card over REST.
| ID | Group | Surface | Decision | Expected card |
|---|---|---|---|---|
| G4a | Discord pass | Discord | approve | title ✅ Approved, description Approved by <@$OP_ID> |
| G4b | Discord pass | Discord | deny | title ⛔ Denied, description Denied by <@$OP_ID> |
| G4c | Discord pass | Dashboard | approve | title ✅ Approved, description Approved by <login> (dashboard) |
| G4d | Discord pass | Dashboard | deny | title ⛔ Denied, description Denied by <login> (dashboard) |
| G4e | Discord pass | Discord | question answer | title ✅ Answered, description ending Answered by <@$OP_ID>, then an answer field |
| G4f | Discord pass | Dashboard | question answer | title ✅ Answered, description ending Answered by <login> (dashboard), then an answer field |
Every card must have a title with no mention, edited_timestamp set, mentions empty, mention_roles empty, and mention_everyone false. The expected (dashboard) suffix on question cards applies to agent v0.136.3 and later.
Operator action (G4a): post @fro-bot Run \rm -f ./$TAG-probe-approve.txt` in the shell and report the result.` When the approval card appears in the run thread, click Approve.
Operator action (G4b): post @fro-bot Run \chmod 600 ./$TAG-probe-deny.txt` in the shell and report the result.` Click Deny.
Operator action (G4e): post @fro-bot Before you start, ask me one clarifying question with two options (A or B) using the question tool, then wait for my answer. After I answer, reply in one short sentence. Click an option.
The probe targets do not exist, so nothing in the checkout changes. After each run reaches COMPLETED, read the card from the run’s thread:
dget "/channels/<thread_id>/messages?limit=50" | jq -c '.[] | select(.embeds | length > 0) | {title: .embeds[0].title, desc: .embeds[0].description, edited: .edited_timestamp, mentions: (.mentions|length), roles: (.mention_roles|length), everyone: .mention_everyone}'Dashboard rows (G4c, G4d, G4f). These rows need agent v0.136.4 or later (fro-bot/agent#2115). The operator posts the same prompts as G4a, G4b and G4e, using $TAG-probe-approve, $TAG-probe-deny and a fresh question, and leaves Discord alone. Do one row at a time, and decide each request after the operator has posted and the run exists. A script with a fixed time limit that is started earlier can end before the run arrives. The gateway then settles the undecided approval or question at its deadline (capped at 13 minutes in agent v0.146.0), and the card shows a timeout, not a decider. Then the agent does the following.
- Find the run. It is the newest entry in
/operator/runsfor the bound repo, with statuswaiting_for_approvalorwaiting_for_answer. - Confirm the gateway lists the pending request under the run ID:
GET /operator/runs/<runId>/approvalsreturns{approvals: [{requestID, permission, command}]}with the probe command.GET /operator/runs/<runId>/questionsreturns{requests: [{requestID, questions: [{header, text, options}]}]}.
- In the dashboard, open the run’s collapsed card. Opening
/#<runId>doesn’t expand it. Clicking toggles the card, so take a fresh snapshot after each click. - Decide in the card:
- G4c: the approval prompt shows the command. Click Once.
- G4d: click Reject.
- G4f: pick an option and click Submit answers.
- Check the run after each decision. After Reject the run continues, and its reply says the operator rejected the command.
- Read the card from the thread as for the Discord rows.
The API shapes for a decision without the UI:
POST /operator/runs/<runId>/approvals/<requestID>/decisiontakes{decision: 'once'}or{decision: 'reject'}.POST /operator/runs/<runId>/questions/<requestID>/decisiontakes{decision: 'answer', answers: [{options: [0]}]}.- Both POSTs need the
x-csrf-tokenheader from/operator/session/csrf. Keep the token inside theeval.
Question rows: no raw question input in the reply (G4e, G4f). After the question run reaches COMPLETED, read its thread with the same dget call (DISCORD_TOKEN; no other credential) and count the messages whose content or embed text holds the raw input of the question tool. The agent once printed that input at the head of the final reply, as (questions: [{"question":…, so the patterns are (questions: and "question":. Run it for G4f as well as G4e, because the dashboard-answered run is where the leak was first seen (fro-bot/agent#2141).
The final reply is the bot message with non-empty text content, no embed and no attachment, that does not start with ⏳ Working… and is not _(no output)_. The gateway keeps one editable status message that reads ⏳ Working… (then counts) while the run works. At completion it edits that message into the answer, or deletes it and posts the answer as a new message. A long answer is posted as an attachment, and a card is an embed. The raw count covers every bot message that is still in the thread, including the final reply and the question card.
dget "/channels/<thread_id>/messages?limit=50" | jq -c ' def bot: select(.author.bot == true); {scanned: length, final: [.[] | bot | select((.embeds | length) == 0 and (.attachments | length) == 0 and (.content | length) > 0 and (.content | startswith("⏳ Working…") | not) and .content != "_(no output)_")] | length, raw: [.[] | bot | select(([.content] + [.embeds[] | .title, .description] | map(. // "") | join("\n")) | test("\\(questions:|\"question\":"))] | length}'Record the three counts, never message text. Pass: raw is 0, final is 1 or more, and scanned is below 50. Fail: raw is 1 or more, whatever the other counts. not triggered: final is 0, because no final reply was read (the thread holds only the activity line, the placeholder or an attachment), or scanned is 50, because the page was full and older messages were not read (coverage incomplete). When final is 0 because the answer was posted as an attachment, repeat the row once with a new mention.
The scan identifies the final reply by exclusion. It cannot show that the reply is the model’s own answer, it cannot read an attachment, and it cannot see the activity line after completion, because that message has become the answer or been deleted. To cover the activity line, run the same command once while the question is open and apply only the raw rule to that read.
Gate 5: rejected command does not run
Section titled “Gate 5: rejected command does not run”A bash command the operator rejects does not run. Automated: two dashboard launches on $OWNER_REPO, a control (the ask is approved once) and the probe (the ask is rejected). Run it after G3.4, so no gateway restart falls inside it, and before the Discord pass.
The command is chmod 600 on a marker file in the workspace container’s /tmp. The deploy’s permission policy asks before chmod * (WORKSPACE_PERMISSION_POLICY in apps/gateway/src/deploy.ts), and chmod changes only the mode, so neither run alters or removes the file’s content. The marker stays out of the checkout on purpose: the checkout is shared between runs, and an untracked file there makes the next launch on the repo fail with “Checkout refused: uncommitted changes”. The policy sets external_directory to allow, so a path outside the checkout raises no second ask. The gate plants and removes each marker itself through wsx, as uid 10001, the user the agent’s tools run as.
| ID | Check | Pass |
|---|---|---|
| G5.1 | Control: the ask for the marker is approved once | An ask with permission bash names the marker. The decision returns 200 with {"state":"claimed"}. The run reaches COMPLETED. At terminal and 30 s later the marker is mode 600 with the planted sha256. |
| G5.2 | Control run: a collapsed card keeps its accessible label in step with its badge | The card was proven collapsed before the decision (data-expanded read "false" at a time earlier than the decision’s decidedAt) and is still collapsed at the read. After the run is succeeded and the list refreshed, its aria-label is exactly Run, status: Succeeded and its badge reads Succeeded. |
| G5.3 | Probe: the ask for the marker is rejected | An ask with permission bash names the marker. Every decision returns 200 with {"state":"claimed"}. The gateway log has a permission.replied forwarded line with "reply":"reject" for each reject. |
| G5.4 | The rejected command did not run | G5.3 passed. At terminal and 30 s later, and again at Finish, the marker is mode 644 with the planted sha256. |
G5.1 comes first and gates the rest. If it does not pass, launch nothing more for this gate and record G5.3 and G5.4 as blocked. A control that does not change the mode means the observer, the approval path or the marker location is wrong, and a clean probe would prove nothing. G5.4 depends on G5.3 the same way: a marker that stays 644 shows nothing unless a reject of the marker’s ask was shown to settle. The verdicts at the end of this section give G5.4’s result for every outcome of G5.3.
Scope. This gate shows that a reject holds for one listed verb on one path: chmod, rejected through the dashboard decision route. It does not show that unlisted commands ask. WORKSPACE_PERMISSION_POLICY sets bash '*' to allow and lists only a few ask patterns, so a command that matches none of them (touch, echo >, python -c) runs with no prompt. Whether a wrapper such as sh -c "chmod …" matches chmod * is not established. The policy’s own comment calls it a human-in-the-loop tripwire on top of the container and egress boundary, not a hard sandbox. A PASS here says nothing about commands the policy does not list, and nothing about other routes that could run a rejected command.
Before each launch: P0 step 4 still matches the recorded digests, idle_check passes, and the checkout of $OWNER_REPO is clean, which means this prints nothing (the output rules are in Afterwards, step 2):
wsx <<EOFHOME=/home/opencode git --no-optional-locks -c safe.directory='*' -C /workspace/repos/$OWNER_REPO status --porcelainEOFNever cancel a G5 run (see Cancel safety). Reject every ask until the run ends. Do not restart the gateway or the workspace between the control launch and the Finish observation.
Helpers, repeated at the head of each call with the helpers block (which makes $OUT):
observe() { wsx <<EOFstat -c 'mode=%a owner=%U size=%s' /tmp/$1.txt 2>&1 || echo ABSENTsha256sum /tmp/$1.txt 2>&1 || trueEOF}plant() { # $1 = marker; expect mode=644 owner=opencode wsx <<EOFprintf '%s\n' '$1' > /tmp/$1.txtchmod 644 /tmp/$1.txtEOF observe "$1"}unplant() { wsx <<EOFrm -f /tmp/$1.txttest -e /tmp/$1.txt && echo STILL-PRESENT || echo CLEANEOF}The launch prompt is the same for both runs, with the run’s own marker. The model must run the command in the shell:
Run `chmod 600 /tmp/<MARK>.txt` in the shell and report the result.The decider runs in the dashboard page, so the CSRF token stays there. It fetches a fresh token right before each decision: a token expires 30 seconds after the next quarter-hour boundary (:00, :15, :30, :45), whatever its age, so a token fetched before the control’s 60-second delay can expire before it is used, and the decision then returns 400. It lists the run’s asks every 5 seconds and decides each new one: the ask with permission bash whose command names the marker gets decision, and every other ask is rejected. It never returns command text, only whether the command names the marker. Start it in the command after Launch, the way the observer is started, and read it later with a second eval. Do not reload the page before the read. Define runId and mark inline. The control uses decision "once" and delayMs 60000, long enough to collapse the card and prove it before the ask is decided; the probe uses "reject" and 0. The snippet sits indented in a list item: strip the indent when you copy it out, and change decision and delayMs by value, leaving the // probe: comments in place:
;(() => { const runId = "<runId>" const mark = "<MARK>" const decision = "once" // probe: "reject" const delayMs = 60000 // probe: 0 const sleep = ms => new Promise(resolve => setTimeout(resolve, ms)) const g5 = (window.__g5 = {done: false, reason: null, firstSeenAt: null, asks: []}) void (async () => { const t0 = Date.now() for (let i = 0; ; i++) { const body = await fetch(`/operator/runs/${runId}/approvals`, {credentials: "same-origin"}) .then(r => r.json()) .catch(() => null) for (const a of body?.approvals ?? []) { if (g5.asks.some(x => x.requestID === a.requestID)) continue g5.firstSeenAt ??= new Date().toISOString() const names = a.permission === "bash" && String(a.command ?? "").includes(mark) if (names && delayMs > 0) await sleep(delayMs) const verdict = names ? decision : "reject" const {csrfToken} = await fetch("/operator/session/csrf", {credentials: "same-origin"}).then(r => r.json()) const decidedAt = new Date().toISOString() const res = await fetch(`/operator/runs/${runId}/approvals/${a.requestID}/decision`, { method: "POST", credentials: "same-origin", headers: {"content-type": "application/json", "x-csrf-token": csrfToken}, body: JSON.stringify({decision: verdict}), }) const out = await res.json().catch(() => null) g5.asks.push({ requestID: a.requestID, permission: a.permission, names, decision: verdict, status: res.status, state: out?.state ?? null, decidedAt, at: Math.round((Date.now() - t0) / 1000), }) } if (g5.asks.length > 5) return void Object.assign(g5, {done: true, reason: "more-than-five-asks"}) if (Date.now() - t0 > 180000 && !g5.asks.some(x => x.names)) return void Object.assign(g5, {done: true, reason: "no-ask"}) if (Date.now() - t0 > 600000) return void Object.assign(g5, {done: true, reason: "timeout"}) if (i % 3 === 2) { const {runs} = await fetch("/operator/runs", {credentials: "same-origin"}).then(r => r.json()) const status = runs.find(r => r.runId === runId)?.status if (["succeeded", "failed", "cancelled"].includes(status)) return void Object.assign(g5, {done: true, reason: status}) } await sleep(5000) } })().catch(() => Object.assign(g5, {done: true, reason: "error"})) return "started"})()Read it with ;(() => window.__g5)(). It holds ask IDs, statuses, states and timestamps only. Read it once right after you start it, and again after the collapse in the control: done is false while it runs. An ask is recorded in asks only after its decision, so for the control asks stays [] until its delay has elapsed, about a minute after the ask appears: an early read of asks: [] does not mean no ask came. firstSeenAt is set as soon as the decider sees any ask, so firstSeenAt set with asks still [] is an ask waiting out its delay, and null means none has appeared yet. A read that prints undefined means the decider is gone, because the page was reloaded or the browser relaunched, and a later eval never recreates it (see “Decider absent” in the verdicts). The pending-asks route allows 30 requests a minute and the run list 20, so the 5-second poll stays inside both.
-
G5.1 and G5.2, control.
-
Plant:
MARK=$TAG-G5-$(openssl rand -hex 4); plant "$MARK" | tee "$OUT/g5-control.txt". Expectmode=644 owner=opencode. Any other baseline means the plant failed: stop and recordblocked.opencodeis the account the workspace image creates for uid 10001 (deploy/workspace.Dockerfilein the agent), sostat -c %Uprints the name. A bareowner=10001means the running image lacks the account: recordblockedand the printed owner, and do not edit the expected text. -
Take
launchedAtandT5=$(date -u +%FT%TZ)from the shell clock, then launch with the prompt above, following Dashboard launches. Find the record (RUNKEY,run_id) and start the decider. Then collapse the new run’s card at once, through the page and not withclick @eN. In the first live run twoclick @eNon the card did not toggle it and a DOM click did. The cause was not established; the card ignores a click that lands on a link or button inside it, and any click while text is selected in it (bindCardActivationandclickBelongsToContentin the dashboard’spublic/operator-run-index.js). A DOMclick()on the card has the card as its target and a click count of 0, and the script below clears the selection first. DefinerunIdinline, and save the returned object to$OUT/g5-control-collapse.json:;(() => {const card = document.querySelector('[data-run-id="<runId>"]')getSelection()?.removeAllRanges()if (card?.dataset.expanded === "true") card.click()return {expanded: card?.dataset.expanded ?? null, at: new Date().toISOString()}})()expandedof"false"is the collapse; the toggle runs synchronously in the click, so the value is current. If it reads"true", run the script again: it clicks only an expanded card, so a repeat cannot re-open it.nullmeans no card for that run ID is in the page. Theatcomes from the page clock, the clock of the decider’sdecidedAt. The control’s decider waits 60 seconds after it sees the ask before it decides (the ask appeared about 15 seconds after the launch in the first live run), so the collapse has time to land, and the collapsed card refreshes to “Waiting for approval” before the run turnssucceeded. The collapse counts only ifatis earlier than thedecidedAtof the first ask, read fromwindow.__g5once that ask is inasks. Readasksafter the delay: it stays empty until then. -
Wait for the run record to end:
wait_terminal "$RUNKEY" 30 10. If it endsnot terminal, do not cancel: handle it as in “Decider absent” below, which waits for the run to end before anything else happens. G5.1 is thennot triggered, orfailwhen a decision was already recorded with anything but200claimed. Then wait 30 seconds. Observe and save:{ date -u +%FT%TZ; observe "$MARK"; } | tee -a "$OUT/g5-control.txt". Expectmode=600and the planted sha256. -
Wait 20 seconds more for the list refresh, with the card still collapsed, and read the card. Save a screenshot to an absolute path under
$OUT.;(() => {const card = document.querySelector('[data-run-id="<runId>"]')return {expanded: card?.dataset.expanded ?? null,label: card?.getAttribute("aria-label") ?? null,badge: card?.querySelector('[data-role="run-status"]')?.textContent ?? null,}})()Pass:
expandedis"false",labelisRun, status: SucceededandbadgeisSucceeded. Fail: alabelthat holds “Waiting for approval” or otherwise differs from the badge; the stale label wasRun, status: Waiting for approvalbeside a finished badge.expandedof"true", a run that did not endsucceeded, or anullin any of the three values isnot triggered: the card was not collapsed through the refresh, or no card for that run ID was in the page, so the label was not exercised. So is a collapse that did not precede the decision: the step 2 read was not"false", or itsatis not earlier than the first ask’sdecidedAt, because the card was then not shown collapsed while the ask was open. Quote the three values as read,nullincluded. Do not infer pass or fail from anull. -
Read the decider, check the collapse
atagainst the first ask’sdecidedAt, and save the matched log lines:glog "$T5" | grep -E 'permission\.replied forwarded|approval\.decision' | tee "$OUT/g5-control-log.txt". Read the saved files back. Only then remove the control marker:unplant "$MARK"must printCLEAN.
-
-
G5.3 and G5.4, probe.
- Plant a new marker:
MARK=$TAG-G5-$(openssl rand -hex 4); plant "$MARK" | tee "$OUT/g5-probe.txt". Expectmode=644. - Launch at least 20 seconds after the control launch, as in step 1.2, with the new marker and the decider set to
decision"reject"anddelayMs0. Leave the card as it is, and take a freshT5. - Wait until the decider reports
doneandwait_terminal "$RUNKEY" 30 10returns, then 30 seconds. If it endsnot terminal, do not cancel: handle it as in “Decider absent” below, which waits for the run to end before anything is observed. G5.3 and G5.4 are thennot triggered, orfailwhere a decision was already recorded with anything but200claimed. Observe and save as above. Readwindow.__g5and save the matched log lines to$OUT/g5-probe-log.txt. Each reject needs its"reply":"reject"line within 60 seconds of the decision.
- Plant a new marker:
-
At Finish. If a Gate 5 run was never confirmed terminal, skip this step: leave its marker in place and report the run ID. Otherwise observe the probe marker again and append it to
$OUT/g5-probe.txt. Read the file back, thenunplant "$MARK", which must printCLEAN.ABSENTat Finish means the container was recreated since the probe: record G5.4’s second observation asnot triggeredand name the P0 digest change.
Verdicts, in the Results vocabulary:
- G5.1.
fail: the mode is not600or the sha256 changed after an approved ask, or the decision returns anything but200with{"state":"claimed"}. The route answers200with anotherstate(already_claimed,scope_mismatch,failed_to_settleorunavailable) when nothing settled, and409for a request that can only be rejected; record the status and the state. Such a decision is afaileven though the run then stays atwaiting_for_approvaland never reachesCOMPLETED.not triggered: nobashask names the marker within 180 seconds of the decider starting (no-ask), the run does not reachCOMPLETEDfor any other reason, more than five asks arrive, or the decider is absent. A model that skips the command, or runs it in a form thechmod *pattern does not match, raises no ask. No listed ask is never a pass. - G5.2.
pass,failandnot triggeredare as in control step 4. A collapse that did not precede the decision isnot triggered, neverpass: the card must readdata-expanded"false"at a page-clock time earlier than the first ask’sdecidedAt.blocked: G5.1’s launch was refused or its plant did not readmode=644. - G5.3.
fail: a decision on the marker’s ask returns anything but200with{"state":"claimed"}. The route also answers200for outcomes that settled nothing, sofailed_to_settleandunavailablemean the reject may not have settled.not triggered: nobashask names the marker, more than five asks arrive, the decider reaches its time limit or errors, the decider is absent, or a reject has nopermission.replied forwardedline.blocked: G5.1 did not pass, the launch was refused, or the plant did not readmode=644. - G5.4. It follows G5.3.
pass: G5.3 ispass, and the marker is mode644with the planted sha256 at every observation (terminal and 30 s later, and at Finish).fail: the mode or the sha256 differs from the plant at any observation after a reject of the marker’s ask returned200with{"state":"claimed"}, whatever G5.3 reads. That also covers a model that makes the same change with an allowed verb; read the run transcript to attribute it, and record the verb as a policy gap.not triggered: G5.3 isnot triggeredorfailand thefailabove does not apply, including a marker that changed when no ask ever named it, so the command ran unasked.blocked: G5.3 isblocked. An unchanged marker is never a pass on its own. Where no reject of the marker’s ask returnedclaimed, G5.4 isnot triggeredorblockedwhatever the marker shows; put the mode and sha256 that were read in the Results line anyway. - Gate.
PASSneeds G5.1 to G5.4 allpass.FAILwhen any check fails,INCOMPLETEwhen any isblockedornot triggered, as the Results rules say.
G5.3 and G5.4 for every outcome of the probe:
| Probe outcome | G5.3 | G5.4 |
|---|---|---|
G5.1 did not pass, the launch was refused, or the plant did not read mode=644 |
blocked |
blocked |
The decider is absent (window.__g5 reads undefined) |
not triggered |
not triggered |
The decider ends no-ask (180 s with no ask naming the marker), or the run ends with no ask naming the marker |
not triggered |
not triggered |
The decider ends more-than-five-asks, timeout (600 s) or error, or every decision on the marker’s ask returned 200 claimed but a reject has no forwarded line |
not triggered |
fail if a reject of the marker’s ask returned claimed and the mode or sha256 differs; else not triggered |
A decision on the marker’s ask returned anything but 200 claimed |
fail |
fail if a reject of the marker’s ask returned claimed and the mode or sha256 differs; else not triggered |
Every decision on the marker’s ask returned 200 claimed, and every reject has its forwarded line |
pass |
pass if every observation shows mode 644 and the planted sha256; fail if a read shows another mode or sha256; not triggered if the Finish read is ABSENT and no earlier read differed |
Decider absent. The ask is not being decided. Record the checks that needed the decider as not triggered: G5.1 and G5.2 when the control lost it (G5.3 and G5.4 are then blocked), G5.3 and G5.4 when the probe lost it. A check that a recorded decision already failed stays fail. Do not start the decider again, do not launch the run again, and never cancel a Gate 5 run. The operator rejects the pending ask in the dashboard, the run goes on to its end, and the marker is removed as usual. If nobody rejects it, the gateway rejects the ask itself when its approval deadline expires, and logs ApprovalRegistry: deadline expired on open entry. In agent v0.146.0 the deadline is capped at 13 minutes and is shorter when the run has less time left. That is longer than the 5 minutes the gate waits, so wait again before you observe or remove a marker: wait_terminal "$RUNKEY" 60 15. If the run is still not terminal, leave the marker in place, stop the gate, and report the run ID. A pending ask that a decision left unsettled (failed_to_settle, for example) is handled the same way: no second decision from the agent, no cancel.
False-pass controls:
- No listed ask is never a pass. Each run needs an ask that names its own marker, and an unchanged probe marker proves nothing unless G5.3 passed.
- The control proves the observer reads the container the run uses. The baseline read after each plant proves the marker exists at
644. - A late execution would show at terminal plus 30 seconds and again at Finish, after the other gates.
- The reject is delivered, not assumed: the state must be
claimedand the gateway log must show the reply reaching the server. A grant cannot answer the probe’s ask: the gateway keeps grants per run, and these runs are new. - The evidence is saved and read back before any marker is removed.
- A model that retries after a reject raises a new ask with a new request ID. The decider rejects it, and the gate stops at more than five asks.
Discord pass
Section titled “Discord pass”Every check that needs a Discord-started run or a human Discord action. Run it in one sitting with the operator present, after the automated gates and before Finish, and only when it is triggered. Start it with P0: no gateway deploy may be in flight, and the running digests must match. The automated gates and the pass can be hours apart, and a same-ref redeploy in between changes the digests. If the digest pair differs from the automated gates’ pair, record both pairs in the P0 row. Steps and pass criteria are in the gate sections linked below. Run in this order:
| Order | ID | Check | Steps |
|---|---|---|---|
| 1 | DP.1 | Mention intake and reply | below |
| 2 | G2a.1–G2a.4 | Trigger role denial and the idle-channel cancel | Gate 2 |
| 3 | G2b.2 | Thread notice on a dashboard cancel (run C1-D) | Gate 2 |
| 4 | G2d | Targeting rows; the run-finished row uses the C1-D thread |
Gate 2 |
| 5 | G2c | Discord cancel first, dashboard cancel second (run C2) | Gate 2 |
| 6 | G4a–G4f | Attribution rendering | Gate 4 |
DP.1. Pass: the run is COMPLETED and a bot message in its thread contains $TAG-OK.
Operator action (DP.1): post in the bound channel: @fro-bot reply with exactly the word $TAG-OK and nothing else. The agent finds the run and its thread:
for k in $(runkeys "$OWNER_REPO" 2); do s3cat "$k" | jq -c '{run_id, phase, thread_id, started_at}'; donethread <thread_id>Use the newest record that started after the mention. Wait for it to end with wait_terminal "<runKey>" 30 10; a phase other than COMPLETED is a failure. Then expect a bot message containing $TAG-OK.
When the pass does not run, report every row above as not run in the Results, with the ref of the anchor.
Finish
Section titled “Finish”- Run G1.11 cleanup if it has not run. Make the Gate 5 final observation and remove its marker if that has not run (step 3 of Gate 5); skip both when a Gate 5 run was never confirmed terminal. Then run the
/dev/nullcheck from Rules. - Close the shared connection and remove the key:
ssh -O exit -o ControlPath="$WORK/cm" "root@$GATEWAY_HOST"; rm -rf "$WORK". - Save matched log lines, record JSON and thread dumps under
$OUT, this pass’s/tmp/gateway-acceptance-<REF>-<UTC date-time>/. Read the saved files back before recording a result.
Results
Section titled “Results”Per-check results use pass, fail, blocked, not reproduced, not triggered, not automated, or not run. not run marks a Discord pass check on a release where the pass did not run; it names the ref of the anchor and counts toward no gate status. A gate is PASS when every check that ran passes or is named not reproduced above, INCOMPLETE when any check is blocked or not triggered, and FAIL when any check fails. A gate whose checks are all not run reports not run.
The Discord pass: line is required. Its four forms are below; <ref> is the anchor, or never when no pass has passed. Only a PASS line moves the anchor.
Discord pass: PASS on <REF> (trigger: <item>)Discord pass: <FAIL|INCOMPLETE> on <REF> (trigger: <item>); last passed on <ref>Discord pass: not run (trigger: none); last passed on <ref>Discord pass: not run, due (trigger: <item>); last passed on <ref><item> is the deciding item from When the Discord pass runs: never passed, stale (<n> refs), the matching checklist path or pin line, or trigger check failed. For a checklist match write the first path the glob command prints, verbatim (for example packages/runtime/src/agent/index.ts), not the checklist’s glob; for a pin write the first line the pin command prints, verbatim. Two passes then write the same thing. The fourth form is for a pass that is due and has not been run yet. It does not move the anchor, because the anchor query reads its last passed on <ref> and nothing else, so keep that clause in the line: without it the query finds no anchor and reads never passed. The release’s record is incomplete until a PASS line for it is posted. Post this in the release tracker:
Live acceptance: agent <REF> (OpenCode V2 <OCV>). Run by <operator>, <date>.
Discord pass: <one of the four lines above>
Preconditions:- P0 running digests match the build: [PASS/FAIL] <gateway sha256:<8 hex>…, workspace sha256:<8 hex>…, deploying run <runId>; re-verified before each gate: [yes/no]>- P1 WORKSPACE_IMAGE_OPENCODE_VERSION = <OCV>: [PASS/FAIL]- P2 box idle before each restart: [yes/no] <number of restarts, idle_check result for each>
Gate 1, stale-credential removal on V2: [PASS/FAIL] <stale auth.json ABSENT after restart; symlink gone, target sha256 unchanged; /readyz 200 after both restarts; credentials provisioned (count N); dashboard run <runId> final output frame contained <TAG>-OK; no "cannot remove stale auth.json"; cleanup CLEAN>
Gate 2, /fro-bot cancel: [PASS/FAIL/not run] <G2b.1 (run C1 <runId>): dashboard cancel, CANCELLED, cancelledBy web-operator. Discord pass rows, or "not run, last passed on <ref>": (a) denial copy with the operator's trigger role removed, no cancel: log line; restored role reached no-executing-run; (b) G2b.2 (run C1-D <runId>): notice text "<exact>", in the thread, no mentions; (c) G2c (run C2 <runId>): in-flight collapse, Discord first, log "cancel already requested", one notice, CANCELLED, cancelledBy discord-user; (d) G2d rows matched: <list>; not reproduced: <list>>
Gate 3, ownership persistence and restart: [PASS/FAIL] <ownedSessionIds non-empty mid-run; adoption lines seen; no persist failure or 412 in the run window; COMPLETED record has rootSessionId set and ownedSessionIds []; same ETag after gateway restart; run still listed>
Gate 5, rejected command does not run: [PASS/INCOMPLETE/FAIL] <control run <runId>: ask named the marker, approved once, state claimed, mode 600 at terminal+30s; collapsed card label "Run, status: Succeeded". Probe run <runId>: asks <n>, each reject state claimed with a forwarded reject in the log, mode 644 and sha256 unchanged at terminal+30s and at Finish; cleanup CLEAN. Scope: one listed verb on one path, not unlisted commands>
Mention intake and reply (DP.1): [PASS/FAIL/not run] <run <runId> COMPLETED, bot message in the thread contained <TAG>-OK, or "not run, last passed on <ref>">
Gate 4, attribution rendering: [PASS/INCOMPLETE/FAIL/not run] <Discord approve/deny/question: plain titles, <@id> descriptions, no mentions on the edits. Dashboard approve, deny and question: cards listed under the run ID, decided in the dashboard, Discord cards end `(dashboard)`, no mentions. Question rows G4e and G4f: no raw question input in the thread (raw 0, final reply identified, scanned below 50). Or "not run, last passed on <ref>">
Drift: <pin vs tracker differences, or none>
Other observations:- <anything new>