dev-instance

dev-instance runs the current worktree with the surfaces needed for the task. Choose web for browser/admin work, Slack for bot work, and both only for cross-surface QA. It is the way to QA a branch end to end: real LLM turns, a real sandbox, a real local Postgres (empty by default; opt in to prod data), and the real Slack/web/admin surfaces.

Use the repo-root launcher (a thin wrapper over the TypeScript CLI in scripts/dev/; every command accepts --json for machine-readable output):

bash scripts/dev-instance.sh up --surface web
bash scripts/dev-instance.sh up --surface slack
bash scripts/dev-instance.sh up --surface both
bash scripts/dev-instance.sh status
bash scripts/dev-instance.sh down
bash scripts/dev-instance.sh doctor
bash scripts/dev-instance.sh canary
bash scripts/dev-instance.sh restart [child]
bash scripts/dev-instance.sh logs [child] [-f]

npm run dev-instance, npm run dev-instance:status, npm run dev-instance:down, and npm run dev-instance:doctor are equivalent. The Codex-visible skill copy lives at .codex/skills/dev-instance/SKILL.md; keep the two skill descriptions equivalent.

New instances default to web only. npm run dev-instance:web, npm run dev-instance:slack, and npm run dev-instance:both select a surface explicitly. Bare up preserves the mode of an existing instance. Explicitly selecting a different surface tears down and boots the worktree again, claiming the appropriate slot. --no-slack (and DEV_INSTANCE_NO_SLACK=1) remain aliases for web only.

What up Starts

up --surface slack and up --surface both claim one free Slack app slot from this machine's pool store (see "Slack reach" below), then spawns a per-slot supervisor daemon that owns the production-shaped stack:

• core API + workers in every mode
• Slack Socket Mode plugin in slack and both
• web UI, admin, and portal front door in web and both

Web-only instances need no Slack app or pool credentials. Slack-only instances skip web dependencies, builds, and web/portal processes.

Port slots expand automatically beyond 16, up to the TCP port range (8,207 slots at the default base port of 8080); machine resources will usually limit capacity first. Slots 1–16 retain their existing ports. Allocation skips occupied port blocks. Startup refuses occupied ports without terminating their listeners; if a port is taken after allocation, stop the failed instance and retry up to select a free block.

The supervisor restarts crashed children with backoff, refuses occupied ports, health-probes everything every 10s, and writes a heartbeat so slot reclaim can tell "actively in use" from "abandoned".

When Slack is enabled, up only prints success after proving the bot is reachable: the Slack socket must be the app's only connection (num_connections == 1, read from the hello frame) and — when the slot has a CANARY_CHANNEL — a posted canary message must arrive back over that same socket. If Slack reports multiple connections, up flags the slot for 30 minutes and auto-rotates to the next one. The count is a snapshot from the last hello frame, not a continuously refreshed inventory. Its debug_info.host identifies Slack's server, not another client's machine; it cannot locate a competing instance. A canary that never returns leaves delivery unverified; up flags and rotates past that too.

Re-running up on a live instance is a reload, not a no-op: it re-reads your shell env, dev.env, and .env, diffs against what the children are running, and does a rolling restart + re-verification when anything changed (--force to restart regardless, --rotate to move to a different Slack app).

Open the portal URL printed by the CLI. Direct web/admin URLs are also printed for debugging, but the portal URL is the prod-like path.

Sandbox: local Docker by default

The agent's execute sandbox runs as a local Docker container (SANDBOX_BACKEND=local) when the worktree's code supports it — no cloud credential, no tunnel, no microVM cold starts, and the sandbox reaches core's self-API at host.docker.internal directly. Build the image once per machine with npm run sandbox:local:build (the CLI warns if it's missing or stale). Use local Docker unless you are specifically testing the cloud path.

For prod-parity QA, pass --sandbox <backend> to run against your deployment's real sandbox provider instead. That path validates the provider credential and opens a tunnel so the remote sandbox can call back to your local core, so it needs whatever that provider requires; the local backend needs nothing but Docker.

Slack reach: your apps vs a teammate's

This is the part people get wrong, so it's worth stating plainly.

A Slack app's Socket Mode connection is single-owner: if two running instances point at the same app, Slack load-balances inbound messages between them and each instance silently loses half. So the rule is one live instance per Slack app.

The way we keep out of each other's way is simple: every developer has their own set of Slack apps, listed as poolN.env files in their own machine's pool store (~/.config/qm/slack-pool). up claims the first free slot on this machine and boots that app's bot; the script prints its @handle. You QA by DMing that bot in example.slack.com.

• Your machine (Alice's): the pool holds pool1.env … pool10.env, which are the apps bot1 … bot10. So up to 10 worktrees can run reachable at once here; each up grabs the next free slot, down frees it. DM whichever @botN the script printed.
• A teammate's machine (e.g. Carol's): a different set of apps in their local pool store. Their instances claim their apps; yours claim yours. You never collide, because you are never pointing at the same app — and there is no shared registry, no #worktree prefix, and no relay to coordinate. Each of you just DMs your own bot.

The per-machine pool lease (an atomic lock per slot) also stops two worktrees on the same machine from grabbing the same slot. Before connecting, up sweeps the machine for orphaned processes still holding the slot's Slack app token (e.g. a plugin another worktree's teardown missed) and kills them. If every slot is taken, up reclaims one — but never a slot whose supervisor heartbeat is fresh: an actively used instance can't be stolen mid-QA; only dead/legacy leases fall back to the old not-today / 4-hour rule.

To add capacity, mint more Slack apps from src/slack/manifest.json (one Socket Mode app each, with connections:write for the app-level token) and drop a poolN.env containing SLACK_BOT_TOKEN (xoxb-), SLACK_APP_TOKEN (xapp-), and HANDLE into your pool store. The boot canary picks its channel automatically: a channel named dev-canary if the bot is in one, else a ci-* throwaway it is in — never a human channel — and the canary message is deleted right after it round-trips. CANARY_CHANNEL=<channel id> in the slot env overrides. With no eligible channel at all, up prints delivery unverified (or fails under --strict).

If its worktree disappears, the supervisor shuts down its children and releases its lease on the next heartbeat (within 15 seconds, plus shutdown time). Always run down before intentionally removing a worktree.

A forgotten instance cleans itself up: after 24 hours with no handled Slack turns, interactions, reloads, or restarts, the supervisor tears itself down and frees the slot on its next idle check (every 10 minutes). Ambient workspace events, health checks, status reads, and canaries (including dev doctor) do not reset the timer. This is an idle timeout, not a fixed 24-hour lifetime. It only retires instances running this supervisor; extra connections on another host must be stopped on that host. DEV_INSTANCE_IDLE_HOURS overrides the timeout; 0 disables it.

Real by Default

The dev instance should exercise the real system:

• real LLM: needs a model credential for the harness you run. Core supports HARNESS=pi|claude; the launcher runs Pi when it finds an Anthropic key and honours an explicit HARNESS=claude. Pass DEV_INSTANCE_ALLOW_MOCK=1 for a deliberate no-model wiring check. Pi's GPT models can also use a ChatGPT subscription: set CODEX_AUTH_FILE to a local Codex CLI auth.json. The auth-file path is for local dev instances; deployed production processes use an API key or a keychain credential (CODEX_AUTH_CREDENTIAL / CLAUDE_AUTH_CREDENTIAL), whose secret lives encrypted in its owner's keychain.
• real durability: uses DATABASE_URL when supplied; otherwise starts/reuses a local Docker Postgres container and runs core with SESSION_STORE=postgres and RUN_STORE=postgres
• production data is never copied into a dev instance
• captured model context: request capture is on unless overridden, so you can read back what the model was actually sent
• fast local edits: runs Node surfaces with --watch and serves the web UI through Vite HMR on the dev surface instead of a separate browser build loop
• stale pool recovery: status shows each deploy's start time and age; if all pool apps are taken, up reclaims a slot not started today or older than 4 hours
• local admin seed: for a self-provisioned local DB, empty admin_grants are seeded from DEV_INSTANCE_ADMIN_PRINCIPAL or the local OS user; explicit ADMIN_GRANTS still wins
• local portal auth: the launcher sets a localhost-only portal auth bypass, signing in as DEV_INSTANCE_ADMIN_PRINCIPAL, the first ADMIN_GRANTS principal, the first durable org_admin in Postgres, or the local OS user. Production portal auth remains real OIDC.

Only use escape hatches for deliberate wiring checks — this is how you ask for less than the full real stack when a test doesn't need it:

DEV_INSTANCE_ALLOW_MOCK=1 bash scripts/dev-instance.sh up
DEV_INSTANCE_ALLOW_MEMORY=1 bash scripts/dev-instance.sh up
DEV_INSTANCE_WATCH=0 bash scripts/dev-instance.sh up
DEV_INSTANCE_RECLAIM_STALE=0 bash scripts/dev-instance.sh up

Env Discovery

The launcher reads values from, in priority order: exported shell env, the machine-global ~/.config/qm/dev.env, your login shell (for a model credential exported there), and this worktree's .env (seeded from the main checkout in linked worktrees). Slack pool tokens default to ~/.config/qm/slack-pool.

When a cloud sandbox backend is configured it also validates that provider's access at startup, refreshes a stale provider token from the provider CLI's own logged-in session where it can, and — if a tunnel binary is present — opens a quick tunnel so sandbox self-API calls can reach your local core. None of that runs on the default local backend.

After Startup

Report the slot, enabled surfaces, their URLs or Slack handle, and log directory. To test Slack-specific behavior, DM the printed @<handle> (on Alice's machine that's one of @bot1 … @bot10) in example.slack.com; for admin and web behavior, open the printed portal URL. Keep review instances running until the PR merges. After merge, run bash scripts/dev-instance.sh down from that worktree with the same pool-store override used for up, before removing the worktree. Verify the instance's supervisor and children have exited and its lease is released. Keep shared Postgres and persistent data intact. If the user asks to keep the instance beyond merge, retain its worktree too. For work without a PR, tear down when QA is finished unless the user requested a running demo.

Troubleshooting

Start with dev doctor (or doctor --json). It runs the checks that used to take a debugging session by hand — socket exclusivity (num_connections), a live canary round trip, env/git drift since boot, per-child health and restart counts, stale leases, port squatters, machine-wide token orphans, Docker daemon — and prints a ranked diagnosis with a remedy per finding. doctor --fix applies the safe ones (child restarts).

Bot never replies to a DM (silent bot). The new up catches the two big causes at boot: multiple reported connections to the same app (flags the slot and auto-rotates) and a Slack app whose events never arrive (canary fails → flags the slot and rotates). Check local processes and other deployments before attributing a connection count to a live competitor. After cleanup, reconnect to obtain a fresh hello count; rereading health alone returns the previous snapshot. The 10s health probe logs DEGRADED: num_connections=N in supervisor.log when it observes a count transition above one, but cannot detect a new competitor while the hello snapshot is unchanged. Periodic canaries can detect delivery loss; dev canary tests delivery on demand. A successful canary does not establish exclusivity. dev up --rotate moves to a fresh slot.

For a slot flagged canary-failed, check networking, event subscriptions, permissions, and competing connections first. If the app configuration needs rebuilding, use api.slack.com signed into the example workspace (the dev console is per-workspace-identity — use "Sign in to another workspace" if it lists the wrong one): Create New App → From a manifest → paste src/slack/manifest.json (give it a unique name and bot display_name; old handles stay taken) → Install to Workspace → Basic Information → App-Level Tokens → Generate with the connections:write scope. Write the Bot token (xoxb-…) and app-level token (xapp-…) into poolN.env, delete poolN.flag.json, then run up again to verify the replacement. Reinstalling an existing app may also resolve configuration drift; verify delivery afterward.

Two smaller gotchas: a freshly-created bot isn't in Slack's "New message" people search for a minute or two — open its DM deterministically via conversations.open (bot token + your user id) and navigate to app.slack.com/client/<team-id>/<D-channel-id>. And the browser's Slack login drops easily during long automated sessions; re-authenticating is a Google SSO you have to complete yourself.