Test Environment Selection

Overview

Before executing a single scenario, decide WHERE the product will run, in this order:

1. Backend, worker or full-stack repository: local. DB rows, logs and third-party stubs are only observable there. First choice start_task_preview — it checks out the task branch in the task workspace, detects how to run it, starts it, stops a stale server, and returns a localhost URL for the browser tools; call it again if the URL isn't there yet. Only when it cannot run the project (e.g. a backend+frontend pair, or a worker it doesn't detect), boot by hand (backend-manual-testing).
2. Frontend-only repository: the task's own preview, else local. When the component is deployed with a per-branch preview environment, get_task_preview returns it — use it when status: "ready" AND built_from_pr_head: true. Anything else (building, built_from_pr_head: false, error, canceled, none, or an empty list) is not the code under test: fall back to start_task_preview.
3. Stage. Used when the repository is configured for stage verification, or when the app cannot boot locally (managed secrets, heavy infra the workspace lacks) — always after confirming stage is actually running the PR head (see Stage rules).

Production is never a test environment. No scenario is ever executed against the prod base URL or prod data — not a "harmless" GET, not seeding, not cleanup.

How to decide

1. Identify the repository shape (backend/worker/full-stack vs frontend-only) and apply the order above.
2. Read the repository settings (list_repositories): build_command, test_command, verify_command — and any comment the developer left on the task (a clean run leaves none). These are the project's declared way of running and checking itself. get_project_brief has the same information plus what each component talks to.
3. If neither a usable preview nor a local boot nor stage works, you are blocked — see below.

Preview rules

• Address. Use open_url in the browser (browser_navigate) — for a protected preview it carries the bypass and sets a cookie, so the pages it links to load too. For HTTP requests (curl, an API client) use branch_url (or url when there is none) and send every header in request_headers on every request.
• Protected without a bypass. When the entry notes that the preview is behind Vercel Deployment Protection with no Protection Bypass for Automation, the preview answers every automated request with a login page. Do not report that page as a failure. Test locally instead and leave one comment asking the human to add "Protection Bypass for Automation" in the Vercel project (Settings → Deployment Protection).
• Never write the bypass secret — nor open_url, which contains it — into a comment, test case, document or bug report: it opens the project's previews to whoever reads it. Refer to "the preview bypass" instead.
• A preview is not prod, but its backing services may be. If the preview talks to a shared database or a third-party account, avoid destructive scenarios there exactly as on stage; they run locally only.
• Put the preview's branch_url and short commit_sha in every review_criterion note — the next reader needs to know exactly what ran, and a pass writes no comment to say it elsewhere.

Stage rules

• Confirm the change is actually there before testing. list_deployments (env=stage) shows the commit of the latest deploy — it must equal the PR head from get_task_pull_request. get_environment says whether stage is bound at all. Not equal → wait for the stage deploy that started when the task entered ready_for_qa (get_pipeline_status shows its progress); never test the old build.
• Stage is shared. Namespace your test data (qa-<task-key>-... prefixes, test-data-and-stubs), avoid destructive scenarios (mass deletes, migrations, load tests) — those run locally only — and clean up what you created.
• The stage base_url is for requests; the health_url is only a liveness probe.

When none is possible

If there is no usable preview, the app cannot boot in the workspace (including via start_task_preview) AND no stage target exists (or it isn't running the PR head), do not fake a verdict and do not read the code as a substitute. Apply the blocker triage (qa-verify-before-verdict): the project cannot boot from its own documented commands → need_revision with the failing command and its output, a developer-fixable defect. A blocker only a human can lift (no stage configured at all, a credential you don't have) → one ask_user question with concrete options, not a need_revision the developer cannot act on.

Red Flags

• A connection string or base URL in your commands points at prod → stop immediately.
• "I'll just check this one thing on prod" → no. Preview, local or stage, always.
• Testing a preview or stage whose commit is not the PR head → the verdict describes an earlier version.