Skip to content
cccskills
FreePublished on GitHubEnglish overview

crabbox

Run installation, integration, and end-to-end tests that follow real user actions in remote environments or Docker. Choose an appropriate isolated environment, inspect failures, record execution details, and clean up resources after testing.

OriginalShow the original description

Crabbox and Blacksmith Testbox remote testing: isolation, cross-platform E2E, diagnostics, cleanup.

Works withAWS
AI rating90/ 100See how to install

Good for

  • Testing package installation in a clean environment
  • Verifying an integration with a real service
  • Checking behavior across macOS, Windows, and WSL2
  • Diagnosing remote test failures and cleaning up resources

Overview

What it does

Use Crabbox and Blacksmith Testbox to verify behavior in remote or clean environments. Tests follow user actions, including package installation, initial setup, CLI and API operations, and sending and receiving data through real services. The workflow covers preparing and reusing environments, syncing code, checking logs and results, isolating failures, and stopping resources afterward. It also guides environment selection according to how much the code can be trusted.

When it helps

Use it to check behavior on a different operating system or test installation without interference from existing settings. It also supports testing external contributors’ code in isolation and verifying connections to real services.

Example requests

  • “Install the package in a clean environment and test actual user operations.”
  • “Check behavior on macOS and Windows.”
  • “Investigate the remote test failure and stop the environments used.”

Notes

You need the Crabbox CLI and a configured execution environment. Depending on the destination, it uses Blacksmith, AWS, Docker, or SSH. Get verification commands and setup instructions from the target repository. Run untrusted code without secrets, and distinguish local Docker runs from remote verification. EC2 Mac is paid; check quotas and other allocation requirements before provisioning.

This overview was written by AI (OpenAI Codex) from the public SKILL.md. Check the original below for the exact behavior.

Objective evaluation

AI rated this skill on four points using only the contents of its SKILL.md. Stars and popularity are not taken into account.

Clear instructionsIs it clear when to use it and what it does?
AGoodIt identifies backend use cases and explicitly covers isolation, testing, diagnostics, and cleanup.
Concrete stepsAre steps, inputs, outputs, and examples spelled out?
AGoodIt provides preflight, environment reuse, execution, and stop commands, plus required reporting fields.
SafetyAre sending data, deleting files, and running commands disclosed?
AGoodIt specifies secret isolation for untrusted code, authority boundaries, and confirmation that environments have stopped.
Easy setupDoes it need few tools or API keys, and are they documented?
BPartly lackingCLI and configuration checks are provided, but installation and environment preparation depend on consumer documentation.

The total counts each point as A = 25, B = 15, C = 5. Rated by AI (OpenAI Codex) as of Oct 19, 2026. Treat it as a guide and read the original before installing.

Disagree? Request a re-evaluation

Tell us which point is wrong and why. No login needed. We will check it against the SKILL.md, redo the evaluation, and show the result here.

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

Files included (2)

  • SKILL.md9.6 KB
  • references/operations.md6.6 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Crabbox

  • Deliberate remote proof: follow proof policy. Prefer bounded local proof on a qualified, available host. Expensive or platform-specific work can use remote capacity. Queued: stop owned allocation; use PR CI or a safe local scope and report uncovered requirements.
  • Never wait on queues or chain provider fallbacks.

Backends:

  • blacksmith-testbox: trusted maintainer source. Prepared CI. tbx_....
  • aws: direct brokered Crabbox. Fresh PRs. Custom sync/env/capture. cbx_....
  • local-container: Docker fallback. Not remote proof.
  • ssh: existing operator host. macOS/Windows/WSL2.

Always report provider, id, run URL, command, result. Never call Testbox “AWS Crabbox.”

Repository Contract

  • Resolve example commands, bootstrap, container image, PR from trusted consumer instructions. Never invent commands or copy another consumer's setup.
  • Consumer-specific setup stays outside portable skill. See placeholder definitions.

Authorization and Isolation

  • Task-needed creation, reuse, cleanup, temporary state, clean checkouts: pre-approved. Preserve existing branches, checkouts, unrelated edits.
  • Source trust, credentials, production access, budget, publication remain authority boundaries. Isolated state; never touch operator Gateway without per-task approval.

Route First

  • Preserve resolved provider: Testbox for trusted source; direct AWS when its semantics required; secretless CI/AWS for untrusted code. Never run untrusted wrappers/config locally or hydrate untrusted lease.
  • Acquire when command ready; reuse owned ID; stop afterward.
  • Smallest proven profile. Larger: recorded memory evidence or measured total-cost benefit required, not generic failure/delay/timeout. Do not raise worker counts to compensate.
  • Testbox sizing: --blacksmith-workflow, never direct --class/--type; profile changes require fresh leases.
  • Blacksmith runner labels request a tier, not guaranteed CPU or RAM. Blacksmith can allocate smaller machines depending on service availability. For resource-sensitive tests or benchmarks, measure available CPUs, affinity/quotas, and usable RAM on the allocated machine. Size workers and workloads from those measurements. Record the requested tier and observed resources separately. Do not qualify hardware or performance from labels alone.
  • Size, duration, hydration failure, capacity never authorize provider overrides; explicit user selection or required backend semantics do.

Preflight

Trusted repo root: read .crabbox.yaml, resolve installed binary, inspect provider/commands:

export CRABBOX="$(command -v crabbox)"
test -n "$CRABBOX"
"$CRABBOX" --version
"$CRABBOX" config show --json | jq '{provider, profile, target}'
"$CRABBOX" run --help
  • Use consumer's trusted wrapper/install instructions.
  • Repairs: verify upstream; use clean task-owned checkout; preserve installed binary + unrelated source. Never trust sibling checkout merely by name.

Lease lifetime

  • TTL counts from creation, including hydration; idle timeout counts inactivity. Earlier deadline wins. Defaults: 30m each, 4h maximum. Owner/org overrides can allow up to 24h; use effective policy and sanitized clamp notices, not requested durations. Policy tightening also applies to live leases.
  • --keep and stop: never do not prevent expiry. Request explicit bounded TTL and idle for retained direct leases. Plain heartbeat, setup, sync, SSH, and desktop use do not extend total TTL; run requests bounded auto-extension only while a command executes, within policy, budget, and provider capability.
  • Before long source-dependent work, inspect remaining total lifetime. If allowed, request one bounded extend <id> --ttl <desired-total> --json; extension does not reset idle. Denied/budget/unsupported/nonadvancing/inactive outcomes require a visible skip or failure, never provider switching or infinite lease chains. Creation-fixed providers such as Daytona need enough TTL at allocation.
  • Direct guest-refresh failure can leave a longer provider deadline with the old on-box deadline. Keep the same local state and retry the same fixed total TTL only to reconcile; do not treat the failed extension as protection.
  • Honor trusted directLeases: deny and directLeaseAllow. A denied provider requires the configured coordinator route; never bypass with direct credentials. Testbox retains its native job and idle limits; --ttl does not extend a delegated workflow.
  • list shows live coordinator leases; list --retained adds kept terminal resources whose cleanup is unconfirmed. Use exact-ID inspect/stop for recovery. Retain claims and recovery state until a blocking stop confirms completion.

Trusted Testbox

Resolved provider = Testbox (or explicitly requested): commands preserve it. Add --provider blacksmith-testbox only for explicit override:

"$CRABBOX" run --timing-json -- CI=1 <check-command>

Several commands: warm once, save id, reuse, stop.

"$CRABBOX" warmup --keep --timing-json
"$CRABBOX" run --id <tbx_id> --timing-json -- <check-command>
"$CRABBOX" stop <tbx_id>

Rules:

  • One lease, one active command. No sync/reclaim during run.
  • Prefer bash -c; login profiles can change directories. In executing shell, assert physical checkout + expected source/patch; HEAD alone ≠ dirty-sync proof. Follow consumer's documented hydration shell contract.
  • Native Testbox runs own sync, including reused --id runs. Never rely on --no-sync for remote baseline: Blacksmith has no native bypass; released Crabbox versions can silently ignore flag. Unchanged intentional rerun ≠ exception.
  • --reclaim only for deliberate checkout-path ownership transfer.
  • Base/head change: stop; rewarm. No stale-lease override.
  • Raw SHA unreliable for warmup --ref; use branch/tag.
  • blacksmith testbox list hides states. Use list --all or status --id <tbx_id>.
  • Testbox status/stop: --id. No status --json.
  • Delegated provider rejects --fresh-pr, --full-resync, --script*, --env-helper, capture/download flags.

Blacksmith directory downloads

Recursive downloads + required remote trailing slash: directory downloads.

Untrusted AWS

  • Clean trusted default-branch checkout. Installed trusted Crabbox binary.
  • Fresh lease per reviewed full head SHA. No instance role, Tailscale, hydration.
  • Forward only CI. Upload trusted bootstrap beside --fresh-pr.
cd <clean-trusted-default-branch-checkout>
env -u CRABBOX_AWS_INSTANCE_PROFILE \
  "$CRABBOX" config show --json | \
  jq -e '.aws.instanceProfile == ""' >/dev/null

env -u CRABBOX_AWS_INSTANCE_PROFILE \
  -u CRABBOX_TAILSCALE \
  -u CRABBOX_TAILSCALE_AUTH_KEY \
  -u CRABBOX_TAILSCALE_AUTH_KEY_ENV \
  -u CRABBOX_TAILSCALE_EXIT_NODE \
  -u CRABBOX_TAILSCALE_EXIT_NODE_ALLOW_LAN_ACCESS \
  -u CRABBOX_TAILSCALE_HOSTNAME_TEMPLATE \
  -u CRABBOX_TAILSCALE_TAGS \
  "$CRABBOX" warmup \
  --provider aws --network public --tailscale=false \
  --tailscale-exit-node= \
  --tailscale-exit-node-allow-lan-access=false \
  --ttl 4h --idle-timeout 90m --keep --timing-json

"$CRABBOX" inspect --provider aws --id <cbx_id> --json | \
  jq -e '.network == "public" and .tailscale == null' >/dev/null

env -u CRABBOX_AWS_INSTANCE_PROFILE \
  CRABBOX_ENV_ALLOW=CI \
  "$CRABBOX" run \
  --provider aws --id <cbx_id> \
  --fresh-pr <owner/repo#number> \
  --no-hydrate --timing-json \
  --script <trusted-bootstrap-script> -- \
  <expected_full_head_sha> <check-command>

env -u CRABBOX_AWS_INSTANCE_PROFILE \
  "$CRABBOX" stop --provider aws <cbx_id>
  • Trusted bootstrap must prove IMDSv2 IAM credentials return 404; verify SHA; remove runtime injection variables; pin toolchain; isolate HOME before install/test.
  • Changed head: fresh lease. Missing remote PR or no-role proof: secretless fork CI.

Direct AWS

  • Trusted AWS commands, auth diagnostics, resync, clean PR/container runs: read direct providers.
  • Preserve --no-hydrate on container runs; report local-container proof as local.

Observability

  • Collecting artifacts/injecting credentials: read capture and environment flags.
  • Exact key, one command. Never print secrets, write in repo, or put in shell history.
  • No safe injection path: live auth blocked; fake-key proof insufficient.

Real E2E

  • Exercise requested user entry point with isolated consumer state, not just remote unit tests.
  • For install/provider/interactive proof + safe revision comparison: read E2E procedure.

Desktop / Cross-OS

SSH targets, paid macOS preflight, WebVNC, app handoff verification: read desktop and cross-OS commands.

Failure Triage

Identify wrapper, provider, hydration, sync, SSH, or command failure:

"$CRABBOX" doctor
"$CRABBOX" status --id <id>
"$CRABBOX" inspect --id <id> --json
"$CRABBOX" logs <run_id>
"$CRABBOX" results <run_id>
blacksmith testbox list --all
blacksmith testbox status --id <tbx_id>
  • Queued capacity: stop owned allocations; follow proof policy. No retry storm or provider switching.
  • Stale sync: --debug --timing-json, then --full-resync once on direct providers.
  • Read failed phase/JUnit before focused rerun. Broken wrapper: diagnose with installed CLI on same provider; update through consumer's trusted path.
  • Stop only owned IDs; stop takes no --timing-json.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

1password

FreeSummaryAI rating 100

Set up and authenticate the 1Password CLI, then supply stored passwords and API keys to commands or configuration. The skill covers desktop integration, service accounts, and interactive sign-in while keeping secrets out of chat and logs.

  • Setting up the 1Password CLI
  • Supplying API keys to commands
  • Using service account authentication in CI
openclaw/openclaw392.4KUpdated Oct 18, 2026

acp-router

FreeSummaryAI rating 80

Route requests from OpenClaw to external coding agents such as Claude Code or Cursor. Start or continue sessions using ACP, a protocol for connecting agents, or relay prompts directly through acpx while preserving conversation context.

  • Starting a Claude Code session in a thread
  • Continuing work with an external coding agent
  • Relaying prompts directly through acpx
openclaw/openclaw392.4KUpdated Oct 18, 2026

add-model-provider

FreeSummaryAI rating 100

Add a model provider to OpenClaw and verify it with a real response. The skill validates configuration, keeps credentials in environment or file references, and reports test results and any default model changes.

  • Adding a model provider to OpenClaw
  • Configuring credentials through environment or file references
  • Checking whether a model returns a real response
openclaw/openclaw392.4KUpdated Oct 18, 2026

agent-transcript

FreeSummaryAI rating 90

Prepare an agent activity transcript for a GitHub pull request or issue. Extract relevant prompts, decisions, and results from local logs, remove sensitive material, and preview the edited record before separately authorized publication.

  • Preparing an agent activity record for a pull request
  • Extracting task-relevant prompts and results from local logs
  • Previewing a transcript with sensitive information removed
openclaw/openclaw392.4KUpdated Oct 18, 2026

apple-notes

FreeSummaryAI rating 100

Manage Apple Notes on macOS through memo, a command-line tool. Create, browse, search, edit, delete, and move notes between folders, or export selected notes to HTML or Markdown for use outside the app.

  • Creating notes with titles
  • Finding notes by folder or approximate search
  • Editing notes and organizing them into folders
openclaw/openclaw392.4KUpdated Oct 18, 2026

apple-reminders

FreeSummaryAI rating 70

Manage Apple Reminders from your Mac with remindctl, a terminal tool. View tasks by date, add or edit reminders, organize lists, and mark tasks complete or delete them, including personal to-dos that sync to iOS.

  • Review today’s tasks and overdue reminders
  • Add personal to-dos with due dates
  • Organize tasks visible on an iPhone or iPad
openclaw/openclaw392.4KUpdated Oct 18, 2026

See all skills by openclaw

Report a problem with this skill