Run and troubleshoot OpenClaw QA scenarios using simulated responses or live services. Inspect reports and saved outputs, extend test scenarios, and compare response styles across models through character evaluations with rankings and transcripts.
Run, watch, debug, extend, or explain OpenClaw qa-lab and qa-channel scenarios, artifacts, and live lanes.
This skill uses OpenClaw’s qa-lab and qa-channel to run, investigate, and add quality assurance scenarios. You can choose an environment with simulated responses or validation against live services, then inspect success and failure counts, reports, and saved evidence.
When a scenario fails, it investigates the product or test-running system, applies a fix, and reruns the relevant full test suite. It can also compare how multiple models speak and portray a character, producing evaluation results that include rankings and complete conversation transcripts.
It suits developers who want to check OpenClaw through a sequence of usage scenarios, as well as people comparing response styles across models. You can also use it to validate OpenTelemetry trace data locally without credentials for an external service.
This skill is intended for a development environment with a source checkout. Standard builds do not include QA features. Live service testing requires the relevant credentials, and WhatsApp testing uses dedicated test accounts. Credentials must be kept out of logs and images.
This overview was written by AI (OpenAI Codex) from the public SKILL.md. Check the original below for the exact behavior.
AI rated this skill on four points using only the contents of its SKILL.md. Stars and popularity are not taken into account.
The total counts each point as A = 25, B = 15, C = 5. Rated by AI (OpenAI Codex) as of Oct 19, 2026. Treat it as a guide and read the original before installing.
Review the instructions the agent will receive before you install.
Use this skill for qa-lab / qa-channel work. Repo-local QA only.
docs/concepts/qa-e2e-automation.mddocs/help/testing.mddocs/channels/qa-channel.mdqa/README.mdqa/scenarios/index.yamlextensions/qa-lab/src/suite.tsextensions/qa-lab/src/character-eval.ts--model, --alt-model, or --fast by default. Omitted
--fast does not mean fast is disabled; fast behavior is source-owned.execution.summary is authoritative
and overrides generic default guidance, including when it requires other
flags. Add explicit provider/model pins only when
execution.config.requiredProvider or requiredModel requires them.mock-openailive-frontierpnpm openclaw qa suite \
--provider-mode live-frontier \
--output-dir .artifacts/qa-e2e/run-all-live-frontier-<tag>
.artifacts/qa-e2e/run-all-live-frontier-<tag>/qa-suite-summary.json.artifacts/qa-e2e/run-all-live-frontier-<tag>/qa-suite-report.mdopenclaw-qa listen port and report http://127.0.0.1:<port>.For local QA-lab OpenTelemetry validation, use:
pnpm qa:otel:smoke
This starts a local OTLP/HTTP trace receiver, runs the otel-trace-smoke
scenario through qa-channel, decodes the emitted protobuf spans, and verifies
the exported trace names and privacy contract. It does not require Opik,
Langfuse, or external collector credentials.
op only inside tmux for QA secret lookup in this repo.op account list
OpenClawTelegram E2EOPENCLAW_QA_TELEGRAM_DRIVER_BOT_TOKENOPENCLAW_QA_TELEGRAM_SUT_BOT_TOKENOPENCLAW_QA_PROVIDER_MODEOPENCLAW_NPM_TELEGRAM_PACKAGE_SPECOpenClawOPENCLAW_QA_CONVEX_SITE_URLOPENCLAW_QA_CONVEX_SECRET_MAINTAINEROPENCLAW_QA_CONVEX_SECRET_CIPrivateOPENCLAW QA, Convex, TelegramOPENCLAW_QA_TELEGRAM_GROUP_ID may be stored separately from Telegram E2EOPENCLAW_QA_TELEGRAM_GROUP_IDOpenClaw/OPENCLAW_QA_CONVEX_SITE_URL; if that is stale or unclear, ask for the active pool URL before runningOPENCLAW_QA_TELEGRAM_GROUP_ID="..." \
OPENCLAW_QA_TELEGRAM_DRIVER_BOT_TOKEN="..." \
OPENCLAW_QA_TELEGRAM_SUT_BOT_TOKEN="..." \
OPENCLAW_QA_PROVIDER_MODE="mock-openai" \
OPENCLAW_NPM_TELEGRAM_PACKAGE_SPEC="openclaw@beta" \
pnpm test:docker:npm-telegram-live
scripts/e2e/npm-telegram-live-runner.ts reads OPENCLAW_NPM_TELEGRAM_PROVIDER_MODEOPENCLAW_QA_PROVIDER_MODE is consumed by that wrapperOPENCLAW_QA_PROVIDER_MODE, map it explicitly to OPENCLAW_NPM_TELEGRAM_PROVIDER_MODE before running the Docker lanepnpm test:docker:npm-telegram-live passed with:
OPENCLAW_QA_CREDENTIAL_SOURCE=convexOPENCLAW_QA_CREDENTIAL_ROLE=maintainerOPENCLAW_QA_CONVEX_SITE_URLOPENCLAW_QA_CONVEX_SECRET_MAINTAINEROPENCLAW_NPM_TELEGRAM_PROVIDER_MODE=mock-openaiop signin blocks, prefer dispatching the manual GitHub lane because the qa-live-shared environment already has Convex CI credentials:gh workflow run "NPM Telegram Beta E2E" --repo openclaw/openclaw --ref main \
-f package_spec=openclaw@YYYY.M.D-beta.N \
-f package_label=openclaw@YYYY.M.D-beta.N \
-f provider_mode=mock-openai
gh run view --json artifacts is not supported; list artifacts with:gh api repos/openclaw/openclaw/actions/runs/<run-id>/artifacts
Use this when setting up or replacing Convex kind=whatsapp credentials.
groupJid; otherwise the group mention-gating scenario should be skipped by default and fail when explicitly requested..tgz payload fields and add a new active Convex credential row. Prefer adding a fresh row and disabling stale/broken rows over overwriting credentials in place.driverPhoneE164, sutPhoneE164, driverAuthArchiveBase64, sutAuthArchiveBase64, and optional groupJid.pnpm openclaw qa whatsapp --credential-source convex --credential-role maintainer --provider-mode mock-openai and preserve artifact paths plus redacted pass/fail summaries.Use qa character-eval for style/persona/vibe checks across multiple live models.
pnpm openclaw qa character-eval \
--output-dir .artifacts/qa-e2e/character-eval-<tag>
pnpm build omits QA Lab + qa-channel by design (source-checkout
only). To exercise openclaw qa/qa-channel from a built dist, build with
OPENCLAW_BUILD_PRIVATE_QA=1 pnpm build (emits dist/plugin-sdk/qa-lab.js,
qa-runtime.js, dist/extensions/{qa-lab,qa-channel}) or run via pnpm dev.--model provider/model,thinking=<level>[,fast|,no-fast|,fast=<bool>]
or --judge-model ... only to replace the corresponding inventory explicitly.--model-thinking; keep that flag as legacy compatibility only.--concurrency <n> and --judge-concurrency <n> to override when local gateways or provider limits need a gentler lane.qa/scenarios/: use index.yaml and
per-scenario *.yaml files with top-level title, scenario, and optional
flow. Never add fenced qa-scenario / qa-flow Markdown files.SOUL.md and blank IDENTITY.md in the scenario flow. Use SOUL.md + IDENTITY.md only when intentionally testing how the normal OpenClaw identity combines with the character.SOUL.md, then normal user turns such as chat, workspace help, and small file tasks; do not ask "how would you react?" or tell the model it is in an eval.Use model refs shaped like codex-cli/<codex-model> whenever QA should exercise Codex as a model backend.
Examples:
pnpm openclaw qa suite \
--provider-mode live-frontier \
--model codex-cli/<codex-model> \
--alt-model codex-cli/<codex-model> \
--scenario <scenario-id> \
--output-dir .artifacts/qa-e2e/codex-<tag>
pnpm openclaw qa manual \
--model codex-cli/<codex-model> \
--message "Reply exactly: CODEX_OK"
CODEX_HOME so Codex CLI auth/config works while keeping HOME and OPENCLAW_HOME sandboxed.CODEX_HOME.CODEX_HOME,
relevant secret-backed auth, and gateway child logs before changing
scenario assertions.codex-cli/<codex-model> as another candidate in qa character-eval; the report should label it as an opaque model name.qa/scenarios/index.yaml and
qa/scenarios/<theme>/*.yaml.extensions/qa-lab/src/suite.tsextensions/qa-lab/src/lab-server.tsextensions/qa-lab/src/gateway-child.tsextensions/qa-channel/extensions/qa-lab/src/suite.tsrepo/... inside seeded workspaceqa/scenarios/; do not add .md scenario
files or fenced YAML blocks.qa/scenarios/index.yaml alignedextensions/qa-lab/src/suite.ts.artifacts/qa-e2e/No reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Set up and authenticate the 1Password CLI, then supply stored passwords and API keys to commands or configuration. The skill covers desktop integration, service accounts, and interactive sign-in while keeping secrets out of chat and logs.
Route requests from OpenClaw to external coding agents such as Claude Code or Cursor. Start or continue sessions using ACP, a protocol for connecting agents, or relay prompts directly through acpx while preserving conversation context.
Add a model provider to OpenClaw and verify it with a real response. The skill validates configuration, keeps credentials in environment or file references, and reports test results and any default model changes.
Prepare an agent activity transcript for a GitHub pull request or issue. Extract relevant prompts, decisions, and results from local logs, remove sensitive material, and preview the edited record before separately authorized publication.
Manage Apple Notes on macOS through memo, a command-line tool. Create, browse, search, edit, delete, and move notes between folders, or export selected notes to HTML or Markdown for use outside the app.
Manage Apple Reminders from your Mac with remindctl, a terminal tool. View tasks by date, add or edit reminders, organize lists, and mark tasks complete or delete them, including personal to-dos that sync to iOS.