Skip to content
cccskills
FreePublished on GitHub

langbot-testing

Test LangBot through its WebUI and compare visible results with backend logs or network behavior. Use it to verify chat, model settings, bots, knowledge bases, and plugins, with recorded evidence for test results.

OriginalShow the original description

Test LangBot WebUI and core product flows with an automated browser and backend logs. Use when validating the configured LangBot frontend, pipeline Debug Chat, model provider setup and test buttons, bot and knowledge-base UI flows, or troubleshooting failed LangBot end-to-end tests.

langbot-app/LangBot
See how to install

Good for

  • Verify pipeline replies in Debug Chat
  • Check model provider setup and test buttons
  • Test bot, knowledge base, and plugin UI flows
  • Investigate failed tests and compile evidence

Files included (183)

  • SKILL.md3.9 KB
  • cases/acp-agent-runner-debug-chat.yaml3.1 KB
  • cases/agent-run-ledger-audit.yaml2.1 KB
  • cases/agent-runner-async-db-readiness.yaml1.2 KB
  • cases/agent-runner-behavior-matrix.yaml1.1 KB
  • cases/agent-runner-fixture-contract.yaml1.2 KB
  • cases/agent-runner-health-visibility.yaml2.0 KB
  • cases/agent-runner-ledger-concurrency.yaml1.6 KB
  • cases/agent-runner-ledger-contention.yaml1.1 KB
  • cases/agent-runner-ledger-invariants.yaml1.3 KB
  • cases/agent-runner-ledger-stress.yaml1.0 KB
  • cases/agent-runner-live-install.yaml1.7 KB
  • cases/agent-runner-qa-debug-chat.yaml2.3 KB
  • cases/agent-runner-release-preflight.yaml3.5 KB
  • cases/agent-runner-runtime-chaos.yaml1.5 KB
  • cases/bot-event-routing-product-flow.yaml2.9 KB
  • cases/box-mcp-heartbeat-recovery.yaml4.2 KB
  • cases/claude-code-agent-debug-chat.yaml3.6 KB
  • cases/codex-agent-debug-chat.yaml3.3 KB
  • cases/dify-agent-debug-chat.yaml2.1 KB
  • cases/langbot-fake-provider-debug-chat-cross-pipeline-isolation.yaml3.8 KB
  • cases/langbot-fake-provider-debug-chat-fault-recovery.yaml4.4 KB
  • cases/langbot-fake-provider-debug-chat-load.yaml3.4 KB
  • cases/langbot-fake-provider-debug-chat-slow-load.yaml3.7 KB
  • cases/langbot-fault-taxonomy-contract.yaml1.5 KB
  • cases/langbot-live-backend-latency.yaml1.6 KB
  • cases/langbot-live-backend-log-health.yaml2.0 KB
  • cases/langbot-live-control-plane-api.yaml2.0 KB
  • cases/langbot-overhead-accounting-contract.yaml1.7 KB
  • cases/langbot-space-debug-chat-concurrency-smoke.yaml3.4 KB
  • cases/langrag-kb-retrieve.yaml2.2 KB
  • cases/langrag-parser-golden-e2e.yaml3.5 KB
  • cases/langrag-sentinel-kb-discover.yaml1.4 KB
  • cases/local-agent-basic-debug-chat.yaml2.6 KB
  • cases/local-agent-combo-rag-compaction-tool-debug-chat.yaml7.4 KB
  • cases/local-agent-complex-coding-task-debug-chat.yaml5.4 KB
  • cases/local-agent-context-compaction-debug-chat.yaml7.7 KB
  • cases/local-agent-effective-prompt-debug-chat.yaml3.0 KB
  • cases/local-agent-model-fallback-before-first-chunk-debug-chat.yaml3.5 KB
  • cases/local-agent-multimodal-debug-chat.yaml3.0 KB
  • cases/local-agent-multitool-rag-compaction-debug-chat.yaml7.2 KB
  • cases/local-agent-nonstreaming-debug-chat.yaml2.5 KB
  • cases/local-agent-parallel-tools-rag-compaction-debug-chat.yaml7.4 KB
  • cases/local-agent-plugin-tool-call-debug-chat.yaml3.6 KB
  • cases/local-agent-rag-debug-chat.yaml3.6 KB
  • cases/local-agent-rag-multimodal-debug-chat.yaml3.1 KB
  • cases/local-agent-steering-debug-chat.yaml3.7 KB
  • cases/local-agent-streaming-post-commit-failure-debug-chat.yaml3.9 KB
  • cases/local-agent-tool-error-recovery-debug-chat.yaml4.4 KB
  • cases/local-agent-tool-loop-limit-debug-chat.yaml4.2 KB
  • cases/mcp-stdio-register.yaml2.4 KB
  • cases/mcp-stdio-tool-call.yaml4.8 KB
  • cases/pipeline-debug-chat-performance.yaml2.9 KB
  • cases/pipeline-debug-chat.yaml2.2 KB
  • cases/plugin-e2e-smoke.yaml2.9 KB
  • cases/provider-deepseek.yaml1.5 KB
  • cases/qa-plugin-smoke-live-install.yaml1.7 KB
  • cases/sandbox-skill-authoring-e2e.yaml1.5 KB
  • cases/sandbox-skill-authoring-edit-existing-e2e.yaml1.8 KB
  • cases/skill-discovery-via-mcp-gateway.yaml3.0 KB
  • cases/webui-login-state.yaml1.3 KB
  • cases/wizard-onebot-agent-runtime.yaml2.4 KB
  • cases/wizard-runner-marketplace-catalog.yaml2.4 KB
  • cases/wizard-scenario-routing.yaml2.2 KB
  • cases/workspace-compatibility-preflight.yaml1.6 KB
  • cases/workspace-plugin-pages-smoke.yaml2.0 KB
  • cases/workspace-repository-contracts.yaml1.5 KB
  • fixtures/agent-runner/qa-runner-behaviors.json1.1 KB
  • fixtures/complex-agent-task/verify.py2.8 KB
  • fixtures/complex-agent-task/workspace/acceptance_check.py946 B
  • fixtures/complex-agent-task/workspace/order_engine/__init__.py213 B
  • fixtures/complex-agent-task/workspace/order_engine/inventory.py613 B
  • fixtures/complex-agent-task/workspace/order_engine/models.py541 B
  • fixtures/complex-agent-task/workspace/order_engine/pricing.py788 B
  • fixtures/complex-agent-task/workspace/order_engine/report.py544 B
  • fixtures/complex-agent-task/workspace/order_engine/service.py910 B
  • fixtures/complex-agent-task/workspace/TASK.md2.1 KB
  • fixtures/complex-agent-task/workspace/tests/test_order_engine.py4.3 KB
  • fixtures/fixtures.json3.5 KB
  • fixtures/mcp/qa_mcp_echo_server.py3.3 KB
  • fixtures/multimodal/red-pixel.png.base6497 B
  • fixtures/multimodal/red-square.png.base64185 B
  • fixtures/plugins/qa-agent-runner/assets/icon.svg301 B
  • fixtures/plugins/qa-agent-runner/components/runner/default.py1.3 KB
  • fixtures/plugins/qa-agent-runner/components/runner/default.yaml690 B
  • fixtures/plugins/qa-agent-runner/dist/qa-agent-runner-0.1.0.lbpkg4.3 KB
  • fixtures/plugins/qa-agent-runner/main.py224 B
  • fixtures/plugins/qa-agent-runner/manifest.yaml549 B
  • fixtures/plugins/qa-agent-runner/README.md410 B
  • fixtures/plugins/qa-plugin-smoke/.gitignore48 B
  • fixtures/plugins/qa-plugin-smoke/assets/icon.svg322 B
  • fixtures/plugins/qa-plugin-smoke/components/events/prompt_probe.py2.5 KB
  • fixtures/plugins/qa-plugin-smoke/components/events/prompt_probe.yaml214 B
  • fixtures/plugins/qa-plugin-smoke/components/pages/smoke/i18n/en_US.json3 B
  • fixtures/plugins/qa-plugin-smoke/components/pages/smoke/i18n/zh_Hans.json3 B
  • fixtures/plugins/qa-plugin-smoke/components/pages/smoke/index.html1.3 KB
  • fixtures/plugins/qa-plugin-smoke/components/pages/smoke/smoke.py469 B
  • fixtures/plugins/qa-plugin-smoke/components/pages/smoke/smoke.yaml322 B
  • fixtures/plugins/qa-plugin-smoke/components/tools/qa_echo.py469 B
  • fixtures/plugins/qa-plugin-smoke/components/tools/qa_echo.yaml515 B
  • fixtures/plugins/qa-plugin-smoke/components/tools/qa_plugin_echo.py475 B
  • fixtures/plugins/qa-plugin-smoke/components/tools/qa_plugin_echo.yaml589 B
  • fixtures/plugins/qa-plugin-smoke/components/tools/qa_plugin_fail.py518 B
  • fixtures/plugins/qa-plugin-smoke/components/tools/qa_plugin_fail.yaml667 B
  • fixtures/plugins/qa-plugin-smoke/components/tools/qa_plugin_sleep.py848 B
  • fixtures/plugins/qa-plugin-smoke/components/tools/qa_plugin_sleep.yaml757 B
  • fixtures/plugins/qa-plugin-smoke/dist/qa-plugin-smoke-0.1.0.lbpkg13.8 KB
  • fixtures/plugins/qa-plugin-smoke/main.py221 B
  • fixtures/plugins/qa-plugin-smoke/manifest.yaml752 B
  • fixtures/plugins/qa-plugin-smoke/README.md383 B
  • fixtures/rag/parser-golden.html817 B
  • fixtures/rag/sentinel-doc.txt259 B
  • probes/agent-runner-async-db-readiness.mjs5.5 KB
  • probes/agent-runner-behavior-matrix.mjs8.4 KB
  • probes/agent-runner-fixture-contract.mjs8.1 KB
  • probes/agent-runner-ledger-concurrency.mjs1.0 KB
  • probes/agent-runner-ledger-contention.mjs8.7 KB
  • probes/agent-runner-ledger-invariants.mjs7.8 KB
  • probes/agent-runner-ledger-stress.mjs7.4 KB
  • probes/agent-runner-runtime-chaos.mjs455 B
  • probes/box-mcp-heartbeat-recovery.mjs15.3 KB
  • probes/langbot-debug-chat-concurrency.mjs30.8 KB
  • probes/langbot-debug-chat-cross-pipeline-isolation.mjs30.1 KB
  • probes/langbot-fault-taxonomy-contract.mjs5.8 KB
  • probes/langbot-live-backend-latency.mjs7.8 KB
  • probes/langbot-live-backend-log-health.mjs8.1 KB
  • probes/langbot-live-control-plane-api.mjs11.1 KB
  • probes/langbot-overhead-accounting-contract.mjs6.2 KB
  • probes/lib/fake-provider-timing.mjs6.6 KB
  • probes/pytest-probe.mjs8.1 KB
  • references/agent-runner-qa-workflow.md8.1 KB
  • references/agent-runner-release-gate.md10.6 KB
  • references/dify-agent-runner.md2.1 KB
  • references/langrag-knowledge-base.md5.9 KB
  • references/local-agent-runner-coverage.md11.8 KB
  • references/local-agent-runner.md6.4 KB
  • references/mcp-stdio-testing.md4.0 KB
  • references/model-provider-testing.md951 B
  • references/performance-reliability-testing.md12.6 KB
  • references/pipeline-debug-chat.md1.7 KB
  • references/plugin-e2e-smoke.md2.4 KB
  • references/sandbox-skill-authoring.md3.3 KB
  • references/skill-all-tool-acceptance.md10.1 KB
  • references/troubleshooting.md4.5 KB
  • references/web-ui-testing.md1.4 KB
  • references/workspace-release-testing.md2.2 KB
  • suites/agent-runner-release-gate.yaml1.5 KB
  • suites/core-smoke.yaml313 B
  • suites/eba-release-gate.yaml456 B
  • suites/langbot-debug-chat-isolation-gate.yaml464 B
  • suites/langbot-debug-chat-load-gate.yaml697 B
  • suites/langbot-live-backend-gate.yaml413 B
  • suites/langbot-performance-contract-gate.yaml384 B
  • suites/langbot-performance-reliability-gate.yaml521 B
  • suites/langbot-user-path-performance-gate.yaml329 B
  • suites/langbot-workspace-contract-gate.yaml553 B
  • suites/langbot-workspace-release-gate.yaml850 B
  • suites/local-agent-gate.yaml1.1 KB
  • troubleshooting/agent-runner-actor-context-fields.yaml1.2 KB
  • troubleshooting/aiosqlite-connect-hangs.yaml1.4 KB
  • troubleshooting/ambiguous-runner-default-label.yaml1.2 KB
  • troubleshooting/backend-not-listening.yaml1.4 KB
  • troubleshooting/box-runtime-silent-action-timeout.yaml1.6 KB
  • troubleshooting/box-session-conflict-logical-metadata.yaml1.4 KB
  • troubleshooting/debug-chat-history-contaminates-automation.yaml1.5 KB
  • troubleshooting/dynamic-form-missing-config-id.yaml983 B
  • troubleshooting/e2b-extra-mount-sync-missing.yaml1.5 KB
  • troubleshooting/local-agent-mcp-text-result-empty-follow-up.yaml1.6 KB
  • troubleshooting/local-agent-model-route-unavailable.yaml2.8 KB
  • troubleshooting/marketplace-network-flaky.yaml1.0 KB
  • troubleshooting/mcp-stdio-args-not-applied.yaml2.4 KB
  • troubleshooting/nsjail-cli-compatibility.yaml1.7 KB
  • troubleshooting/pipeline-form-controlled-warning.yaml1.0 KB
  • troubleshooting/plugin-dependency-install-offline.yaml1.7 KB
  • troubleshooting/plugin-runtime-timeout.yaml1.7 KB
  • troubleshooting/provider-image-parse-error.yaml1.4 KB
  • troubleshooting/proxy-env-mismatch.yaml1.1 KB
  • troubleshooting/sandbox-native-tools-unavailable.yaml1.8 KB
  • troubleshooting/socks-proxy-without-socksio.yaml1.3 KB
  • troubleshooting/survey-widget-blocks-debug-chat.yaml1022 B
  • troubleshooting/telemetry-proxy-noise.yaml1.4 KB
  • troubleshooting/tool-name-collision-between-mcp-and-plugin.yaml1.7 KB
  • troubleshooting/uv-run-resyncs-local-sdk.yaml1.4 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

LangBot Testing

Use this skill when an agent needs to verify LangBot behavior through the WebUI instead of only reading code.

Routing

  • General WebUI testing: read references/web-ui-testing.md.
  • Pipeline Debug Chat: read references/pipeline-debug-chat.md.
  • Dify Runner: read references/dify-agent-runner.md.
  • Model provider setup or test button: read references/model-provider-testing.md.
  • Plugin install/runtime/tool/page smoke: read references/plugin-e2e-smoke.md.
  • Local Runner: read references/local-agent-runner.md.
  • Local Runner path coverage: read references/local-agent-runner-coverage.md.
  • Diff-aware Runner QA after code changes: read references/agent-runner-qa-workflow.md.
  • Runner release gate: read references/agent-runner-release-gate.md.
  • Sandbox-backed skill authoring: read references/sandbox-skill-authoring.md.
  • LangRAG knowledge bases: read references/langrag-knowledge-base.md.
  • MCP stdio tool testing: read references/mcp-stdio-testing.md.
  • Performance, reliability, or chaos probes: read references/performance-reliability-testing.md.
  • Cross-repository workspace and release gates: read references/workspace-release-testing.md.
  • Drive a live instance over MCP (not raw HTTP): use the langbot-mcp-ops skill — the instance exposes an MCP server at http://<host>:5300/mcp (reuses API keys). Useful for setting up bots/pipelines/models as test fixtures programmatically.
  • Known failures and fixes: read references/troubleshooting.md.
  • Reusable test groups: run bin/lbs suite list and bin/lbs suite plan <suite-id> before manually assembling a case set.

Rules

  • Read ../.env first and use LANGBOT_FRONTEND_URL and LANGBOT_BACKEND_URL instead of hardcoded ports.
  • If a standalone frontend dev server is running, LANGBOT_FRONTEND_URL may point to LANGBOT_DEV_FRONTEND_URL; otherwise it may point to the backend WebUI.
  • Confirm the backend and frontend are actually running before testing.
  • Run bin/lbs fixture check before fixture-heavy MCP, RAG, multimodal, or plugin smoke tests.
  • For runner externalization release checks, run bin/lbs test run agent-runner-release-preflight before the full agent-runner-release-gate suite so configuration blockers are separated from product failures.
  • Read Manual Readiness in bin/lbs test plan <case-id>; manual_check means the declared preconditions or setup still need operator confirmation for this run.
  • Use an authenticated browser profile prepared by langbot-env-setup.
  • Do not expose API keys, OAuth secrets, tokens, or localStorage token values in output.
  • A WebUI test is not complete until the visible UI result is checked against backend logs or network behavior.
  • A performance result is not complete without metrics evidence and a clear split between LangBot overhead and external provider/tool/network time.
  • A chaos or reliability result is not complete until the fault scope, cleanup, and recovery checks are recorded.
  • For a suite, use bin/lbs suite start <suite-id> to create the suite evidence root, per-case directories, and suite-start.json/suite-start.md handoff files; use bin/lbs test result <case-id> to write final per-case result.json, then run bin/lbs suite report <suite-id> --evidence-dir <dir>.
  • Do not mark a case pass until test result --evidence covers every value in the case's evidence_required.
  • For runner-specific Debug Chat cases, use the case-specific pipeline env declared by automation_pipeline_url_env / automation_pipeline_name_env; do not silently reuse a generic LANGBOT_PIPELINE_URL.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

langbot-deploy

FreeSummary

Deploy and configure LangBot with Docker Compose or Kubernetes. Set up application settings, plugin execution, the Box sandbox for isolated commands, and API authentication, with guidance on hosting choices and common deployment problems.

  • Deploy LangBot on a self-hosted server
  • Configure Box and the plugin runtime
  • Enable API authentication for agent access
langbot-app/LangBot18.3KUpdated Oct 19, 2026

langbot-dev

FreeSummary

Develop and debug LangBot's backend and web frontend. The skill guides environment setup, API additions, authentication, database migrations, and testing, including updates to matching MCP tools so agents can use changed API features.

  • Set up the LangBot backend and frontend for development
  • Add an API endpoint and its matching MCP tool
  • Update the database schema with Alembic
langbot-app/LangBot18.3KUpdated Oct 19, 2026

langbot-eba-adapter-dev

FreeSummary

Develop and migrate LangBot messaging adapters for Event-Based Agents, its event-driven architecture. Check conversion into shared event and message formats, implement platform APIs, and verify behavior through unit tests and real platform interactions.

  • Adding a messaging platform adapter to LangBot
  • Migrating an existing adapter to the EBA layout
  • Testing event and message conversion in both directions
langbot-app/LangBot18.3KUpdated Oct 19, 2026

langbot-env-setup

FreeSummary

Prepare a local LangBot environment for development and browser testing. The skill covers service startup, shared connection settings, browser access, proxy troubleshooting, and reusable login profiles, with guidance for Linux and WSL environments.

  • Prepare a local Linux or WSL development environment
  • Start services and verify WebUI access
  • Choose Computer Use or Playwright MCP for browser control
langbot-app/LangBot18.3KUpdated Oct 19, 2026

langbot-mcp-ops

FreeSummary

Manage LangBot bots and processing workflows through MCP, a standard for connecting agents to tools. Configure authenticated access, inspect resources, update supported settings, and investigate execution histories, logs, and model request diagnostics.

  • Connect an MCP client to LangBot
  • Create or update bots and processing workflows
  • Investigate execution logs and errors
langbot-app/LangBot18.3KUpdated Oct 19, 2026

langbot-plugin-dev

FreeSummary

Build and debug LangBot plugins that handle messages, commands, and AI tool calls. The skill covers SDK usage, common errors, test environment setup, WebSocket message testing, and preparation for publishing to LangBot Space.

  • Create a plugin that reacts to incoming messages
  • Fix plugin SDK errors and missing responses
  • Test group and private chats through WebSocket
langbot-app/LangBot18.3KUpdated Oct 19, 2026

See all skills by langbot-app

Report a problem with this skill