Run the official OpenResponses compliance suite against a local or remote Response API endpoint. Check basic and streaming responses, tool calls, images, and multi-turn input, then use request and response details to investigate authentication or schema failures.
Use for OpenResponses/Response API compliance tests, endpoint schema failures and test runs.
Run score Not tested
See how to installThis skill runs the official OpenResponses compliance suite against a local or remote Response API endpoint. You can check standard and streaming responses, system prompts, tool calls, image input, and multi-turn conversations together, or select tests with --filter. Verbose output shows request and response details, and JSON results can be used in CI. When a test fails, the symptoms can point to issues such as an authentication error returning HTML, missing LLM execution, a required field missing from the response, or a response format mismatch.
Use it after implementing a Response API to check whether it follows the expected behavior. It can also help narrow down a problem by rerunning selected tests. It is useful when investigating endpoint schema or authentication changes, and when a team wants to include compliance tests in CI.
streaming-response and tool-calling.”The target server must be running and have the api/v1/responses route registered. For mock development authentication, set ENABLE_MOCK_DEV_USER=true in .env. On the first run, the official test repository is fetched and its CLI runner is executed.
This overview was written by AI (OpenAI Codex) from the public SKILL.md. Check the original below for the exact behavior.
AI checked the SKILL.md for operations that need care and for setup requirements. The score comes only from actually running the skill (hands-on test).
Not counted in the score. Rated by AI (OpenAI Codex) as of Oct 26, 2026. Treat it as a guide and read the original before installing.
Review the instructions the agent will receive before you install.
Run the official OpenResponses compliance test suite against the local (or remote) Response API endpoint.
# From the openapi package directory
cd packages/openapi
# Run all tests (dev mode, localhost:3010)
APP_URL=http://localhost:3010 bun run test:response-compliance -- \
--auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1
# Run specific tests only
APP_URL=http://localhost:3010 bun run test:response-compliance -- \
--auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1 \
--filter basic-response,streaming-response
# Verbose mode (shows request/response details)
APP_URL=http://localhost:3010 bun run test:response-compliance -- \
--auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1 -v
# JSON output (for CI)
APP_URL=http://localhost:3010 bun run test:response-compliance -- \
--auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1 --json
ENABLE_MOCK_DEV_USER=true in .envapi/v1/responses route registered (via src/app/(backend)/api/v1/[[...route]]/route.ts)| Mode | Flags |
|---|---|
| Dev (mock user) | --auth-header "lobe-auth-dev-backend-api" --no-bearer --api-key 1 |
| API Key | --api-key lb-xxxxxxxxxxxxxxxx |
| Custom | --auth-header <name> --api-key <value> |
Available --filter values:
| ID | Description | Related Issue |
|---|---|---|
basic-response | Simple text generation (non-streaming) | LOBE-5858 |
streaming-response | SSE streaming lifecycle + events | LOBE-5859 |
system-prompt | System role message handling | LOBE-5858 |
tool-calling | Function tool definition + call output | LOBE-5860 |
image-input | Multimodal image URL content | — |
multi-turn | Conversation history via input items | LOBE-5861 |
| Variable | Default | Description |
|---|---|---|
APP_URL | http://localhost:3010 | Server base URL (auto-appends /api/v1) |
API_KEY | — | API key (alternative to --api-key flag) |
The script (packages/openapi/scripts/compliance-test.sh) clones the official openresponses/openresponses repo into scripts/openresponses-compliance/ (gitignored) and runs its CLI test runner. First run clones; subsequent runs update from upstream.
-v to see full request/response payloadspackages/openapi/src/types/responses.type.tspackages/openapi/src/services/responses.service.tspackages/openapi/src/controllers/responses.controller.tspackages/openapi/src/routes/responses.route.tspackages/openapi/scripts/compliance-test.shsrc/app/(backend)/api/v1/[[...route]]/route.tsNo reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Use this skill to verify a delivery by exercising the real product through a suitable surface such as a CLI, web app, desktop app, or iOS Simulator. It helps create or find a verification plan, capture evidence, and publish an acceptance report.
Use this skill to add an AI model provider to LobeHub across runtime integration, model cards, settings, branding, documentation, and acceptance checks. It also covers provider documentation updates and distinguishes new providers from models added to an existing provider.
Use this skill to add a server-side environment variable that sets a default for a user setting. It walks through validation, server configuration and store merging, plus updating the example environment file and self-hosting documentation.
Provides guidance for registering hooks around an agent’s execution steps, tool calls, human approvals, context compression, and sub-agent calls. Hooks can observe events, supply local mock tool results, or use webhooks to allow or deny tool results under configured policies.
Build event-driven background workflows for agents by defining sources, interpreting them as signals, and running actions through policies. It also guides asynchronous handoffs, deduplication, scoped execution, completion receipts, and observability.
Use this skill to run end-to-end acceptance checks for bot channels in real desktop chat apps. It drives apps such as Discord, Slack, Telegram, WeChat, Lark, and QQ to send a message, wait for a response, and capture the result.