Testing and evaluation framework for LLM prompts and applications -- promptfooconfig.yaml, assertions, model-graded evals, red teaming, CI/CD integration, custom providers, and comparative evaluation
日本語の概要は準備中です。原文の説明を表示しています。
190 件 ・ 関連度順
概要と使いどころ
Testing and evaluation framework for LLM prompts and applications -- promptfooconfig.yaml, assertions, model-graded evals, red teaming, CI/CD integration, custom providers, and comparative evaluation
日本語の概要は準備中です。原文の説明を表示しています。
The measurement surface for ALL Pulp DSP and audio-pipeline work — read it BEFORE writing or gating DSP, not only when something already sounds wrong. Covers the C++ harness (signal generators, metrics, assertions, RenderScenario, contracts), the offline Audio Doctor (magnitude/frequency response, THD/THD+N, phase/group delay), and their Python sibling the Audio Quality Lab (tools/audio/quality-lab — null residual + alignment, LTAS log-spectral distance, spectral flux/centroid, HNR, Theil-Sen drift slope, Kaiser-sinc resampling, license-guarded corpus, regression-net ratchet). TRIGGER on AUTHORING work — "build/design an oscillator/filter/synth/effect", "add a DSP module", "what should the acceptance gate be", "how do I measure aliasing / anti-aliasing / alias floor", "null against a reference", "is this DSP correct", "choose a tolerance", "golden/regression corpus for audio", "measure drift or jitter", "A/B two renders" — AND on DEBUGGING work — "is there sound / no audio / I hear nothing", "does this filter/compressor/synth/delay produce the right signal", "prove the DSP / prove the contract", "measure the frequency response", "what's the THD / is it distorting", "what's the group delay / phase response / measured latency", "magnitude response curve", "render a test tone and assert", "audio regression", "64-frame works but 128 is silent", "sample-rate change pitch-shifted it", "describe what's in this buffer", "audio doctor", "compare before/after a DSP refactor". Reach for this BEFORE hand-rolling any FFT, null test, alias measurement, pitch tracker, or golden-render script — most of it already exists in one of the two lanes. Test/tool layer over HeadlessHost — deterministic, no audio device, no speakers. Off the realtime thread entirely.
日本語の概要は準備中です。原文の説明を表示しています。
Generates EditMode and PlayMode tests using NUnit, mapped from GDD Gherkin scenarios. Use when writing tests, validating features, ensuring code quality, or verifying behavior after changes. Triggers on requests like "write tests for this", "test the health system", "add unit tests", "verify this works", "add PlayMode tests", "test this feature", "write integration tests", "make sure this doesn't break", "add test coverage", "TDD this feature", or any task involving test creation, test-driven development, or quality assurance. Maps Given/When/Then scenarios directly to test methods using NUnit assertions.
日本語の概要は準備中です。原文の説明を表示しています。
Create, run, and maintain API test collections using Bruno (OpenCollection YAML format and legacy Bru format). Use when the user wants to: (1) create a Bruno API test collection from scratch or from OpenAPI/Swagger specs, (2) write API request files with tests and assertions, (3) run API tests using bru CLI, (4) generate test reports (HTML, JUnit, JSON), (5) set up CI/CD pipelines (GitHub Actions) for automated API testing, (6) debug or fix failing Bruno API tests, (7) add environment configurations for API testing, (8) chain API requests with data extraction, or (9) work with any .yml/.bru Bruno collection files. Triggers on mentions of 'Bruno', 'bru CLI', 'API testing collection', 'OpenCollection', or requests to automate API testing with file-based collections.
日本語の概要は準備中です。原文の説明を表示しています。
Set up an end-to-end test suite in any repo, following practices that make e2e a reliable per-PR gate: real flows over bypass, layered assertions, a reusable auth/session helper, video+trace evidence, and a compounding suite. Use when a repo has no e2e (or weak e2e) and you want system-level tests — "set up e2e", "add end-to-end tests", "scaffold a test gate".
日本語の概要は準備中です。原文の説明を表示しています。
Refresh Claude Ads platform, API, policy, regulation, benchmark, issue, pull-request, fork, and repository evidence. Use for ads research refresh, expired refresh_due dates, stale API or platform claims, reverify-or-demote decisions, release-current claim validation, ecosystem review, current platform changes, or updating paid-media knowledge. When tools or sources are unavailable, still demote the stale claim for the current run and block dependent release-current assertions before requesting access.
日本語の概要は準備中です。原文の説明を表示しています。
Apex test patterns for @InvocableMethod agent actions: per-reason-code branch coverage, bulk safety, callout mocking, deterministic assertions. NOT for testing whether the agent routes to the right topic or answers well — use agentforce/agent-testing-and-evaluation. NOT for designing the error envelope the tests assert on — use agentforce/agent-action-error-handling.
日本語の概要は準備中です。原文の説明を表示しています。
Test UI components the way users interact with them using Testing Library — query by role, text, and label instead of implementation details. Use when someone asks to "test React components", "Testing Library", "user-centric testing", "test accessibility", "test without implementation details", or "render and query components in tests". Covers React Testing Library, queries, user events, async testing, and accessibility assertions.
日本語の概要は準備中です。原文の説明を表示しています。
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
日本語の概要は準備中です。原文の説明を表示しています。
QA Automation Engineer skill. Use this to write or refactor unit tests. Ensures tests follow the project's xUnit, FluentAssertions, and Moq standards.
日本語の概要は準備中です。原文の説明を表示しています。
C# and .NET testing patterns with xUnit, FluentAssertions, mocking, integration tests, and test organization best practices.
日本語の概要は準備中です。原文の説明を表示しています。
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
日本語の概要は準備中です。原文の説明を表示しています。
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
日本語の概要は準備中です。原文の説明を表示しています。
Migrate test files from `as` type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace `as` in tests, or needs partial test data.
日本語の概要は準備中です。原文の説明を表示しています。
GEODE prompt-writing standards. Use when creating or editing runtime prompts, system prompts, tool/developer instructions, model-facing identity text, prompt templates, or scaffold rules that affect model behavior; especially when avoiding identity assertions such as "You are ...".
日本語の概要は準備中です。原文の説明を表示しています。
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
日本語の概要は準備中です。原文の説明を表示しています。
Audit whether tests detect broken behavior by tracing assertions and running narrow controlled mutations in an isolated workspace; use for test-quality reviews, not routine test execution.
日本語の概要は準備中です。原文の説明を表示しています。
Migrate test files from `as` type assertions to @total-typescript/shoehorn. Use when user mentions shoehorn, wants to replace `as` in tests, or needs partial test data.
日本語の概要は準備中です。原文の説明を表示しています。
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
日本語の概要は準備中です。原文の説明を表示しています。
BEFORE writing/changing tests, mocks, or test-only methods on production classes — mocking-the-mock, production pollution, overfit assertions; for order, `test-driven-development` wins.
日本語の概要は準備中です。原文の説明を表示しています。
Use when a diff may lack tests — missing assertions, uncovered branches, over-mocking, no regression test for a bug fix — dispatched by /review-changes, /do-and-judge, /judge, even without 'tests'.
日本語の概要は準備中です。原文の説明を表示しています。
Use when writing API endpoint tests — integration tests, contract validation, response assertions, mocked external services — even when the user says 'test this route' without naming API testing.
日本語の概要は準備中です。原文の説明を表示しています。
Use when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidence before assertions always
日本語の概要は準備中です。原文の説明を表示しています。
TDD methodology for production-quality tests. Write tests FIRST driven by PRD, Tech Spec, Implementation Plan. Covers minimal mocking, realistic test data, strong assertions, test independence.
日本語の概要は準備中です。原文の説明を表示しています。