Results for “regression-testing”

Use when reviewing, shipping, or regression-testing a SKILL.md and you hold one or more captured trajectories where that skill was actually loaded. Translates the skill's instructions into behavior constraints, then reports which were EXERCISED (coverage) versus untested, and flags any that were in-scope but VIOLATED (compliance) — a cheap single-trajectory test-adequacy gate. It surfaces untested instructions rather than auto-blocking, and escalates fail-closed only when a violation touches shared repo state, never-surface memory, or self-modification. Complements skill-counterfactual-audit (which measures EFFECT) and skill-causal-curation (which measures contribution); this one asks ADEQUACY. Backed by the agent-skill-coverage metric (arxiv 2606.20659v2). Triggers on skill review, ship gating, and regression-testing a skill against its trajectories.

The English summary is being prepared. Showing the original description.

Add regression tests to AI-assisted code changes so previously found bugs are checked automatically. Test required API fields and sandbox response contracts, then run tests and builds before AI review to catch recurring omissions and inconsistent behavior.

• Add regression tests after fixing a bug
• Check required API response fields
• Review sandbox and production response consistency

Compare prompt revisions on a fixed set of test cases to identify regressions, meaning behavior that gets worse after a change. Record model settings, case-level evidence, missing results, costs, and evaluation limits for a reviewable decision.

• Comparing original and revised prompts
• Checking required JSON fields after prompt changes
• Reviewing tool selection and task completion

Set up repeatable security tests for LLM apps with Promptfoo and DeepTeam. Connect them to automated build pipelines, detect renewed vulnerabilities to malicious instructions or data leakage, and compare results across model or prompt changes.

• Running automated attack tests on every code change
• Blocking merges when security tests fail
• Comparing security across model or prompt versions

Fix defects in Composio SDK through focused reproduction, root-cause analysis, regression tests that check for recurrence, and targeted verification. Use it for reported bugs, failing tests, CI regressions, runtime errors, or incorrect SDK behavior.

• Reproducing and fixing reported SDK bugs
• Investigating failing tests
• Fixing regressions in automated CI checks

Regression testing strategies for AI-assisted development. Sandbox-mode API testing without database dependencies, automated bug-check workflows, and patterns to catch AI blind spots where the same model writes and reviews code.

The English summary is being prepared. Showing the original description.

Test and evaluation harness for AI agents — scenario suites, deterministic replay, regression diffing, cost and latency budgets. Use when agent quality is vibe-checked, before shipping a prompt or model change, or when evals drift.

The English summary is being prepared. Showing the original description.

Automated cookie consent validation using Selenium and Playwright. Covers banner interaction testing, consent state verification, tag firing audit after consent choices, regression testing for cookie compliance, and CI/CD pipeline integration.

The English summary is being prepared. Showing the original description.

Prompt engineering regression test matrix register: baseline outputs, semantic drift thresholds, judge evaluations, and golden dataset.

The English summary is being prepared. Showing the original description.

AI辅助开发的回归测试策略。沙盒模式API测试，无需依赖数据库，自动化的缺陷检查工作流程，以及捕捉AI盲点的模式，其中同一模型编写和审查代码。

The English summary is being prepared. Showing the original description.

修改/重构/Bug 修复后避免老问题复发时使用。适用于版本回归、修复后回归、重构验证。融合 Impact Analysis、Test Suite Pruning、Smoke/Sanity/Full 三层套件。

The English summary is being prepared. Showing the original description.

Use when testing, evaluating, or building regression suites for Agentforce agents: conversation testing in Agent Builder, topic (now subagent) coverage and utterance testing, Testing API and AiEvaluationDefinition metadata, Agentforce DX CLI test runs (sf agent generate test-spec, sf agent test create/run/resume/results/list), evaluation metrics (containment rate, escalation rate, CSAT, topic activation accuracy), and post-deploy analytics via Enhanced Event Logs. Triggers: 'how do I test my Agentforce agent', 'agent routes to wrong subagent', 'write utterance tests', 'regression test after topic change', 'measure agent quality', 'agent containment rate', 'run agent tests from the CLI'. NOT for agent creation, topic design, or action contract design — use agentforce/agentforce-agent-creation, agentforce/agent-topic-design, or agentforce/agent-actions respectively.

The English summary is being prepared. Showing the original description.

Regression testing strategies for AI-assisted development. Sandbox-mode API testing without database dependencies, automated bug-check workflows, and patterns to catch AI blind spots where the same model writes and reviews code.

The English summary is being prepared. Showing the original description.

Regression testing strategies for AI-assisted development. Sandbox-mode API testing without database dependencies, automated bug-check workflows, and patterns to catch AI blind spots where the same model writes and reviews code.

The English summary is being prepared. Showing the original description.

Wire Promptfoo and DeepTeam into CI/CD for automated regression red-teaming of LLM apps against OWASP LLM Top 10 and OWASP Agentic presets, failing the build when jailbreak or injection vulnerabilities regress.

The English summary is being prepared. Showing the original description.

Produce a release test-impact analysis for a Tamanu version upgrade: compare two Tamanu releases and classify every functional area as Full Test, Smoke Test, or Skip, each with a short reason and the Linear card or PR reference. Use when the user wants a test plan, QA scope, regression scope, or "what to test" for a Tamanu release or upgrade, wants to compare or diff two Tamanu versions (e.g. v2.57 to v2.59), or asks which areas changed between two releases. Handles Tamanu's train-release model (release/X.Y branches vs vX.Y.Z tags) and maps changed files and PRs to Tamanu functional sections (Patient Management, Encounters/Charting, Surveys & Programs, Medications, Labs, Imaging, Appointments, Vaccinations, Sync, Settings, Reports, Admin/Permissions, Mobile, etc.). Optionally maps against a regression-testing spreadsheet's sections when one is provided.

The English summary is being prepared. Showing the original description.

Use when comparing prompt changes against a fixed evaluation set: record actual model settings, required checks, missing results, judge limitations and case-level regression evidence.

The English summary is being prepared. Showing the original description.

Regression testing strategies for AI-assisted development. Sandbox-mode API testing without database dependencies, automated bug-check workflows, and patterns to catch AI blind spots where the same model writes and reviews code.

The English summary is being prepared. Showing the original description.

AI支援開発のためのリグレッションテスト戦略。データベース依存なしのサンドボックスモードAPIテスト、自動バグチェックワークフロー、同一モデルがコードの作成とレビューを行う際のAIの盲点を検出するパターン。

The English summary is being prepared. Showing the original description.