Testland

Authors and runs Ragas - RAG-pipeline evaluation framework with metrics organized into RAG (Faithfulness, Response Relevancy, Context Precision/Recall, Context Entities Recall, Noise Sensitivity), Natural Language Comparison (Factual Correctness, Semantic Similarity, BLEU/ROUGE/CHRF/Exact Match), Agents/Tool-Use (Topic Adherence, Tool Call Accuracy/F1, Agent Goal Accuracy), General Purpose (Aspect Critic, Rubrics-based Scoring), Nvidia (Answer Accuracy, Context Relevance, Response Groundedness), and Summarization. Use when the user evaluates a RAG pipeline (retriever + generator) and needs the deepest metric variety in the OSS LLM-eval space.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Reads `EXPLAIN` / `EXPLAIN ANALYZE` output from PostgreSQL, MySQL, or SQLite - identifies the dominant cost (sequential scan, nested loop, sort spill, missing index, type-cast preventing index use), proposes the specific index or query rewrite to fix it, and emits the candidate `CREATE INDEX` statement. Use when load testing or production telemetry shows the database as the bottleneck and the team needs targeted query-level remediation.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Reads CPU flame-graph output from py-spy (Python), async-profiler (JVM), Go pprof, or Node.js `perf_hooks` / clinic.js: identifies the hot path (top sample-time frames), classifies the bottleneck (CPU-bound vs lock contention vs allocator pressure), and proposes the next investigation step. Use when a perf regression is bisected to a commit but the hot path inside it is unclear; for tail-latency percentiles use the latency-percentiles reference in k6-load-testing, and for a slow SQL hot path use db-query-plan-analyzer.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Authors Apache JMeter `.jmx` test plans (Thread Groups + HTTP samplers + assertions + listeners) in the JMeter GUI, runs them headlessly via `jmeter -n -t plan.jmx -l results.jtl`, generates an HTML dashboard with `-e -o`, and gates CI on JTL parsing. Use when the project has an existing JMeter investment, needs JVM-native load tooling, or works in domains with strong JMeter community support (banking, telecom, enterprise).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Authors k6 JavaScript load-test scripts (VU loops + checks + sleeps), configures the `options` block with `stages` (ramp-up patterns) and `thresholds` (p(95) latency, error rate), runs via `k6 run script.js` or `--vus / --duration` ad-hoc flags, and uses thresholds as the CI pass/fail signal. Includes a latency-percentile interpretation reference: tail ratio (p99/p50), bimodal-distribution detection, coordinated omission and why naive p99 is optimistic, and constant-vus vs constant-arrival-rate executors. Use when the project ships HTTP / WebSocket / gRPC load tests and the team wants developer-friendly JavaScript authoring, or when a k6 threshold passes but the system still feels slow.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Configures Lighthouse CI (`@lhci/cli`) to audit Web Vitals (LCP, INP, CLS) on every PR, asserts against canonical thresholds (LCP ≤2.5s, INP ≤200ms, CLS ≤0.1 at the 75th percentile), uploads Lighthouse reports as build artifacts, and posts deltas as PR comments. Includes a budget-authoring reference: per-route LCP/INP/CLS thresholds by traffic class (cached / dynamic / api-heavy / form-heavy / media-heavy) via `assertMatrix`, plus `budget.json` resource-size caps (JS / CSS / images / total bytes). Use when the project ships a web frontend and the team needs continuous Web Vitals monitoring tied to PR gating, or needs its first Lighthouse budgets drafted.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Teaches load and performance testing from zero: a tool-selection table choosing between k6, JMeter, Gatling, Locust, and Artillery from observable project facts; the six load profiles (smoke, average-load, stress, spike, soak, breakpoint); open vs closed workload models; why percentiles beat averages; turning a run into a pass/fail CI gate with a first runnable k6 script; a performance-incident triage workflow (confirm with a k6 smoke run, flame-graph the hot path, check slow queries, localize the cause); and full Gatling (Simulation DSL, injectOpen/injectClosed, setUp().assertions()) and Locust (HttpUser + @task locustfile, headless / distributed runs, CSV gating) deep dives in references. Use when a service needs performance coverage and the tool, load profile, or pass/fail threshold has not been decided yet, or when a live performance incident needs cause localization.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Builds a unified release-readiness gate that aggregates verdicts from any combination of k6 / JMeter / Gatling / Locust load runners and Lighthouse CI Web Vitals, applies severity-aware pass/fail thresholds, and emits a single go / no-go decision with per-metric deltas vs the main-branch baseline. Posts the delta as a PR comment when the team has the integration set up. Use when authoring a CI step that gates a deployment on cross-runner perf compatibility.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Pure-reference catalog of hardware-in-the-loop (HIL) testing for embedded systems. Defines the HIL pattern (ECU-under-test + real-time plant simulator + I/O cards emulating sensors / actuators / buses), the V-cycle progression (MIL → SIL → PIL → HIL), the canonical vendor stack (NI VeriStand + PXI / CompactRIO, dSPACE SCALEXIO / MicroAutoBox, Vector CANoe + VT System, Speedgoat real-time targets), bus emulation per protocol (CAN / CAN FD / LIN / FlexRay / Automotive Ethernet / SOME-IP), fault-injection patterns (short-to-ground, open-circuit, signal corruption), DO-178C / ISO 26262 / IEC 61508 alignment, and the test-evidence chain HIL produces. Use as the HIL terminology + vendor + standard reference when scoping an embedded test rig or interpreting an automotive / aerospace / industrial HIL test report.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Author and run GoogleTest 1.17+ for embedded C++ on ARM targets - TEST() / TEST_F() / TEST_P() / TYPED_TEST(), EXPECT_* vs ASSERT_* assertions, fixtures with SetUp() / TearDown(), value-parameterised tests, GoogleMock when paired, cross-compile with arm-none-eabi-g++, run on host or under QEMU via the qemu-system-test-runner skill, --gtest_filter / --gtest_output=xml:results.xml / --gtest_shuffle / --gtest_repeat command-line flags, and XML / JSON output parsing for CI. Use when the unit-under-test is C++ (modern C++17+) and the team wants the de-facto C++ test framework instead of the C-only Unity. For C use unity-test-framework-c; for pure C mocks see the CMock reference in ceedling-build-runner.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Pure-reference catalog of code-coverage strategy for embedded C/C++: the criteria hierarchy (statement / branch / decision / condition / MC/DC), the gcov toolchain (.gcno/.gcda, --coverage), the LLVM source-based toolchain (llvm-profdata / llvm-cov), host-build vs QEMU-build instrumentation, MISRA-C:2012 and DO-178C structural-coverage expectations by safety level (DAL A maps to MC/DC), and report-format choices. Use when choosing what structural-coverage level to require and wiring gcov / llvm-cov into the build; physical .gcda retrieval from hardware is in hardware-in-loop-reference, QEMU machine flags in qemu-system-test-runner, and to author the embedded tests themselves use googletest-embedded-arm or unity-test-framework-c.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Author and run the Ceedling build system for C unit testing - the canonical build orchestration on top of Unity (assertions) + CMock (mocks) + CException (exceptions). Covers ceedling new scaffolding, the project.yml schema (:project / :paths / :files / :defines / :flags / :tools / :test_runner / :cmock / :unity / :cexception / :gcov / :plugins), the task surface (test:all, test:{name}, test:pattern, test:path, release, clean / clobber, gcov:all, module:create, environment, dumpconfig), JUnit XML via the report_tests_* plugins, gcov integration, host vs cross-build flow, and CI wiring. CMock semantics - the generated Expect / Ignore / IgnoreArg / ReturnThruPtr / AddCallback / Stub / ExpectAndThrow family, cmock.yml :plugins, mock naming, tearDown verification, strict-vs-ignore matching - are in references/cmock.md. Use when a C project wants the ThrowTheSwitch trio bundled by one build command, or when authoring / reading CMock mocks. For the Unity assertion API see unity-test-framework-c.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Build a trace specification document per feature - defines the trace shape (root span + child spans + key attributes per OpenTelemetry semantic conventions) that production code MUST emit. The spec drives both implementation reviews AND trace-assertion tests, so a single declarative document is the source of truth for what observability "looks like" for a feature. Use before instrumenting a new feature, when existing spans have grown organically with no agreed shape, or after an incident where a debugging session stalled on missing span attributes.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Validates OpenTelemetry Collector pipeline configurations and verifies spans flow end-to-end through the collector: runs `otelcol validate --config`, wires the `debug`/`file` exporter for span-output assertions, and integrates the full cycle into CI. Use when a collector config change (new receiver, processor swap, exporter wiring) needs correctness verification before deployment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Author trace-shape assertions in tests using OpenTelemetry SDK in-memory exporter - capture spans during test execution, assert on span name + attributes + status + parent-child structure + duration. Cross-language patterns (Python `InMemorySpanExporter` + `SimpleSpanProcessor`, JS `getRecordedSpans()`, Java `OpenTelemetryExtension`); CI integration. Use when a service is instrumented with the OpenTelemetry SDK and downstream alerts, SLOs, or dashboards depend on specific span names or attributes that a refactor could silently drop.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Author integration tests that query a tracing backend for cross-service trace verification - Jaeger, Zipkin, or Grafana Tempo, same run-query-assert workflow. Jaeger all-in-one Docker for CI (OTLP gRPC :4317 + HTTP :4318 ingest, query API on :16686), `/api/traces?service=X&operation=Y` query patterns, span set + parent-child + duration assertions; Zipkin (:9411 REST API, B3 single/multi-header propagation tests, dependency graph) in references/zipkin.md; Tempo (TraceQL span selectors + structural operators, /api/search, single-binary Docker) in references/tempo.md. Use when verifying that a request produces the expected spans across service boundaries in a running Jaeger, Zipkin, or Tempo backend.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Authors and runs Windows UI tests against the WinAppDriver UIA surface via both invocation paths - the direct Microsoft W3C-WebDriver service (installing + launching `WinAppDriver.exe` on `127.0.0.1:4723`, `app` / `platformName` / `appArguments` / `appTopLevelWindow` capabilities) and the actively-maintained Appium 2.x wrapper (`appium driver install windows`, `appium:` prefixed capabilities, `windows:` gestures, PowerShell prerun/postrun hooks). Covers UWP / WPF / WinForms / Win32 apps, `AccessibilityId` / `Name` / `ClassName` locators, and Windows-runner CI. Use when driving a native Windows app from a Selenium-style client (C#, Java, Python, Ruby, JS) - directly when no Appium install is wanted, via Appium when the stack already runs Appium for iOS / Android / Mac2; for a C#-only FlaUI client use flaui-tests, and to choose among Windows desktop drivers first use desktop-test-strategy-reference.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Authors and runs Qt Test - the first-party C++ unit + GUI test framework that ships with Qt 6 (via the `QtTest` module header). Covers the `QTEST_MAIN` / `QTEST_APPLESS_MAIN` / `QTEST_GUILESS_MAIN` entry-point macros, the `QObject` private-slot test pattern, `QVERIFY` / `QCOMPARE` / `QFETCH` assertions, GUI event simulation (`QTest::mouseClick`, `QTest::keyClick`, `QTest::touchEvent`), `QSignalSpy` for signal introspection, `QBENCHMARK` for performance regression, and the `-o file,junitxml` CI output. Use for in-process testing of Qt widgets, QObject signal/slot chains, and Qt Quick / QML application logic; for out-of-process Qt-app driving, use an OS-native accessibility driver instead.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Authors and runs FlaUI-based Windows UI tests - the .NET-native wrapper around Microsoft UI Automation (UIA2 + UIA3). Covers the `FlaUI.Core` / `FlaUI.UIA2` / `FlaUI.UIA3` NuGet packages, `Application.Launch` / `Application.Attach` lifecycles, `ConditionFactory` + `FindFirstDescendant` locator patterns, `Retry` waits, and xUnit / NUnit / MSTest harness integration. Use when the test stack is C# / .NET-first and the team wants idiomatic in-process UIA calls rather than the HTTP/JSON wire protocol of `winappdriver` (direct or Appium-wrapped).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Authors Playwright `_electron` tests for packaged Electron desktop apps - launches the app via `electron.launch({ args })`, returns an `ElectronApplication` handle, drives renderer windows as Playwright `Page` objects, and probes the main process via `electronApp.evaluate(({ app, BrowserWindow }) => …)`. Distinct from ordinary browser page automation: this wraps the `_electron` API for launching packaged Electron apps and probing main + renderer processes. Includes the legacy Spectron reference and Spectron-to-Playwright migration shopping list (references/spectron-migration.md). Use for end-to-end tests of Electron apps where main-process state, IPC, and renderer DOM must all be asserted from one suite, or when migrating a deprecated Spectron suite.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Reference catalog of desktop GUI test strategies across Windows, macOS, and Linux. Defines the three accessibility-tree backends (Microsoft UI Automation on Windows, Apple Accessibility / XCTest on macOS, AT-SPI on Linux), the wrapper-tools that drive each backend, the cross-toolkit Electron + Qt paths, the project-marker detection table plus one-driver-per-app decision table (FlaUI / WinAppDriver / electron-playwright / QtTest / XCUITest / AT-SPI), an accessibility-first locator strategy, and a desktop test-review hazard checklist (screen-object encapsulation, locator stability, explicit waits, STA / foreground-lock / elevation). Deep operational detail (per-OS async-wait hierarchies, parallel-test policy, UAC / TCC / AT-SPI elevation hazards, the high-DPI matrix) lives in references/. Use when choosing how to test or automate a desktop GUI application on Windows, macOS, or Linux, or when reviewing an existing desktop UI test suite - the strategic reference ahead of the per-tool implementation skills.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Pure-reference catalog for defect classification: severity (impact on the system / user) vs priority (urgency of fix) on independent axes - the canonical 5-point severity scale (Critical / High / Medium / Low / Trivial), the 5-point priority scale (Immediate / High / Medium / Low / Deferred), the 5x5 matrix with worked S1/P5 and S5/P1 examples, and IEEE 1044-2009 severity classes; plus the full defect lifecycle (ISTQB-canonical states new / open / assigned / fixed / verified / closed / reopened / deferred / rejected / duplicate, allowed and forbidden transitions, tracker vocabulary maps) and the defect-categorisation taxonomies (IEEE 1044 anomaly classification, ISTQB CTAL-TA root-cause categories, Orthogonal Defect Classification) in references. Use when triaging or classifying a defect, configuring a tracker's severity/priority/state fields, reviewing a bug report's classification, or running root-cause analysis.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Procedure for proving that a claimed defect fix actually reached the build under test and actually works. Covers the merge-base ancestry check that proves the running build contains the fix commit rather than trusting a version label, the priority order for choosing which reproduction to re-run, and the VERIFIED / NOT FIXED / BLOCKED verdict table whose governing rule is that any ambiguous, flaky, or unreproducible result resolves to BLOCKED and is never guessed. Scoped to ISTQB confirmation testing (does this specific fix work?), not regression testing (did the fix break something else?), and not triage or severity assignment. Use when a developer has marked a defect Fixed and someone must decide whether it moves to Verified or back to Reopened.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Repairs defect bookkeeping that reports the wrong numbers - a weekly summary showing zero in the top severity band, a 'new defects this week' figure counting items already fixed, duplicates that were never merged, or one script moving issues across several boards whose workflows disagree. Files, transitions, dedupes, and searches bugs through one tracker-agnostic workflow across Jira, Linear, GitHub Issues, and Azure DevOps: authenticate, dedupe-search before creating, classify severity and priority, transition lifecycle states, and wire idempotent CI-driven filing from test failures. Jira Cloud REST API v3 is worked in full in the body (ADF descriptions, runtime transition lookup, JQL triage and duplicate queries, dry-run bulk transitions). Use when tracker data, defect metrics, or cross-board transitions are wrong or need automating.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.