Test design, vulnerability assessment, and quality assurance (13K skills)
Map sensitive data across the systems you manage, explore what a compromised asset could expose, and maintain a risk register. Record likelihood, impact, owners, mitigations, and review dates to support ongoing security planning.
Build repeatable DeepEval tests for AI agents, chatbots, and applications that retrieve documents to answer questions. Reuse or generate evaluation datasets, inspect failures, and improve prompts, tools, or retrieval through repeated evaluation runs.
Test LangBot through its WebUI and compare visible results with backend logs or network behavior. Use it to verify chat, model settings, bots, knowledge bases, and plugins, with recorded evidence for test results.
Helps review application code, authentication, access controls, and dependencies for security flaws. Uses OWASP guidance and CVE reports, records of known vulnerabilities, to assess impact and recommend practical fixes.
Test and debug REST and GraphQL APIs using curl, httpie, Postman collections, or scripted tests. Check authentication, invalid inputs, pagination, and response details to identify failures that a successful status code alone may miss.
Plan encryption, TLS connections, digital signatures, and key management using established cryptographic libraries. Get guidance on choosing algorithms, storing passwords, rotating keys, and avoiding implementation mistakes such as reused nonces or comparisons that reveal information through timing.
Plan compliance work for SOC 2, GDPR, HIPAA, and PCI-DSS by connecting requirements to technical safeguards, responsible teams, policies, and audit evidence. Get practical guidance for ongoing operations and audit preparation.
Design evaluation systems that measure agent results across accuracy, completeness, citations, and tool efficiency. Build repeatable tests, compare changes against a baseline, and define quality checks that catch regressions before deployment and monitor production performance.
Check a paper's numerical consistency using fixed rules, then record blocking or warning decisions and outstanding findings. The workflow preserves a traceable correction history, while explicitly leaving broader semantic review unavailable in its Codex mode.
Choose and run checks that cover a Verdaccio change, including affected packages, dependent packages, API tests, and relevant client or UI flows. Rebuild first and report coverage limits so passing results reflect the edited code.
Guides developers in restricting server file access to approved directories. It covers validated download paths, agent file tools, local data connectors, and sandbox settings to address path traversal, where supplied paths escape their allowed location.
Test local web applications with Python scripts using Playwright, a browser automation tool. Verify interface behavior, investigate problems, capture screenshots, and inspect browser logs, with a helper that manages one or more application servers.
Test local web applications with Python scripts using Playwright, a browser automation tool. Inspect rendered pages, exercise interface controls, capture screenshots and console logs, and use a helper to manage one or more application servers during tests.
Analyze git history to map contributors to security-sensitive files, identify reliance on too few people, and find stale areas. Export CSV and JSON results for ownership reviews, graph exploration, and comparison with CODEOWNERS.
Analyze a codebase to identify valuable assets, attack entry points, and realistic abuse paths. Produce a concise Markdown threat model that explains risk priorities, distinguishes existing protections from proposed mitigations, and makes assumptions explicit.
Review Python, JavaScript/TypeScript, and Go code for security issues using guidance for the project's languages and frameworks. Get prioritized findings, suggested improvements, or help writing new code with safer defaults.
Run MNN checks and regression tests on a host computer or Android device, with stage summaries and failure logs. A separate workflow benchmarks LLM prompt processing and token generation on real iPhones and iPads, including branch comparisons.
Run clang-tidy static analysis, which checks code without executing it, on added or modified BE and Cloud C++ files. Focus diagnostics on changed lines, investigate bug, performance, and readability warnings, and verify fixes with another check.
Prepare a clean Doris package and run Docker regression tests, which check whether changes break existing behavior. Restore configuration and startup files from Git, build the test image, and investigate failures using configuration checks and runtime logs.
Validate OpenHarness through real model API calls on an unfamiliar project. Tests cover conversations across multiple turns, actual tool execution, and combined features, with guidance for distinguishing setup problems from product failures.
Run repeatable, read-only checks on an Obsidian wiki to find broken or ambiguous links, isolated pages, missing metadata, empty sections, outdated index entries, and source-record issues. Review findings with paths, line numbers, and counts.
Review source code, dependencies, and infrastructure settings for security issues. Find exposed credentials, unsafe input handling, and access control gaps, then organize findings into a report with severity levels, affected locations, and recommended fixes.
Request a new API key or login through a single-use, expiring link. The submitted credential is encrypted and saved to the person's keychain, keeping secrets out of chat and making them available to the task.
Check mobile app changes in an iOS simulator using real local services. It covers user interactions, service failures and recovery, language and text size checks, with screenshots and recordings as evidence.