“leakage”的搜索结果

Audit whether a paper's EVALUATION DESIGN actually measures what it claims and whether its reporting is complete — the validity layer family D (experiment-forensics) cannot reach. Three patterns: train/test leakage means the reported score may not measure generalization (HP-EVAL-LEAKAGE — adopts the Kapoor & Narayanan 8-type / 3-category leakage taxonomy; the illegitimate-proxy / sampling-bias / pretraining-contamination subtypes hand off as needs_external_check, naming but NEVER running Oren-2023 exchangeability / Shi-2023 Min-K% / Golchin-2023 Time-Travel / BIG-bench canary); a load-bearing LLM judge is conflicted (same model/family as a compared system) or unvalidated (no human-agreement, no bias control) (HP-JUDGE-VALIDITY); a declared condition/metric is dropped or switched to favor the method, or 'best' is chosen with no held-out set (HP-SELECTIVE-REPORTING). Verdict-bearing at L0/L1 from the DESCRIBED protocol — NOT repo-gated like experiment-forensics; L2 only CONFIRMS against split/preprocessing/result files. A fresh cross-model reviewer (gpt-5.6-sol xhigh, read-only, fresh thread per pass) PROPOSES findings, each span-anchored to a ledger claim_id; tools/adjudicate_findings.py DECIDES the verdict. Leakage and under-reporting are usually HONEST methodological errors — every finding describes a discrepancy to CHECK, never an accusation. An LLM generating GROUND-TRUTH labels is HP-FAKE-GT (experiment-forensics) — routed there, not here. Emits eval-design-forensics.findings.json; computes NO verdict. Detect-only. Triggers: "eval design audit", "evaluation validity", "train/test leakage", "data leakage", "is the score measuring generalization", "LLM judge bias", "is the judge validated", "selective reporting", "cherry-picked results", "评估设计审计", "评测有效性", "数据泄漏", "训练测试集泄漏", "裁判模型有没有验证", "选择性报告".

中文简介正在准备中。当前显示原文说明。

Plan and audit preprocessing for grouped time-series data, such as matches or patient records. Check missing-value handling, scaling scope, feature meaning, and data leakage, where information unavailable at prediction time enters model inputs.

中文简介正在准备中。当前显示英语简介。

• Planning missing-value handling within each group
• Choosing within-group or global scaling
• Auditing preprocessing code for data leakage

Reviews machine learning preprocessing, features, and data splits for data leakage: information unavailable when making a real prediction. Identifies risky statistics, future data, and validation setups, then suggests corrections using only appropriate training information.

中文简介正在准备中。当前显示英语简介。

• Reviewing normalization and missing value imputation
• Checking feature selection and target encoding
• Finding future information in time series features

Assessing privacy risks in large language model outputs including training data memorisation, PII leakage in generated text, prompt injection leading to data extraction, and hallucinated personal data. Covers output filtering, guardrails, and monitoring. Keywords: LLM privacy, output risk, memorisation, PII leakage, prompt injection, hallucinated PII.

中文简介正在准备中。当前显示原文说明。

敵対的分析手法を統合したレビューの entry skill。認知バイアス対策の3手法 （Pre-mortem / War Game / Logic Torturing）と、宣言・主張と実態の乖離を突く claim-vs-actual 検出3パターン（Self-Contradiction / Refactor-Claim Audit / Cross-File Leakage）へルーティングし、通常のレビューでは見えない設計の盲点・ 防御の穴・論理の弱点・宣言と実装のズレを可視化する。

中文简介正在准备中。当前显示原文说明。

Design evaluation tasks that check whether agents actually complete work. Define requests, simulated user behavior, starting conditions, and verifiable success criteria, then plan difficulty levels, training-data leakage checks, and ongoing maintenance.

中文简介正在准备中。当前显示英语简介。

• Creating evaluation tasks for real business workflows
• Reviewing success criteria and verification logic
• Designing checks for evaluation data leakage into training

Inspect supported local scientific files before modeling, summarize structure and missing values, and flag possible data leakage or outlier effects. Produces bounded aggregate reports with identifiers obscured and provides a structured analysis report template.

中文简介正在准备中。当前显示英语简介。

• Check local CSV or TSV data before modeling
• Audit missing values and overlap between data splits
• Compare outlier influence and transformation sensitivity

Design layered protections for AI agents against malicious instructions, tool misuse, and data leakage. Provides guidance and code examples for input checks, isolated tool execution, permission limits, and audit logging to help review and strengthen agent security.

中文简介正在准备中。当前显示英语简介。

• Review defenses against malicious instructions
• Isolate tool execution and restrict permissions
• Assess data leakage risks between tenants

Test whether an AI agent reveals sensitive information through progressively stronger dialogue prompts. Check API keys, personal data, system instructions, and internal settings, then report findings only when sensitive content is actually accessible.

中文简介正在准备中。当前显示英语简介。

• Check for API key and credential leaks
• Test exposure of system instructions
• Inspect personal data and internal settings

Investigate Next.js security issues across Server Actions, middleware, image optimization, and cached pages. Provides testing workflows for authentication bypass and data exposure, with checks to distinguish actual vulnerabilities from normal responses.

中文简介正在准备中。当前显示英语简介。

• Checking authentication in Server Actions and middleware
• Testing image optimization for unintended server requests
• Inspecting page data for sensitive information

Review build and deployment pipelines for unsafe workflows, exposed credentials, and configuration flaws across GitHub Actions, Jenkins, GitLab CI, and Terraform state files. Use concrete evidence and false-positive checks to distinguish exploitable issues from suspicious settings.

中文简介正在准备中。当前显示英语简介。

• Review GitHub Actions workflows for unsafe execution paths
• Investigate Jenkins exposure and affected versions
• Check logs, artifacts, and container images for leaked secrets

Use when you need framework-agnostic Hexagonal architecture guidance for Java projects - ports and adapters boundaries, application core independence, driving and driven adapters, dependency direction, infrastructure leakage detection, optional architecture tests, and evidence-backed remediation. This should trigger for requests such as Review Hexagonal architecture; Review ports and adapters boundaries; Enforce dependency direction; Add ArchUnit architecture tests; Detect infrastructure leakage in domain code; Improve ports and adapters boundaries. Part of Plinth Toolkit

中文简介正在准备中。当前显示原文说明。

Think and work like an expert Data Scientist. Use when a task calls for Data Scientist judgment. Reasons from CRISP-DM business estimands, leakage-safe sklearn Pipelines and nested CV, SQL/warehouse semantic metrics, A/B power and SRM/AA guardrails, causal DAG covariate discipline, and Model Cards/Datasheets while treating train-test leakage, Simpson's paradox, peeking, and PSI>0.25 drift as first-class failure modes.

中文简介正在准备中。当前显示原文说明。

Audit existing model training code — find reproducibility issues, data leakage, and missing best practices. Use when asked to "audit our training code", "is our training reproducible", or "check for training data leakage".

中文简介正在准备中。当前显示原文说明。

Audit feature engineering code for leakage, quality issues, and pipeline correctness. Use when asked to "audit our feature pipeline", "is there data leakage", or "find feature quality issues".

中文简介正在准备中。当前显示原文说明。

Audit existing eval coverage — gaps, metric validity, benchmark leakage, dataset freshness. Use when asked "what eval coverage do we have", "audit our evals", or "check for benchmark leakage".

中文简介正在准备中。当前显示原文说明。

Review a data analysis for quality, correctness, and reproducibility. Covers data quality assessment, assumption checking, model validation, data leakage detection, and reproducibility verification. Use when reviewing a colleague's analysis before publication, validating an ML pipeline before production deployment, auditing a report for regulatory or business decision-making, or performing a second-analyst review in a regulated environment.

中文简介正在准备中。当前显示原文说明。

Extract sensitive training data and internal capabilities from AI models through repeated programmatic querying. Use this skill when testing LLMs and ML models for data privacy leaks, intellectual property exposure, or unintended memorization. Covers membership inference, prompt extraction, training data reconstruction, and API boundary testing.

中文简介正在准备中。当前显示原文说明。

Checks layer leakage in method signatures, missing DTOs, entity leakage to API, inconsistent error contracts. Returns findings with penalty-based scoring.

中文简介正在准备中。当前显示原文说明。

Design leakage-safe machine learning experiments that mirror real deployment and support fair model comparisons. Use when defining prediction timing, feature eligibility, train-validation-test splits, baselines, metrics, or controlled model iterations; not for auditing whether raw labels are trustworthy.

中文简介正在准备中。当前显示原文说明。

モジュール/スキルの構造変更で当該ファイルは更新したが、caller 側が古い構造（旧参照・旧シグネチャ・旧採番）を参照したまま残るパターンを検出する

中文简介正在准备中。当前显示原文说明。

Tests APIs for excessive data exposure where endpoints return more data than the client application needs, relying on the frontend to filter sensitive fields. The tester intercepts API responses and analyzes them for leaked PII, internal identifiers, debug information, or sensitive business data that the UI does not display but the API transmits. This maps to OWASP API3:2023 Broken Object Property Level Authorization. Activates for requests involving API data leakage testing, excessive data e...

中文简介正在准备中。当前显示原文说明。

Validates predictive models on omics and biomedical data with nested cross-validation, group/batch/temporal-aware splits, the full data-leakage taxonomy, probability calibration, decision-curve net benefit, optimism correction, sample-size planning, and TRIPOD+AI reporting. Use when estimating model performance honestly, choosing a CV scheme, detecting leakage, or judging whether reported discrimination means the model is actually useful. For feature selection itself see machine-learning/biomarker-discovery; for confirmatory-trial inference see clinical-biostatistics/trial-reporting.

中文简介正在准备中。当前显示原文说明。

Tests APIs for excessive data exposure where endpoints return more data than the client application needs, relying on the frontend to filter sensitive fields. The tester intercepts API responses and analyzes them for leaked PII, internal identifiers, debug information, or sensitive business data that the UI does not display but the API transmits. This maps to OWASP API3:2023 Broken Object Property Level Authorization. Activates for requests involving API data leakage testing, excessive data exposure, response filtering bypass, or API over-fetching.

中文简介正在准备中。当前显示原文说明。