4,126 件 ・ 関連度順
概要と使いどころ
Evaluates and sharpens content hooks using The Hook Stack™ framework. Use when scoring headlines, refining hooks for video/social/newsletter, or when asked to "evaluate this hook", "run through hook stack", or "score my headline".
日本語の概要は準備中です。原文の説明を表示しています。
Evaluate a single retort experiment run. Score the generated code against the task's TASK.md requirements, run its build and tests, compute metrics, and emit a structured evaluation report plus a machine-readable findings file.
日本語の概要は準備中です。原文の説明を表示しています。
Think and work like an expert Computer Vision Scientist. Use when a task calls for Computer Vision Scientist judgment. Reasons from image formation, projective geometry (pinhole intrinsics and extrinsics, epipolar/PnP/bundle adjustment, similarity-scale ambiguity), and COCO/LVIS AP mechanics through DINOv3/SigLIP 2 foundation baselines, RF-DETR/YOLO26/SAM 3 models, COLMAP 4/GLOMAP and VGGT geometry, pycocotools/TrackEval/BOP evaluation, and CVPR reporting and EU AI Act limits while treating train–test and pretraining leakage, AP evaluation-setting gaming, preprocessing mismatches (EXIF, BGR, aliased resizing), label noise, and camera-convention and scale errors as first-class failure modes.
日本語の概要は準備中です。原文の説明を表示しています。
Plan, implement, or review agentic eval work in an existing codebase with compatibility, security, and verification controls. Use when the user explicitly requests agentic eval work.
日本語の概要は準備中です。原文の説明を表示しています。
Test AI/LLM features that ship in your product. Covers prompt regression testing, response quality evaluation, tool-call validation, hallucination and RAG grounding checks, nondeterministic-output strategies, red-team/safety scans, eval frameworks, and agent-as-target injection (indirect injection via tool output / RAG / scan reports, self-propagating payloads, data exfiltration via an agent) plus a bundled detector for untrusted content. Use when: "test our LLM feature," "prompt regression test," "eval framework," "hallucination test," "RAG grounding," "nondeterministic output," "AI feature testing," "red-team our chatbot," "indirect prompt injection," "agent reading untrusted tool output," "production AI quality." Not for: using AI to generate your own test code — use ai-test-generation. Not for: classifying CI failures with AI — use ai-bug-triage. Not for: EU AI Act / GDPR conformity of an AI feature — use compliance-testing. Not for: canary/flag rollout of an AI feature — use testing-in-production. Related: ai-test-generation, ai-qa-review, api-testing, compliance-testing, security-testing, risk-based-testing, test-data-management.
日本語の概要は準備中です。原文の説明を表示しています。
Expert guidance for DeepEval, the open-source framework for unit testing LLM applications. Helps developers write test cases, define custom metrics, and integrate LLM quality checks into CI/CD pipelines using a pytest-like interface.
日本語の概要は準備中です。原文の説明を表示しています。
Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when "langfuse, llm observability, llm tracing, prompt management, llm evaluation, monitor llm, debug llm, langfuse, observability, tracing, llm-monitoring, evaluation, prompt-management, debugging, analytics" mentioned.
日本語の概要は準備中です。原文の説明を表示しています。
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarksUse when "agent testing, agent evaluation, benchmark agents, agent reliability, test agent, testing, evaluation, benchmark, agents, reliability, quality" mentioned.
日本語の概要は準備中です。原文の説明を表示しています。
Audit experiment integrity against the evidence ledger. At L2 (repo + result files present) a fresh cross-model reviewer reads the eval code line-by-line for fake/derived ground truth, score self-normalization, phantom results (a paper number with no backing file/key), dead/uncalled metric code, verified-scope inflation, method-described ≠ method-evaluated drift, synthesized-looking results, placeholder/fake data still wired into a released result, code-output ≠ reported-number mismatch, and missing reproducibility artifacts (an empirical/agent/LLM paper shipping neither code nor the prompts/configs its results need) — every finding span-anchored to a ledger claim_id. At L0/L1 (PDF / source only) the same patterns are surfaced as info-level 'could-not-verify' signals where the ledger gives an anchor (observability_level_required:2) — NEVER a fraud verdict from a PDF. The reviewer PROPOSES findings; tools/adjudicate_findings.py computes the verdict. Detect-only. Triggers: "experiment forensics", "audit the results", "check the eval code", "实验诚实度".
日本語の概要は準備中です。原文の説明を表示しています。
AI Engineering Toolkit workflow skill. Use this skill when the user needs 6 production-ready AI engineering workflows: prompt evaluation (8-dimension scoring), context budget planning, RAG pipeline design, agent security audit (65-point checklist), eval harness building, and product sense coaching and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
日本語の概要は準備中です。原文の説明を表示しています。
Agent Evaluation workflow skill. Use this skill when the user needs Testing and benchmarking LLM agents including behavioral testing, and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
日本語の概要は準備中です。原文の説明を表示しています。
Hamel Husain (@HamelHusain) 视角. 服务咨询派 + practitioner-as-figure 代表, 「evals are the new code」论断的提出者. ex-Airbnb / GitHub principal eng → 2017 起做 independent ML/AI consultant (parlance-labs) → 2023+ 转向 AI evals 专精. hamel.dev 长文 corpus (40+ 篇 1500+ words technical) + Maven「AI Evals for Engineers」课程 (3000+ paid 学员, 与 Shreya Shankar 共同主理). 在 monetize-agents 行业里代表「不规模化 / 不 productize 成 SaaS / 不融资」的第三条路 — independent consultant + course creator. 用途: 当用户面临「AI agent 上线不可靠 / 客户卡在 prompt 调不动 / 该不该雇团队 / 该不该做 SaaS / 怎样把 expertise productize 成课程而非公司」类问题时, 切换到这副镜片.
日本語の概要は準備中です。原文の説明を表示しています。
调香师与香水 (调香师 / 香水 (Perfumery & Fragrance) — 香水创作与香气调配的职业认知操作系统，从业者/调香师/评香师/学习者视角。覆盖: (a) 第一性张力 — codifiable 理论层 (原料知识/香料化学/嗅觉家族分类/accord 和弦/配方结构 %/挥发度金字塔/IFRA 法规) ⇄ tacit 校准层 (嗅觉记忆/闻香训练/「the nose」/创作直觉/盲嗅识别)，这行的核心矛盾是「可背诵的原料与公式」与「不可言传的嗅觉记忆与品味」之间，嗅觉记忆是极致 tacit 技能; (b) 法系传统 vs 分子/合成派 — 法国 Grasse 天然香料 + haute parfumerie 艺术传统 (天然精油/原精/学徒制) vs 现代分子派 (aroma chemicals 香基/captives 专利分子/headspace 顶空捕香/功能香精)，两套对「香水是什么」的世界观; (c) 培养/传承 — ISIPCA (凡尔赛香水学校) + 四大香精公司内部调香学校 (Givaudan/Firmenich(DSM-Firmenich)/IFF/Symrise) + 学徒制 + 「鼻子」tacit 校准; 角色分工: perfumer(parfumeur 调香师) vs evaluator(评香师) vs flavorist(食用香精师); (d) 行业结构 — 四大香精公司寡头 (Givaudan/DSM-Firmenich/IFF/Symrise) + Mane/Robertet/Takasago，为品牌做 B2B 命题创作 (the brief → creation → mods 模型); vs niche/indie 小众独立香水 (artisanal/直营); 天然 (Grasse/精油) vs 合成 (香基/captives) 的成本与可持续之争; (e) 创作层 — 香调金字塔 (top/heart/base notes 前中后调)、accord 和弦、配方 (酒精中 % 浓度)、浓度分级 (parfum/EDP/EDT/EDC)、调香师的「香风琴」orgue、trials/mods 试样迭代、weighing 称量精度; (f) 法规与安全 — IFRA standards (原料限用)、EU 致敏原标签 (26→80+ allergens)、REACH、橡苔/atranol 限用、经典香水 (Mitsouko/Chanel No.5) 因法规被迫重配 (reformulation) 与创作自由之张力; (g) 评价/批评 — Luca Turin & Tania Sanchez (《Perfumes: The Guide》《The Secret of Scent》气味的科学)、香评人、Fragrantica/Basenotes 社区、vintage vs reformulation 之争、niche vs designer 之争。学派分歧: 法系天然传统派 vs 分子合成派、天然纯粹派 vs 合成实用派、haute parfumerie/niche 艺术派 vs mass designer 商业派、「鼻子=艺术家」vs「调香师=香气工程师/技师」、嗅觉科学之争 (Luca Turin 振动理论 vs 主流形状/受体理论)。不含: 香薰蜡烛/家居香氛纯生产营销、芳香疗法健康宣称 (aromatherapy)、纯化妆品配方学 (超出香精范畴)、DIY 家用香氛套件、精油传销/MLM。) Master OS — automated mastery of 调香师 / 香水 (Perfumery & Fragrance) — 香水创作与香气调配的职业认知操作系统，从业者/调香师/评香师/学习者视角。覆盖: (a) 第一性张力 — codifiable 理论层 (原料知识/香料化学/嗅觉家族分类/accord 和弦/配方结构 %/挥发度金字塔/IFRA 法规) ⇄ tacit 校准层 (嗅觉记忆/闻香训练/「the nose」/创作直觉/盲嗅识别)，这行的核心矛盾是「可背诵的原料与公式」与「不可言传的嗅觉记忆与品味」之间，嗅觉记忆是极致 tacit 技能; (b) 法系传统 vs 分子/合成派 — 法国 Grasse 天然香料 + haute parfumerie 艺术传统 (天然精油/原精/学徒制) vs 现代分子派 (aroma chemicals 香基/captives 专利分子/headspace 顶空捕香/功能香精)，两套对「香水是什么」的世界观; (c) 培养/传承 — ISIPCA (凡尔赛香水学校) + 四大香精公司内部调香学校 (Givaudan/Firmenich(DSM-Firmenich)/IFF/Symrise) + 学徒制 + 「鼻子」tacit 校准; 角色分工: perfumer(parfumeur 调香师) vs evaluator(评香师) vs flavorist(食用香精师); (d) 行业结构 — 四大香精公司寡头 (Givaudan/DSM-Firmenich/IFF/Symrise) + Mane/Robertet/Takasago，为品牌做 B2B 命题创作 (the brief → creation → mods 模型); vs niche/indie 小众独立香水 (artisanal/直营); 天然 (Grasse/精油) vs 合成 (香基/captives) 的成本与可持续之争; (e) 创作层 — 香调金字塔 (top/heart/base notes 前中后调)、accord 和弦、配方 (酒精中 % 浓度)、浓度分级 (parfum/EDP/EDT/EDC)、调香师的「香风琴」orgue、trials/mods 试样迭代、weighing 称量精度; (f) 法规与安全 — IFRA standards (原料限用)、EU 致敏原标签 (26→80+ allergens)、REACH、橡苔/atranol 限用、经典香水 (Mitsouko/Chanel No.5) 因法规被迫重配 (reformulation) 与创作自由之张力; (g) 评价/批评 — Luca Turin & Tania Sanchez (《Perfumes: The Guide》《The Secret of Scent》气味的科学)、香评人、Fragrantica/Basenotes 社区、vintage vs reformulation 之争、niche vs designer 之争。学派分歧: 法系天然传统派 vs 分子合成派、天然纯粹派 vs 合成实用派、haute parfumerie/niche 艺术派 vs mass designer 商业派、「鼻子=艺术家」vs「调香师=香气工程师/技师」、嗅觉科学之争 (Luca Turin 振动理论 vs 主流形状/受体理论)。不含: 香薰蜡烛/家居香氛纯生产营销、芳香疗法健康宣称 (aromatherapy)、纯化妆品配方学 (超出香精范畴)、DIY 家用香氛套件、精油传销/MLM。: top builders' mental models, tool stack, current workflows, jargon, and where to keep up. Trigger this skill when the user works on 调香师 / 香水 (Perfumery & Fragrance) — 香水创作与香气调配的职业认知操作系统，从业者/调香师/评香师/学习者视角。覆盖: (a) 第一性张力 — codifiable 理论层 (原料知识/香料化学/嗅觉家族分类/accord 和弦/配方结构 %/挥发度金字塔/IFRA 法规) ⇄ tacit 校准层 (嗅觉记忆/闻香训练/「the nose」/创作直觉/盲嗅识别)，这行的核心矛盾是「可背诵的原料与公式」与「不可言传的嗅觉记忆与品味」之间，嗅觉记忆是极致 tacit 技能; (b) 法系传统 vs 分子/合成派 — 法国 Grasse 天然香料 + haute parfumerie 艺术传统 (天然精油/原精/学徒制) vs 现代分子派 (aroma chemicals 香基/captives 专利分子/headspace 顶空捕香/功能香精)，两套对「香水是什么」的世界观; (c) 培养/传承 — ISIPCA (凡尔赛香水学校) + 四大香精公司内部调香学校 (Givaudan/Firmenich(DSM-Firmenich)/IFF/Symrise) + 学徒制 + 「鼻子」tacit 校准; 角色分工: perfumer(parfumeur 调香师) vs evaluator(评香师) vs flavorist(食用香精师); (d) 行业结构 — 四大香精公司寡头 (Givaudan/DSM-Firmenich/IFF/Symrise) + Mane/Robertet/Takasago，为品牌做 B2B 命题创作 (the brief → creation → mods 模型); vs niche/indie 小众独立香水 (artisanal/直营); 天然 (Grasse/精油) vs 合成 (香基/captives) 的成本与可持续之争; (e) 创作层 — 香调金字塔 (top/heart/base notes 前中后调)、ac
日本語の概要は準備中です。原文の説明を表示しています。
Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles.
日本語の概要は準備中です。原文の説明を表示しています。
Write, run, and analyze structured test suites for Agentforce agents — functional AND security. TRIGGER when: user writes or modifies test spec YAML (AiEvaluationDefinition); runs sf agent test create, run, run-eval, or results commands; asks about test coverage strategy, metric selection, or custom evaluations; interprets test results or diagnoses test failures; asks about batch testing, regression suites, or CI/CD test integration; requests security testing, OWASP LLM Top 10, red-teaming, penetration testing, prompt-injection tests, a security grade, or a vulnerability assessment of an agent. DO NOT TRIGGER when: user creates, modifies, previews, or debugs .agent files (use agentforce-generate); deploys or publishes agents; writes Agent Script code; uses sf agent preview for development iteration; analyzes production session traces (use agentforce-observe); performs a static safety review of .agent file content (use agentforce-generate Section 15).
日本語の概要は準備中です。原文の説明を表示しています。
Objectively evaluate a UI/web design against the pols.dev anti-slop design law: detect catalogued slop tells with cited evidence, score 8 weighted axes (color, type, components, layout, motion, execution, signature, cohesion), and emit a Slop Report with a 0–100 Slop Index and grade. Use when the user asks to "evaluate design slop", "slop report", "is this design AI slop", "audit this landing page design", "de-slop review", or wants an objective score of how generic/machine-made a design looks. To fix text (not design), use human-ai or humanizar skills instead.
日本語の概要は準備中です。原文の説明を表示しています。
Run scored behavioral evaluations and repair gaps. Use when the user says "evaluate this agent" or "I want proof this behaves correctly"
日本語の概要は準備中です。原文の説明を表示しています。
Builds QA harnesses and agent evals. Use when evaluating tool, trace, red-team, regression, multi-agent, or carried-workspace coding trajectories. Not for metric design (ai-evals).
日本語の概要は準備中です。原文の説明を表示しています。
Designs RAG retrieval for LLM grounding: chunking, BM25+vector hybrid, reranking, GraphRAG. Use when adding embeddings to SQLite FTS5 search or running held-out retrieval eval.
日本語の概要は準備中です。原文の説明を表示しています。
Designing search engines and vector DBs for full-text, vector, and hybrid retrieval, including permission-aware retrieval for multi-tenant or per-role corpora. Use for search design, index optimization, the RAG retrieval layer, or deciding where ACL filtering belongs in the query path.
日本語の概要は準備中です。原文の説明を表示しています。
Designing and evaluating AI/ML systems: prompt engineering, RAG design, LLM application patterns, AI safety, evaluation frameworks, MLOps, cost optimization. Use for AI pipelines or eval harnesses.
日本語の概要は準備中です。原文の説明を表示しています。
Generate retrieval practice questions at varied difficulty levels for a topic or concept. Use when creating quiz starters, revision activities, or low-stakes testing materials.
日本語の概要は準備中です。原文の説明を表示しています。
Remix v2 data loading and mutations. Use when writing loaders, actions, deferred data, revalidation logic, or pending state. Triggers on loader, action, useLoaderData, useActionData, json(), defer(), <Await>, shouldRevalidate, useRevalidator, useNavigation, useTransition (v1 holdover).
日本語の概要は準備中です。原文の説明を表示しています。
Run a Meta-Harness-style optimization loop NATIVELY — automatically search over the scaffolding around a FIXED base model (memory, retrieval, context construction, prompt templates, summarization, tool-selection logic) by proposing candidate variants, scoring each on a cheap deterministic eval, and keeping a Pareto frontier of quality vs cost — using native Agent / Workflow / loop tools instead of a standalone Python harness. Use this whenever the user wants to optimize, evolve, tune, distill, or search over a harness, scaffold, prompt system, memory or retrieval policy, context-assembly code, or summarizer while keeping the model fixed; whenever they mention Meta-Harness, harness optimization, scaffold evolution, automatic prompt/memory optimization, an evolutionary or Pareto search over candidate implementations, or "make the harness/agent better without retraining"; and whenever the gain must come from the code AROUND the model rather than the model weights. Reproduces the Meta-Harness paper's method natively, with no claude_wrapper.py and no metered solver API.
日本語の概要は準備中です。原文の説明を表示しています。