'eval-plan' 검색 결과

Plan a paid evaluation with predefined success criteria, comparison groups, decoy items, and stopping rules. Rehearse without spending, then run a three-item paid pilot and use its spending table to inform approval.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Define evaluation criteria before spending
• Rehearse an evaluation at zero cost
• Check spending with a three-item paid pilot

Scope a Claude Managed Agent through six questions about its job, trigger, inputs, actions, success criteria, and recurrence. Produce a validated build sheet with configuration components, deferred features, and an evaluation plan, without requiring an API key.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Turn an agent idea into a concrete build plan
• Define success criteria before implementation
• Separate the initial scope from later integrations

Prepare an AI-SPEC.md design agreement for an AI development phase. The skill coordinates framework selection, documentation and domain research, and evaluation planning so the phase has a defined technical direction and assessment strategy.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Define the design agreement for an AI development phase
• Research frameworks, documentation, and the application domain
• Plan how to evaluate an AI system

Design an evaluation plan for an LLM or AI feature before shipping it. Use when asked how to evaluate a prompt/model/agent, set up an eval harness, define quality metrics for an AI feature, or build a regression gate. Produces an eval plan — task definition, datasets, metrics & rubrics, baselines, automated + human evals, a pass bar, and a regression gate.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Builds an eval plan for an AI feature - error analysis on real traces, failure-mode ranking, code checks and binary LLM judges validated against human labels, CI tests, production monitoring, and a launch bar. Use when you ask how do we eval this, how do I test an AI feature, our LLM outputs are inconsistent, are the evals good enough, LLM-as-judge, hallucinations, or what accuracy is good enough to launch. Draws on 41 Lenny's Podcast guests incl. Hamel Husain, Shreya Shankar and Sander Schulhoff.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.