Build and run evaluators for AI/LLM applications using Phoenix.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
설치 방법 보기설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.
Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.
Starting Fresh: observe-tracing-setup → error-analysis → axial-coding → evaluators-overview
Building Evaluator: fundamentals → common-mistakes-python → evaluators-{code|llm}-{python|typescript} → validation-evaluators-{python|typescript}
RAG Systems: evaluators-rag → evaluators-code-* (retrieval) → evaluators-llm-* (faithfulness)
Production: production-overview → production-guardrails → production-continuous
| Prefix | Description |
|---|---|
fundamentals-* | Types, scores, anti-patterns |
observe-* | Tracing, sampling |
error-analysis-* | Finding failures |
axial-coding-* | Categorizing failures |
evaluators-* | Code, LLM, RAG evaluators |
experiments-* | Datasets, running experiments |
validation-* | Validating evaluator accuracy against human labels |
production-* | CI/CD, monitoring |
| Principle | Action |
|---|---|
| Error analysis first | Can't automate what you haven't observed |
| Custom > generic | Build from your failures |
| Code first | Deterministic before LLM |
| Validate judges | >80% TPR/TNR |
| Binary > Likert | Pass/fail, not 1-5 |
아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.
로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.
아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.
Primary accessibility skill for VS Code. REQUIRED for new feature and contribution work, and also applies to updates of existing UI. Covers accessibility help dialogs, accessible views, verbosity settings, signals, ARIA announcements, keyboard navigation, and ARIA labels/roles.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use this skill when the user explicitly asks to map, document, or onboard into an existing codebase. Trigger for prompts like "map this codebase", "document this architecture", "onboard me to this repo", or "create codebase docs". Do not trigger for routine feature implementation, bug fixes, or narrow code edits unless the user asks for repository-level discovery.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Act on user feedback attached to the current session. Use when the user submits feedback on the session's changes via the Submit Feedback button.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Add educational comments to the file specified, or prompt asking for file to comment if one is not provided.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when adding, modifying, or reviewing VS Code configuration policies. Covers the full policy lifecycle from registration to export to platform-specific artifacts. Run on ANY change that adds a `policy:` field to a configuration property.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Patterns and techniques for adding governance, safety, and trust controls to AI agent systems. Use this skill when: - Building AI agents that call external tools (APIs, databases, file systems) - Implementing policy-based access controls for agent tool usage - Adding semantic intent classification to detect dangerous prompts - Creating trust scoring systems for multi-agent workflows - Building audit trails for agent actions and decisions - Enforcing rate limits, content filters, or tool restrictions on agents - Working with any agent framework (PydanticAI, CrewAI, OpenAI Agents, LangChain, AutoGen)
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.