4.5천건 · 관련도순
Create and analyze AI model evaluation experiments in Arize using the ax CLI, a command-line tool. Collect real model outputs, export results, compare evaluation scores, and identify examples where a new model performs worse.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Design and implement RAG systems, which retrieve documents to inform AI answers. Get document preparation strategies, vector database comparisons, combined keyword and meaning-based search, result reranking, and an evaluation plan for retrieval quality.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Evaluate local Codex skills and plugins, understand their scores, and identify fixes to prioritize. Get explanations of token estimates, set up benchmark scenarios, and prepare workflows for measuring actual token usage.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Evaluate a local Codex skill for metadata, structure, reference links, size, and helper code quality. Get prioritized fixes and explanations, with benchmark setup for deeper analysis and measurement recommendations when observed usage is available.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Capture and review Agent Skill observations, and create, run, diagnose, or iteratively improve Skill evaluations (evals) with the skill-up CLI. Use when the user asks to record explicitly attributed Skill usage or feedback; review observations or user-supplied traces/transcripts; turn an approved observation into a regression case; evaluate, test, regress, verify, fix, improve, iterate, or evolve a Skill; add or strengthen eval cases; write eval.yaml/case.yaml; run skill-up run/validate/list-cases/report/import/init; or migrate from Anthropic evals.json. Plugin observation capture/review require either the Codex skill-up plugin or an observer-enabled DSH skill-up plugin. User-supplied external evidence can be reviewed without a plugin; evaluation remains multi-engine.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when the user has a RAG (Retrieval-Augmented Generation) system and wants to evaluate its quality — separating retrieval issues from generation issues. Also use when the user mentions RAG evaluation, faithfulness checking, hallucination detection in RAG, retrieval quality, chunking optimization, or "is my RAG pipeline working." Outputs a diagnostic matrix that pinpoints whether problems are in retrieval or generation.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when the user needs to design evaluation datasets, create test cases, stratify samples, generate adversarial examples, extract eval dimensions from traces/specs, or build a labeled evaluation set. Also use when the user mentions test data design, eval coverage, difficulty stratification, synthetic data generation for eval, or "how to create good evaluation data." Outputs datasets in OpenJudge-compatible format.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Advanced Evaluation workflow skill. Use this skill when the user needs This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Design eval harnesses — task schemas, metrics, dataset versioning, eval-as-code patterns. Use when asked to "build an eval harness", "set up eval-as-code", or "version our eval datasets".
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Author and run offline evals for Agentforce agents: fixture format, scoring rubrics, hallucination checks, regression baselines, CI integration, prompt-change safety. Trigger keywords: agent eval, hallucination, doesn't hallucinate, agentforce offline evals, rubric for agent response. NOT for deciding which layers of testing the agent needs — use agentforce/agentforce-testing-strategy. NOT for Testing Center, AiEvaluationDefinition or sf agent test runs — use agentforce/agent-testing-and-evaluation.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals automation.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
인공지능 에이전트의 지속 작업 루프를 설계하고 품질 확인, 평가, 실패 시 중지와 재시도 방침을 결합해 진행 정체와 반복 실패에 대응합니다.
Evaluate BMad skills using your agent's configured runner. Compare results with the model alone or another version, grade outputs against criteria, and test whether descriptions trigger correctly, preserving transcripts and evidence for each run.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Evaluate how Chatbox answers questions about large attachments using synthetic and real documents. Check whether the model retrieves file content when needed, skips retrieval for unrelated requests, and investigate failures through the actual application flow.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Create evaluation drafts for existing skills using actual conversations and user corrections. Produce criteria and repeatable test cases, distinguish evidence from specification assumptions, and prepare an anonymized draft for human review.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Review completed AI work by scoring task difficulty and execution quality separately. A fixed scoring matrix, arguments for higher and lower ratings, and saved score history help examine weaknesses and detect repetitive scoring patterns across sessions.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Evaluate a machine learning GPU cluster during a cloud trial or acceptance test. Measure GPU computing performance, communication within and between machines, and local and shared storage, then compile the results into a dated Markdown report.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Evaluate a local Codex plugin to understand its scores and identify what to fix first. Review stronger and weaker skills, prepare benchmark runs, compare saved results across versions, and produce a readable report.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Diagnose why an existing skill fails evaluations or shows no clear improvement. Separate content problems from test data, execution failures, activation issues, and insufficient evaluation tasks, then apply targeted fixes and validate them.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Regenerate sounds in a draft PeonPing pack based on feedback. This internal workflow revises sound prompts or speech text, preserves the pack's style and voice, renders replacement audio, and records each successful change.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Query the H2 database created by tika-eval to get counts and comparisons missing from standard reports. Includes connection guidance and SQL examples for comparing extracted text, tracking encoding changes, and joining related file records.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when measuring or improving agent quality and performance — set up evaluators, online monitoring, CI/CD quality gates, observability, or cost optimization. Triggers on: "evaluate my agent", "add evaluator", "measure quality", "quality gate", "run evals", "agent too slow", "why is it slow", "reduce latency", "set up observability", "CloudWatch dashboard", "how much does my agent cost", "cost optimization", "logs not showing up", "logs missing", "spans not found", "eval failing", "eval error", "dev traces", "local traces", "agentcore dev traces", "traces to CloudWatch". Not for debugging errors or crashes — use agents-debug. Slow but correct routes here; broken routes to debug.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Design an evaluation plan for an LLM or AI feature before shipping it. Use when asked how to evaluate a prompt/model/agent, set up an eval harness, define quality metrics for an AI feature, or build a regression gate. Produces an eval plan — task definition, datasets, metrics & rubrics, baselines, automated + human evals, a pass bar, and a regression gate.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.