25건 · 관련도순
Design metrics that measure whether a workflow achieved its intended outcome using recorded session evidence. Choose built-in checks or custom JavaScript, retain suitable Bash evaluators, and define tests that distinguish success, failure, and missing evidence.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Build and run evaluations for AI applications using Phoenix. Start with observed failures, create checks in code or use a language model as a judge, run experiments, and validate automated judgments against human labels.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Create shareable links that open specific Arize resources, including traces, datasets, and evaluators. Use IDs from logs or exported data, with suitable time ranges, to direct teammates to the relevant view.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Guide for evaluation — inline evaluators, Inspect AI integration, and custom evaluators for measuring training progress. Use when the user asks about evaluation, metrics, benchmarks, or how to measure model quality during training.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Apply AlphaGo-style patterns: learned priors plus search, mixed evaluators, self-play, and asynchronous fast/slow loops. Use for intractable search and compute allocation. NOT for basic supervised learning, tractable optimization, or domains without search bottlenecks.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Organize repeated experiments to improve models, prompts, or agent systems against measurable goals. The skill records hypotheses and lessons in a persistent tree, isolates trials, and compares promising candidates using evaluation data kept separate from development.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Manage the credentials Arize uses to call LLM providers such as OpenAI and Anthropic. Create, inspect, update, or delete connections, configure allowed models and authentication methods, and retrieve connection IDs for use in evaluations.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Create and run evaluations in Arize using an LLM as the judge. Define scoring criteria, connect them to actual data fields, evaluate recorded operations or experiment results, and set up continuous scoring for new data.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Train sentence-transformers models for search, similarity, and reranking, which reorders search results by relevance. Choose a model type, prepare data, configure training, compare results against a baseline, and publish the trained model to Hugging Face Hub.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Train sentence-transformers models for search, similarity, and reranking, which reorders search results by relevance. Choose a model type, prepare data, configure training, compare results against a baseline, and publish the trained model to Hugging Face Hub.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Use when measuring or improving agent quality and performance — set up evaluators, online monitoring, CI/CD quality gates, observability, or cost optimization. Triggers on: "evaluate my agent", "add evaluator", "measure quality", "quality gate", "run evals", "agent too slow", "why is it slow", "reduce latency", "set up observability", "CloudWatch dashboard", "how much does my agent cost", "cost optimization", "logs not showing up", "logs missing", "spans not found", "eval failing", "eval error", "dev traces", "local traces", "agentcore dev traces", "traces to CloudWatch". Not for debugging errors or crashes — use agents-debug. Slow but correct routes here; broken routes to debug.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Builds, configures, debugs, and optimizes AWS observability - operator-symptom questions and detecting Omni vs classic CloudWatch. CloudWatch on an already-reporting service: Log Insights, metric/composite/anomaly alarms, custom metrics/EMF, dashboards, X-Ray/ADOT tracing, canaries, CloudTrail, Dynamic Instrumentation (live breakpoints/snapshots), the Application Signals service map, and fleet health views. CloudWatch Omni on an existing Space: SQL over logs and traces, PromQL over metrics, Omni dashboards, Omni alerts, context graph for root cause, programmatic/IaC access (API/SDK/CLI/CloudFormation), driving Omni from a coding agent or skills, and evaluating AI agent quality from traces - on-demand and online scoring of live traffic, readback, custom evaluators. For first-time setup - creating an Omni Space, granting access, ingestion, or ADOT instrumentation for Application Signals (ServiceEvents, CI/CD metadata) or Omni - use setting-up-cloudwatch-observability. Not for app logging or threat detection.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Build a production behavior-tree runtime (Blackboard, action/condition leaves, sequence/selector/parallel composites, decorators) and a Utility AI system (response curves — linear, exponential, sigmoid, quadratic — considerations, and action evaluators), plus hybrid BT-drives-Utility agents. Use when implementing a reusable behavior-tree or utility-based decision system, or tuning enemy/NPC decisions beyond a simple FSM, or when the user mentions behavior tree, blackboard, decorator, selector, sequence, tick status, utility AI, response/scoring curve, or consideration. For choosing between FSM/BT/steering or for pathfinding, use game-ai; for Unreal's BehaviorTree/Blackboard assets, use unreal-behavior-trees.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Write code evaluators for known failure modes with objective rules. Use when code can check the rule from a trace, with or without a reference answer. Use `write-judge-prompt` when the rule requires interpretation.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judge prompt (write-judge-prompt) when you need to verify alignment before trusting its outputs. Do NOT use for code-based evaluators (those are deterministic; test with unit tests per `write-code-eval`).
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Design LLM-as-Judge evaluators for subjective criteria that code-based checks cannot handle. Use when a failure mode requires interpretation (tone, faithfulness, relevance, completeness). Do NOT use when the failure mode can be checked with code (regex, schema validation, execution tests); use `write-code-eval`. To validate an existing judge, use `validate-evaluator`.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when packaging ACM CCS artifacts for the artifact-evaluation committee and the ACM badges — Artifacts Available, Artifacts Evaluated Functional, Artifacts Evaluated Reusable, and Results Reproduced — covering what security evaluators inspect, how to make attacks and defenses turnkey, and how to justify withheld artifacts.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Suede-owned comparison-page discipline for accurate, evidence-backed alternative, versus, and competitor-comparison content that wins evaluators and search intent. Use when planning or writing a public page that positions products against named alternatives from verified evidence. NOT FOR: gathering the underlying competitor evidence (use suede-competitor-profiling), internal battle cards (use suede-sales-enablement), or scaled page generation (use suede-programmatic-seo).
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Create and maintain AgentV YAML evaluation files for testing AI agent performance. Use this skill when creating new eval files, adding eval cases, or configuring custom evaluators (code validators or LLM judges) for agent testing workflows.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
You are an expert in creating competitor comparison and alternative pages. Your goal is to build pages that rank for competitive search terms, provide genuine value to evaluators, and position your product effectively.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when user says "simmer this", "refine this", "hone this", "iterate on this", or asks to improve a specific artifact over multiple rounds. Runs an iterative refinement loop with investigation-first judges that read the code, understand the problem, and propose evidence-based improvements. Auto-selects single judge or multi-judge board based on complexity. Works on any artifact type: documents, prompts, specs, emails, creative writing, API designs, pipelines, codebases. Supports multi-file workspace targets, runnable evaluators, and open-ended optimization (model selection, pipeline topology, prompt tuning).
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder, dense or static embedding model for retrieval, similarity, clustering, classification, paraphrase mining, dedup, multimodal), `CrossEncoder` (reranker, pair scoring for two-stage retrieval / pair classification), `SparseEncoder` (SPLADE, sparse embedding model for learned-sparse retrieval), and `MultiVectorEncoder` (ColBERT / late-interaction, per-token embeddings scored with MaxSim). Covers loss selection, hard-negative mining, evaluators, distillation, LoRA, Matryoshka, and Hugging Face Hub publishing. Use for any sentence-transformers training task.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Build and run evaluators for AI/LLM applications using Phoenix.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
INVOKE THIS SKILL for LLM-as-judge evaluation workflows on Arize: creating/updating evaluators, running evaluations on spans or experiments, tasks, trigger-run, column mapping, and continuous monitoring. Use when the user says: create an evaluator, LLM judge, hallucination/faithfulness/correctness/relevance, run eval, score my spans or experiment, ax tasks, trigger-run, trigger eval, column mapping, continuous monitoring, query filter for evals, evaluator version, or improve an evaluator prompt.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.