“eval”的搜索结果

Create consistent records for evaluating AI agents, including task success, invalid tool calls, token use, and actual costs. Produce CSV tables, database definitions, and JSON validation schemas to support reproducible comparisons and checks for performance declines.

中文简介正在准备中。当前显示英语简介。

• Recording agent evaluation results in a shared format
• Comparing a candidate agent with a baseline
• Tracking actual cost per solved task

Turns supplied AI agent evaluation results into reports that distinguish autonomous success, human-assisted success, failure, timeout, and invalid runs. Makes metric denominators, retries, timing, costs, and comparison conditions explicit to support evidence-based decisions.

中文简介正在准备中。当前显示英语简介。

• Separating autonomous and human-assisted success rates
• Reporting evaluations with retries and invalid runs
• Comparing agents or releases under matched conditions

Prepare an evaluation framework before fine-tuning a language model. Build labeled examples, checks for each failure type, and a base-model benchmark, while calibrating model-based judges against human labels to support comparisons after training.

中文简介正在准备中。当前显示英语简介。

• Build an evaluation set from agent traces
• Create separate checks for each failure type
• Calibrate an LLM judge against human labels

Evaluate LLM applications using automated scores, human reviews, and model-based judging. Build repeatable checks to compare models or prompts, assess answer quality, and detect declines in performance before deployment.

中文简介正在准备中。当前显示英语简介。

• Comparing models and prompts
• Checking quality after prompt changes
• Detecting performance declines before deployment

Run repeated change-and-evaluation cycles for one predefined improvement mission within an explicit runtime limit. Save each evaluation as JSON and each decision in readable Markdown logs, with support for periodic reruns through Claude Code native cron.

中文简介正在准备中。当前显示英语简介。

• Repeating improvements under a fixed evaluation method
• Keeping evaluation results and decision logs
• Running experiments within a time limit

Evaluate LLM application quality using automated scores, human reviews, and another language model as a judge. Compare models or prompts, establish baseline results, and detect quality drops through repeatable tests and statistical comparisons.

中文简介正在准备中。当前显示英语简介。

• Comparing models and prompts
• Checking quality before deployment
• Evaluating search quality in RAG applications

Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.

中文简介正在准备中。当前显示英语简介。

• Investigate AI application problems using execution traces
• Evaluate answer quality across a test dataset
• Compare prompts and models through experiments

Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.

中文简介正在准备中。当前显示英语简介。

• Investigating failures in LLM applications
• Evaluating answers against test datasets
• Monitoring production latency and costs

Turn a feature into an agent skill or improve an existing one. Define purpose-specific quality criteria, review outputs with models from three providers, then add tests and check that user requests reach the intended skill.

中文简介正在准备中。当前显示英语简介。

• Turn a reusable feature into an agent skill
• Audit an existing skill for missing parts
• Check quality after fixing or extending a skill

Improve prompts through test cases and baseline comparisons, measure how well AI answers use retrieved sources, and check agent configurations. Includes Python tools for prompt analysis, retrieval quality reports, and token estimates, with costs calculated from supplied rates.

中文简介正在准备中。当前显示英语简介。

• Compare prompt revisions against a baseline
• Measure answer quality against retrieved sources
• Check agent tool settings and loop risks

Evaluate a Claude Managed Agent against a written rubric, a set of checkable criteria, and guide improvements within a fixed attempt limit. After a version passes, prepare separate evaluation cases to check performance beyond the examples used for revision.

中文简介正在准备中。当前显示英语简介。

• Defining checkable criteria for agent outputs
• Improving an agent within a fixed attempt limit
• Testing a passing version on unseen cases

Evaluate Google Cloud workloads against built-in best practices or custom organizational rules using Workload Manager. Use public client libraries or the REST API to run checks, review violations and scanned resources, and export results to BigQuery.

中文简介正在准备中。当前显示英语简介。

• Evaluate workloads against Google Cloud best practices
• Inspect rules for SAP and SQL Server workloads
• Run evaluations using custom organizational rules

Improve code, prompts, skills, or agent definitions through repeated changes and evaluation. Measure numerical targets such as latency or size, or use an LLM to judge pass-or-fail criteria, keeping changes that improve the result.

中文简介正在准备中。当前显示英语简介。

• Improving page load performance
• Reducing build output size
• Refining prompts against evaluation criteria

Train sentence-transformers models for search, similarity, and reranking, which reorders search results by relevance. Choose a model type, prepare data, configure training, compare results against a baseline, and publish the trained model to Hugging Face Hub.

中文简介正在准备中。当前显示英语简介。

• Fine-tuning an embedding model for domain-specific search
• Training a model to rerank search results
• Training SPLADE or ColBERT retrieval models

Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.

中文简介正在准备中。当前显示英语简介。

• Investigate AI application problems using execution traces
• Evaluate answer quality across a test dataset
• Compare prompts and models through experiments

Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.

中文简介正在准备中。当前显示英语简介。

• Investigating failures in LLM applications
• Evaluating answers against test datasets
• Monitoring production latency and costs

Train sentence-transformers models for search, similarity, and reranking, which reorders search results by relevance. Choose a model type, prepare data, configure training, compare results against a baseline, and publish the trained model to Hugging Face Hub.

中文简介正在准备中。当前显示英语简介。

• Fine-tuning an embedding model for domain-specific search
• Training a model to rerank search results
• Training SPLADE or ColBERT retrieval models

Evaluate code against a product requirements document, scoring implementation and tests separately for each acceptance criterion. Produce an evidence-backed final grade, identify gaps, and compare implementations using a fixed checklist and scoring method.

中文简介正在准备中。当前显示英语简介。

• Checking whether a feature fulfills its requirements
• Comparing implementations of the same specification
• Auditing unit and end-to-end test coverage

Audit AGENTS.md, rules, skills, and their references for broken paths or commands. Optional independent agent reviews assess redundant instructions and practical usefulness, producing reports that identify what to retain, review, or consider trimming.

中文简介正在准备中。当前显示英语简介。

• Checking paths and commands cited in agent instructions
• Finding redundant rules across skills and instructions
• Assessing which instruction files change agent behavior

Evaluate staged Git changes before committing using an LLM, a language model acting as a reviewer. Receive correctness, completeness, and safety scores, plus issues by severity, to decide whether to fix, commit, or stop.

中文简介正在准备中。当前显示英语简介。

• Evaluate significant changes before committing
• Review changes in unfamiliar code
• Check changes to security-sensitive code

Guide development of a LangChain or LangGraph project created with agents-cli. It explains tool additions, local runs, evaluation, and serving while identifying the required graph structure and commands that differ from ADK projects.

中文简介正在准备中。当前显示英语简介。

• Adding tools to an existing agent
• Running and evaluating an agent locally
• Editing a graph while preserving its required interface

Manage crypto asset cost basis in TRES Finance, from choosing how acquisition costs are assigned to reviewing realized gains and losses. Recalculate results, investigate missing prices, manage revaluations, and export cost basis reports.

中文简介正在准备中。当前显示英语简介。

• Reviewing cost basis and realized gains by asset
• Changing the costing method and recalculating
• Filling missing fiat prices

Develop a broad research interest into evaluated project options. Specialized agents generate ideas, assess feasibility and competition, and draft abstracts and conclusions, helping researchers decide what to pursue, postpone, or drop and which assumption to test first.

中文简介正在准备中。当前显示英语简介。

• Turn a broad research interest into project ideas
• Compare the novelty and feasibility of research proposals
• Check competing work before choosing a direction

Evaluate papers, research proposals, and literature reviews using structured criteria. Produces evidence-based feedback on research questions, methods, data, analysis, and writing, with optional scores and prioritized recommendations tailored to the work's stage and purpose.

中文简介正在准备中。当前显示英语简介。

• Reviewing a paper before submission
• Assessing a research proposal's methods and feasibility
• Checking the coverage and synthesis of a literature review