“evaluation”的搜索结果

Compare agent, prompt, or tool configuration changes using versioned test cases and explicit outcome checks. Record each attempt, distinguish agent failures from evaluation or infrastructure problems, and report regressions, permission violations, and uncertainty.

中文简介正在准备中。当前显示英语简介。

• Compare agent behavior before and after prompt changes
• Reproduce an observed failure with preserved evidence
• Run regression tests after a bounded fix

Configure supervised fine-tuning for models that process images and text. Define which components remain unchanged, set LoRA adapters for updating selected model weights, choose image resolution limits, and investigate training runs whose evaluation results fail to improve.

中文简介正在准备中。当前显示英语简介。

• Configure training for an image-and-text task
• Decide whether to train vision layers
• Investigate training with flat evaluation scores

Investigate LLM application failures with Phoenix CLI by reviewing execution records, recording observations, and grouping mistakes into counted categories. Inspect datasets and experiments to identify which failure types deserve evaluation tests first.

中文简介正在准备中。当前显示英语简介。

• Investigating failures in LLM applications
• Grouping observed mistakes into failure categories
• Choosing which evaluation tests to build first

Build and run evaluations for AI applications using Phoenix. Start with observed failures, create checks in code or use a language model as a judge, run experiments, and validate automated judgments against human labels.

中文简介正在准备中。当前显示英语简介。

• Turning observed failures into evaluation criteria
• Building code-based or language-model evaluators
• Evaluating retrieval and answer faithfulness in RAG

Improve LLM prompts using Arize production traces, evaluation results, and human feedback. Extract current instructions, identify recurring failures, revise the prompt, and compare experiments to measure changes in output quality and catch regressions.

中文简介正在准备中。当前显示英语简介。

• Improve a prompt using failed production traces
• Fix responses that use the wrong output format
• Review unsupported answers in retrieval-based responses

Run performance improvements as a measured workflow in OMX and Codex goal mode. Define an evaluation command and success criteria, make small reversible changes, and retain evidence before declaring the work complete.

中文简介正在准备中。当前显示英语简介。

• Reducing CLI startup time against a defined target
• Repeating improvements with benchmarks and regression tests
• Recording evaluation results and blockers

Build AI workflows with DSPy by defining inputs and outputs and combining reusable components. Create question answering systems that use retrieved documents, tool-using agents, and automatically optimized prompts, then compare results with evaluation data.

中文简介正在准备中。当前显示英语简介。

• Build question answering systems that use retrieved documents
• Optimize prompts using examples and evaluation metrics
• Combine AI tasks into reusable processing stages

Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.

中文简介正在准备中。当前显示英语简介。

• Investigating failures in LLM applications
• Evaluating answers against test datasets
• Monitoring production latency and costs

Design tests that assess an AI agent's capabilities and reliability. Covers repeated runs to measure variation, checks for required behavior, and adversarial testing to expose failures that ordinary successful examples may miss.

中文简介正在准备中。当前显示英语简介。

• Measuring variation across repeated runs of the same test
• Checking whether an agent follows required behavior
• Testing responses to unexpected or adversarial inputs

Create evaluation drafts for existing skills using actual conversations and user corrections. Produce criteria and repeatable test cases, distinguish evidence from specification assumptions, and prepare an anonymized draft for human review.

中文简介正在准备中。当前显示英语简介。

• Turn user corrections into repeatable test cases
• Compare promised behavior with actual usage
• Draft an evaluation from conversation history

Learn how to author a third-party gbrain skillpack through a reference example. It explains required files, ten validation criteria, tests, and evaluation coverage, helping authors understand what to check before packaging and publishing.

中文简介正在准备中。当前显示英语简介。

• Learning the structure of a gbrain skillpack
• Checking requirements before publication
• Understanding test and evaluation files

Build and maintain a medical device quality management system using ISO 13485:2016 workflows. It supports gap analysis, controlled documentation, internal audits, corrective and preventive actions, process validation, and supplier evaluation, with templates and an audit checklist generator.

中文简介正在准备中。当前显示英语简介。

• Planning a medical device quality management system
• Standardizing document approval and revision control
• Preparing internal audits and certification reviews

Prepare a graded Claude Managed Agent for recurring schedules, event triggers, or on-demand use. Validate timing, build deployment request data, optionally include self-evaluation for each run, and document the next steps in a roadmap.

中文简介正在准备中。当前显示英语简介。

• Schedule an agent to run every morning or weekly
• Prepare a curl trigger for an external event
• Add self-evaluation to each recurring run

Plan a Claude Managed Agent around one clear job, then route work through setup, launch preparation, evaluation, and scheduling. Produce execution plans and curl commands that users run with their own Anthropic API key.

中文简介正在准备中。当前显示英语简介。

• Defining one clear job for a managed agent
• Preparing launch commands for an Anthropic account
• Planning evaluation and revision with an iteration limit

Create and improve prompts for agents, workflows, and AI evaluations. Reusable Handlebars templates, which insert data into a fixed structure, help generate consistent instructions from YAML or JSON inputs for use in other tasks.

中文简介正在准备中。当前显示英语简介。

• Improving prompts for recurring tasks
• Generating agent briefs from templates
• Creating AI evaluation and scoring prompts

Improve code, prompts, skills, or agent definitions through repeated changes and evaluation. Measure numerical targets such as latency or size, or use an LLM to judge pass-or-fail criteria, keeping changes that improve the result.

中文简介正在准备中。当前显示英语简介。

• Improving page load performance
• Reducing build output size
• Refining prompts against evaluation criteria

Design the controls around autonomous agents: what they can edit, how results are evaluated, how progress is logged, and when human approval is required. Build research and experiment loops with clear recovery and review rules.

中文简介正在准备中。当前显示英语简介。

• Designing autonomous research and experiment loops
• Separating editable work from fixed evaluation rules
• Preserving progress and failed attempts across long runs

Write and review launch prompts for agents working autonomously for hours or days. Define successful deliverables, excluded partial results, stopping rules, worker handoffs, and evidence-based review requirements so completion can be assessed against explicit criteria.

中文简介正在准备中。当前显示英语简介。

• Turning vague requests into precise task briefs
• Defining stopping rules for long autonomous runs
• Specifying deliverables for parallel workers

Design systems that use language models to judge AI responses. Covers scoring criteria, comparisons between two answers, bias reduction, confidence calibration, and checks against human judgments to make automated quality assessments more consistent and traceable.

中文简介正在准备中。当前显示英语简介。

• Compare responses from different models
• Run A/B tests for prompt changes
• Create shared scoring rubrics

Plan and improve AI agent systems by managing the information models receive. This collection covers memory, agent coordination, tool design, context compression, evaluation, and the operating controls needed for reliable autonomous workflows.

中文简介正在准备中。当前显示英语简介。

• Planning information management for a new AI agent
• Investigating failures in long agent conversations
• Designing agent memory and multi-agent coordination

Design loops that improve an agent’s own prompts, workflows, or execution code from failure records. Define bounded edits, preserve candidate histories, and accept changes through independent tests that check improvements and regressions.

中文简介正在准备中。当前显示英语简介。

• Designing agent self-improvement from failure records
• Defining acceptance checks for self-modifying workflows
• Diagnosing score gaming and stalled improvement

Monitor model training through WandB metrics to detect invalid numbers, worsening loss, and declining evaluation results. Ambiguous trends receive a Codex review, while stopped runs retain key metrics and the reason for stopping.

中文简介正在准备中。当前显示英语简介。

• Check loss and evaluation trends during model training
• Detect NaN values and diverging loss
• Review flat or noisy training metrics

Plan a 10–12 page systems research paper for conferences such as OSDI and SOSP. Get section page budgets, paragraph roles, and writing patterns that connect the research problem, design decisions, and evaluation evidence.

中文简介正在准备中。当前显示英语简介。

• Planning sections within a systems conference page limit
• Structuring an abstract and introduction
• Explaining design choices alongside alternatives

Audit research experiments before using their results to support claims. An independent model checks reference data, metric calculations, result files, and evaluation scope, producing reports with evidence, findings, and specific actions to address concerns.

中文简介正在准备中。当前显示英语简介。

• Verify the source of reference data
• Check score normalization and metric calculations
• Match paper claims against saved results