'evaluation' 검색 결과

Design the controls around autonomous agents: what they can edit, how results are evaluated, how progress is logged, and when human approval is required. Build research and experiment loops with clear recovery and review rules.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Designing autonomous research and experiment loops
• Separating editable work from fixed evaluation rules
• Preserving progress and failed attempts across long runs

Design systems that use language models to judge AI responses. Covers scoring criteria, comparisons between two answers, bias reduction, confidence calibration, and checks against human judgments to make automated quality assessments more consistent and traceable.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare responses from different models
• Run A/B tests for prompt changes
• Create shared scoring rubrics

Plan and improve AI agent systems by managing the information models receive. This collection covers memory, agent coordination, tool design, context compression, evaluation, and the operating controls needed for reliable autonomous workflows.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Planning information management for a new AI agent
• Investigating failures in long agent conversations
• Designing agent memory and multi-agent coordination

Monitor model training through WandB metrics to detect invalid numbers, worsening loss, and declining evaluation results. Ambiguous trends receive a Codex review, while stopped runs retain key metrics and the reason for stopping.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Check loss and evaluation trends during model training
• Detect NaN values and diverging loss
• Review flat or noisy training metrics

Plan a 10–12 page systems research paper for conferences such as OSDI and SOSP. Get section page budgets, paragraph roles, and writing patterns that connect the research problem, design decisions, and evaluation evidence.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Planning sections within a systems conference page limit
• Structuring an abstract and introduction
• Explaining design choices alongside alternatives

Audit research experiments before using their results to support claims. An independent model checks reference data, metric calculations, result files, and evaluation scope, producing reports with evidence, findings, and specific actions to address concerns.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Verify the source of reference data
• Check score normalization and metric calculations
• Match paper claims against saved results

Turn a research proposal into an experiment plan tied to specific paper claims. Define comparisons, evaluation measures, tests that remove individual components, run order, and compute estimates, with a written plan and tracker for execution.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Turning a research proposal into concrete experiments
• Designing comparisons and component-removal tests
• Prioritizing runs within a compute budget

Build MCP servers, which give AI agents tools for external services, using Python or TypeScript. The skill guides tool design, input validation, error handling, code review, and evaluations based on realistic tasks.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build an MCP server for an external API
• Design tools around complete agent workflows
• Review input validation and error handling

Build MCP servers, which let AI agents use external APIs and services through tools. The skill guides tool design, authentication, input validation, implementation, testing, and evaluation using TypeScript or Python.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Connecting an external API to an AI agent through MCP
• Building an MCP server in TypeScript or Python
• Designing tool inputs, responses, and error messages

Set up a CARLA driving evaluation environment for Leaderboard 1.0, 2.0, or 2.1. Match the simulator and scenario_runner versions, configure Python paths, and verify that the required components load correctly.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an environment to evaluate a driving agent
• Checking version differences behind inconsistent scores
• Recreating the Leaderboard 1.0 evaluation environment

Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating failures in LLM applications
• Evaluating answers against test datasets
• Monitoring production latency and costs

Build AI workflows with DSPy by defining inputs and outputs and combining reusable components. Create question answering systems that use retrieved documents, tool-using agents, and automatically optimized prompts, then compare results with evaluation data.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build question answering systems that use retrieved documents
• Optimize prompts using examples and evaluation metrics
• Combine AI tasks into reusable processing stages

Navigate Apache Beam contributions from finding an issue to submitting a pull request, a proposed code change, and responding to reviews. Covers environment setup, tests, formatting, licensing requirements, and closer evaluation of changes to core components.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Prepare a first Apache Beam contribution
• Organize a pull request and its related issue
• Check tests and formatting before submission

Assesses smart contract and blockchain codebases against Trail of Bits’ nine-category maturity framework. Produces evidence-backed ratings covering areas such as arithmetic, access controls, documentation, and testing, plus a prioritized improvement roadmap with effort estimates.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Assessing the maturity of a smart contract project
• Identifying gaps in tests and documentation
• Reviewing access controls and operational practices together

Improve an evaluated Codex skill by turning plugin-eval findings into prioritized changes. Refine activation descriptions, shorten SKILL.md, move lengthy details into supporting files, fix structural issues, and compare evaluation results before and after the rewrite.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Prioritizing fixes after a skill evaluation
• Shortening SKILL.md and reducing token use
• Clarifying when a skill should activate

Submit security scans for AI services, tools, skills, and agents through AI-Infra-Guard, then retrieve progress and results. It also supports jailbreak evaluation, testing whether prompts can bypass a model’s safety restrictions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Scanning AI services for known vulnerabilities
• Auditing MCP tools and skill source packages
• Checking agents from saved settings or YAML files

Work on a git escrow bounty by reading its tests, writing solution code, committing it, and submitting it for automated evaluation. Also supports submitting an existing solution repository and explains how to collect the token reward after approval.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reviewing an open bounty's tests and reward
• Writing and submitting code to satisfy tests
• Submitting an existing solution repository and commit

Review and improve SKILL.md files using nine evaluation criteria and test prompts. Independent agents compare original and revised versions, while user checkpoints, version history, and result cards help track which changes are retained or rolled back.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Assess skill quality without editing files
• Improve instructions in a specific SKILL.md
• Prioritize weaker skills for revision

Build MCP servers, which let AI agents use external APIs and services through tools. The skill guides tool design, authentication, input validation, implementation, testing, and evaluation using TypeScript or Python.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Connecting an external API to an AI agent through MCP
• Building an MCP server in TypeScript or Python
• Designing tool inputs, responses, and error messages

Register and search existing Git repositories and worktrees, then choose which projects Ouroboros interviews use as context. You can also generate a configuration containing verified build, test, and other check commands for the evaluation workflow.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Selecting existing projects as interview context
• Finding registered repositories by topic
• Rescanning after repositories are added or moved

Routes AI development questions to focused references on agents, language model security, retrieval-augmented generation, which uses retrieved information in answers, and prompt design and evaluation. It also outlines principles for validation, safety boundaries, version control, and ongoing assessment.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Find guidance for agent development
• Locate references on prompt injection defenses
• Explore retrieval and reranking for RAG

Find scientific papers in PubMed by turning a research question into targeted search terms and synonyms. Collect titles, authors, abstracts, and identifiers into a structured paper list ready for relevance evaluation.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Finding an initial paper set for a research question
• Searching for methods, compounds, or diseases
• Filtering papers by date, journal, or author

Generate a Chinese master's thesis review from the full thesis and a user-provided score. Produce an academic evaluation plus specific shortcomings and revision suggestions, with tone and criticism adjusted to the supplied score.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Drafting a Chinese master's thesis review
• Matching review wording to an assigned score
• Turning thesis-specific issues into revision suggestions

Develop literature reviews on AI for medical imaging, including segmentation, detection, and classification. Organize research from multiple sources into a structured manuscript with method comparisons, evaluation metrics, supporting citations, and limitations.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Drafting literature reviews on medical imaging AI
• Comparing methods for segmentation, detection, and classification
• Combining literature from ArXiv, PubMed, and Zotero