'lua' 검색 결과

Assess language models with over 60 academic benchmark tests using consistent prompts and scoring. Compare models, track performance during training, and organize research results with support for HuggingFace models, vLLM, and APIs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare models on standard knowledge and reasoning tests
• Track model performance across training checkpoints
• Organize evaluation results for research reports

Evaluate a machine learning GPU cluster during a cloud trial or acceptance test. Measure GPU computing performance, communication within and between machines, and local and shared storage, then compile the results into a dated Markdown report.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Evaluating trial GPU machines from a cloud provider
• Running acceptance tests for a GPU cluster
• Measuring GPU performance under load from neighboring GPUs

Design Redis caching, leaderboards, notifications, and message processing. Helps choose data structures, write Lua scripts for operations that run together without interference, and plan expiration, memory use, and persistence settings.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Designing caches with expiration times
• Building leaderboards and request rate limits
• Processing messages with Redis Streams

Assess language models with over 60 academic benchmark tests using consistent prompts and scoring. Compare models, track performance during training, and organize research results with support for HuggingFace models, vLLM, and APIs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare models on standard knowledge and reasoning tests
• Track model performance across training checkpoints
• Organize evaluation results for research reports

Add team-specific evaluation criteria to plugin-eval by designing a metric pack, a set of custom checks and measurements. Create its manifest and JSON-producing script, then run it through the evaluator using a consistent output structure.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Add team-specific evaluation criteria
• Define custom checks and numerical metrics
• Build a metric-pack manifest and script

Evaluate AI agents by preparing test cases, scoring responses and tool use, investigating failures, and comparing results after changes. Supports built-in checks and custom scoring rules, including evaluations performed by a language model.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Create test cases for an AI agent
• Check response quality and tool use
• Investigate failed evaluation cases

Advanced Evaluation workflow skill. Use this skill when the user needs This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment and the operator should preserve the upstream workflow, copied support files, and provenance before merging or handing off.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Evaluate Jira ticket complexity + inconsistencies via the jira-evaluator agent. Read-only. Triggers: 'evaluate KEY', 'analyze ticket complexity for KEY', 'check ticket quality for KEY'. NOT for estimation (mk:jira-estimator); NOT for full RCA (mk:jira-analyst).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Investigate AI observability evaluations of both types — `hog` (deterministic code-based) and `llm_judge` (LLM-prompt-based). Find existing evaluations, inspect their configuration, run them against specific generations, query individual pass/fail results, and generate AI-powered summaries of patterns across many runs. Use when the user asks to debug why an evaluation is failing, surface common failure modes, compare results across filters, dry-run a Hog evaluator, prototype a new LLM-judge prompt, or manage the evaluation lifecycle (create, update, enable/disable, delete).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

공통 과제로 대규모 언어 모델의 지식·추론·코드 생성을 평가합니다. 모델 비교표와 학습 단계별 성적 추이를 만들고 평가 결과와 개별 답변을 저장합니다.

• MMLU와 GSM8K로 모델 평가
• 여러 모델의 성적 비교표 작성
• 학습 중 모델의 성적 추적

Select, inspect, validate, and report Paperclip evaluations for Runner protocol behavior or complete product workflows. Interpret results with evidence, source revisions, costs, and failure categories, and refresh reports from retained evidence without new provider calls.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choosing between Runner and Product E2E evaluations
• Inspecting evaluation catalogs without credentials
• Interpreting failures using evidence and execution records

Estimate a public company's intrinsic value per share using discounted cash flow, a method that values future cash in today's terms. Produce a Markdown report documenting financial inputs, assumptions, five-year forecasts, and sensitivity to valuation assumptions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Estimate a public company's fair value per share
• Compare the current share price with estimated value
• Check how valuation changes under different assumptions

Train and evaluate models that estimate human poses from WiFi signals. Compare results against a model that always predicts the average pose, check for data leakage, and report measured improvements with clear evaluation conditions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Train a pose model using camera labels
• Evaluate pose estimation without a camera at inference
• Compare accuracy against an average-pose baseline

Create a fixed JSON evaluation suite from a repository's existing tests and check its structure for errors. Use the suite with harness-evolve to compare variants against consistent tasks, independently of the repository's normal test command.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Create a fixed evaluation suite from existing tests
• Check evaluation suite structure in CI
• Compare variants across commits using consistent tasks

Add support for a programming language to codegraph, from connecting its parser and implementing code extraction to writing tests. Evaluate extraction quality and compare agent effort with and without codegraph across three real codebases.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Adding a new programming language to codegraph
• Testing extraction of functions, imports, and calls
• Comparing agent effort across three real codebases

Build financial models for valuation and investment analysis using discounted future cash flows, assumption testing, and probability simulations. Outputs include Excel models, comparison tables, charts, and valuation ranges to examine possible outcomes and risks.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build a company valuation model from financial statements
• Test how growth and discount rates affect valuation
• Simulate acquisition valuation under uncertain assumptions

Build and evaluate time-to-event models with scikit-survival, including records whose events remain unobserved when follow-up ends. Keep preprocessing separate from evaluation data, compare models, predict survival probabilities, and summarize censoring-aware metrics.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Validate event indicators and follow-up times in local CSV data
• Train survival models without leaking evaluation data
• Evaluate survival probabilities with censoring-aware metrics

Create consistent records for evaluating AI agents, including task success, invalid tool calls, token use, and actual costs. Produce CSV tables, database definitions, and JSON validation schemas to support reproducible comparisons and checks for performance declines.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Recording agent evaluation results in a shared format
• Comparing a candidate agent with a baseline
• Tracking actual cost per solved task

Turns supplied AI agent evaluation results into reports that distinguish autonomous success, human-assisted success, failure, timeout, and invalid runs. Makes metric denominators, retries, timing, costs, and comparison conditions explicit to support evidence-based decisions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Separating autonomous and human-assisted success rates
• Reporting evaluations with retries and invalid runs
• Comparing agents or releases under matched conditions

Evaluate LLM applications using automated scores, human reviews, and model-based judging. Build repeatable checks to compare models or prompts, assess answer quality, and detect declines in performance before deployment.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing models and prompts
• Checking quality after prompt changes
• Detecting performance declines before deployment

Build and run evaluations for AI applications using Phoenix. Start with observed failures, create checks in code or use a language model as a judge, run experiments, and validate automated judgments against human labels.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Turning observed failures into evaluation criteria
• Building code-based or language-model evaluators
• Evaluating retrieval and answer faithfulness in RAG

Build an end-to-end evaluation pipeline for Python AI applications using pixie-qa and real language-model calls. Define criteria, prepare realistic test data, score application runs, and analyze the evidence to produce a prioritized improvement plan.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up quality checks for Python AI applications
• Creating realistic scenarios for known failure modes
• Scoring full application runs with real language-model calls

Run repeated change-and-evaluation cycles for one predefined improvement mission within an explicit runtime limit. Save each evaluation as JSON and each decision in readable Markdown logs, with support for periodic reruns through Claude Code native cron.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Repeating improvements under a fixed evaluation method
• Keeping evaluation results and decision logs
• Running experiments within a time limit

Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating failures in LLM applications
• Evaluating answers against test datasets
• Monitoring production latency and costs