「benchmark」の検索結果

Experimental/manual harness canary procedure. It records benchmark manifests and scores but does not provide a fully automated runner.

日本語の概要は準備中です。原文の説明を表示しています。

Deterministically score comparable imported review-run lanes with root-cause metrics and uncertainty. Use only when the user explicitly asks to benchmark normalized local review runs or use the bundled synthetic fixture.

日本語の概要は準備中です。原文の説明を表示しています。

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks

日本語の概要は準備中です。原文の説明を表示しています。

Avalia modelos de geração de código em HumanEval, MBPP, MultiPL-E e 15+ benchmarks com métricas pass@k. Use ao comparar modelos de código, avaliar capacidades de codificação, testar suporte multilíngue ou medir qualidade de geração de código. Padrão da indústria do BigCode Project usado pelos leaderboards do HuggingFace.

日本語の概要は準備中です。原文の説明を表示しています。

Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill, benchmark skill performance with variance analysis, or optimize a skill's description for better triggering accuracy. Triggers: create a skill, new skill, build a skill, skill creation, write a SKILL.md, evaluate skill, test skill, skill eval, benchmark skill, optimize skill description.

日本語の概要は準備中です。原文の説明を表示しています。

Benchmark CodeGraph retrieval quality on a real codebase by comparing agent behavior with vs without CodeGraph. Use when the user runs /agent-eval or asks to test, benchmark, audit, or validate a codegraph version (the local dev build or a published npm version) against a language's repo.

日本語の概要は準備中です。原文の説明を表示しています。

Compare model candidates using weighted metrics and deterministic ranking outputs. Use for benchmark leaderboards and model promotion decisions.

日本語の概要は準備中です。原文の説明を表示しています。

Compare standardized metrics across multiple ego-benchmark-harness runs by recalculating them from tasks.jsonl and session JSONL, never from HTML reports. Covers overall aggregates (success rate/average score), Odysseys per-rubric pass rates, duration distributions (avg/p50/p90/max), turns/tools/cost, error and exception counts, per-task success matrices, helper distributions, and the share of DOM operations. Use whenever users ask to compare data or metrics across runs, calculate aggregate/Odysseys/duration/turn/error/helper metrics, inspect differences among runs, or compare multiple rwb/real-world-bench runs. Use pi-session-analyzer instead for deep forensic analysis of why an individual task was slow or failed.

日本語の概要は準備中です。原文の説明を表示しています。

Analyze Codex provider rollout JSONL and ego-benchmark-harness run artifacts to identify slow or failed individual tasks, tool retries, token/cost anomalies, and differences in agent execution paths. Use when users ask to analyze a Codex session or rollout, explain why a Codex task was slow or failed, inspect a Codex provider execution trace, compare Codex runs, count function_call or tool output events, or investigate agent-home/.codex/sessions.

日本語の概要は準備中です。原文の説明を表示しています。

Analyze pi (coding-agent) session JSONL and ego-benchmark-harness run artifacts to explain slow or failed individual tasks, compare execution-path differences between agents or browser tools, and count tool calls and error types. Use this skill whenever users ask to analyze a pi session, explain why a task was slow or failed, compare two runs, investigate an error, locate the session corresponding to tasks.jsonl, inspect an agent execution path, count tool calls, analyze a run ID, or perform any forensic or retrospective analysis of JSONL under ~/.pi/agent/sessions/ or agent-home/.pi/agent/sessions/.

日本語の概要は準備中です。原文の説明を表示しています。

Author or expand browser-agent benchmark tasks in data/real_world_bench.json (Odysseys schema, per-rubric graded). Use when creating, adding, or expanding real-world browser tasks, writing rubrics, or picking bot-friendly sites — every task MUST be verified runnable through ego-browser before it is written.

日本語の概要は準備中です。原文の説明を表示しています。

Use when working with Apptio — apptio and Cloudability IT financial management and cloud cost optimization. Covers IT cost allocation, cloud cost analysis, benchmarking, budget forecasting, showback/chargeback reports, and technology business management. Use when managing IT financial planning, benchmarking cloud spend, or creating chargeback models.

日本語の概要は準備中です。原文の説明を表示しています。

Use when: benchmark model latency, throughput, or quality.

日本語の概要は準備中です。原文の説明を表示しています。

Use when the user asks about finding the best, top, or recommended model for a task, wants to know what AI model to use, or wants to compare models by benchmark scores. Triggers on: "best model for X", "what model should I use for", "top models for [task]", "which model runs on my laptop/machine/device", "recommend a model for", "what LLM should I use for", "compare models for", "what's state of the art for", or any question about choosing an AI model for a specific use case. Always use this skill when the user wants model recommendations or comparisons, even if they don't explicitly mention HuggingFace or benchmarks.

日本語の概要は準備中です。原文の説明を表示しています。

Expert en évaluation d'agents IA (benchmarking, human eval, latence, coût, précision, safety testing)

日本語の概要は準備中です。原文の説明を表示しています。

Design and run reproducible benchmarks. Use when measuring performance scaling, comparing branches, or analyzing speedup metrics.

日本語の概要は準備中です。原文の説明を表示しています。

Orchestrate Xcode build optimization by benchmarking first, running the specialist analysis skills, prioritizing findings, requesting explicit approval, delegating approved fixes to xcode-build-fixer, and re-benchmarking after changes. Use when a developer wants an end-to-end build optimization workflow, asks to speed up Xcode builds, wants a full build audit, or needs a recommend-first optimization pass covering compilation, project settings, and packages.

日本語の概要は準備中です。原文の説明を表示しています。

bu skill covers hardening container images by minimizing attack surface, removing unnecessary packages, implementing multi-stage builds, configuring non-root users, and applying CIS Docker Benchmark recommendations to produce secure production-ready images.

日本語の概要は準備中です。原文の説明を表示しています。

Docker Bench for Security is an open-source script that checks dozens of common best practices around Dağıt:ing Docker containers in production. Based on the CIS Docker Benchmark, it audits host confi

日本語の概要は準備中です。原文の説明を表示しています。

Hardening Docker containers for production involves applying security best practices aligned with CIS Docker Benchmark v1.8.0 to minimize attack surface, prevent privilege escalation, and enforce leas

日本語の概要は準備中です。原文の説明を表示しています。

Performing comprehensive security assessments of Google Cloud Platform environments using Forseti Security, Security Command Center, and gcloud CLI to audit IAM policies, firewall rules, storage permissions, and compliance against CIS GCP Foundations Benchmark.

日本語の概要は準備中です。原文の説明を表示しています。

Harden the Docker daemon by configuring daemon.json with user namespace remapping, TLS authentication, rootless mode, and CIS benchmark controls.

日本語の概要は準備中です。原文の説明を表示しています。

Generate HTML reports and structured metrics from model evaluation results. Creates publication-ready reports with embedded outputs (images, audio, charts) and metrics.json for cross-model comparison. Use when generating reports, writing metrics, creating evaluation summaries, or formatting benchmark results. Triggers on "generate report", "write metrics", "create report", "evaluation summary", "benchmark results", "format results".

日本語の概要は準備中です。原文の説明を表示しています。

Continuous improvement system for ToolUniverse tools, skills, and plugin. Run benchmarks, diagnose failures, route fixes to devtu skills, retest. Use after skill optimization, tool additions, or as regression check.

日本語の概要は準備中です。原文の説明を表示しています。