Results for “bench”

Reverse-engineer raw materials (Sparse idea, Dense idea, experimental log) from an existing AI research paper to build a benchmark case for evaluating paper-writing pipelines. Replicates the PaperWritingBench dataset construction procedure from arXiv:2604.05018 §3 / App. C. TRIGGER when the user asks to "build a benchmark case from this paper", "reverse-engineer raw materials", or "evaluate my pipeline against PaperWritingBench".

The English summary is being prepared. Showing the original description.

Use when generating analysis reports for WB-Bench-Office, WB-Bench-Web, or WB-Bench-Code from a single Harbor evaluation run artifact directory; route to the matching reference workflow.

The English summary is being prepared. Showing the original description.

Use when running throughput/latency (serving speed) benchmarks against an already-running model server. Uses sgl_jax.bench_serving to sweep batch-size/concurrency and measure throughput and latency, producing summary.csv and per-point bs_<N>/result.jsonl. Speed only — accuracy/eval belongs to model-accuracy-benchmark. Triggers on throughput, latency, bench_serving, batch-size sweep, tok/s, TTFT, TPOT — not accuracy/eval.

The English summary is being prepared. Showing the original description.

dontbesilent 对标分析。用五重过滤法帮你找到值得模仿的对标，排除一切关于「我」的噪音。 触发方式：/dbs-benchmark、/对标、「帮我找对标」「我该模仿谁」 Benchmark analysis using dontbesilent's five-filter method. Trigger: /dbs-benchmark, "find me a benchmark", "who should I copy"

The English summary is being prepared. Showing the original description.

Reverse-engineer raw materials (Sparse idea, Dense idea, experimental log) from an existing AI research paper to build a benchmark case for evaluating paper-writing pipelines. Replicates the PaperWritingBench dataset construction procedure from arXiv:2604.05018 §3 / App. C. TRIGGER when the user asks to "build a benchmark case from this paper", "reverse-engineer raw materials", or "evaluate my pipeline against PaperWritingBench".

The English summary is being prepared. Showing the original description.

解析中文自然语言中的 AISBench 测评需求，检查 Python 与 AISBench 运行环境，通过分轮追问确认关键配置，将基准名称映射为仓库内的数据集任务，在用户最终确认后生成并执行 AISBench CLI，并返回精度指标和产物路径。适用于用户要求使用 AISBench 评测模型或运行数据集；不适用于实现新数据集或仅解释基准概念。

The English summary is being prepared. Showing the original description.

Add a new benchmark to sgl-eval by vendoring its NeMo-Skills dataset module and registering it in `_TABLE`. Use when the user asks to "add <benchmark>" / "support <benchmark>" / "register <benchmark>" inside the sgl-eval repo.

The English summary is being prepared. Showing the original description.

Align, run, audit, and publish GEODE benchmark comparisons against Agent-World arXiv 2604.18292v1 across MCP-Mark, BFCL V4, and tau2-bench. Use for Agent-World or AgentWorld comparison requests, cross-suite agent benchmark design, Tau2 comparator/profile diagnosis, paired thin-runtime-versus-GEODE experiments, mean-accuracy@8 replication, or benchmark comparability and artifact reviews.

The English summary is being prepared. Showing the original description.

Measure web page loading times, rendering milestones, and resource sizes in a real browser. Save baseline measurements, compare results before and after code changes, and inspect historical trends to spot performance regressions.

• Saving performance baselines before changes
• Checking page speed after a code change
• Finding slow resources and large transfers

Investigate failed GAIA benchmark questions by inspecting execution records and classifying causes such as missing tools, reasoning errors, or answer extraction bugs. Follow targeted fixes, verify them through another run, and save findings for future debugging.

• Investigate an incorrect answer for a specific task
• Diagnose a question that timed out
• Investigate a drop in benchmark pass rate

Guide GAIA benchmark runs from environment checks and cost estimates to result comparison. Create a signed submission package compatible with HAL, including agent traces, metadata, and an audit report that checks for benchmark exploitation.

• Check readiness for a GAIA benchmark run
• Estimate costs and run an evaluation
• Package existing results for submission

Benchmark security detection systems against 10 known vulnerabilities and nine harmless decoys. Compare four approaches, including an evolved champion, and receive structured JSON covering detection, false alarms, patch tests, reproduction, unsafe outputs, and overall pass or fail.

• Comparing security detection approaches
• Tracking changes in detection and false alarms
• Flagging failed checks in scheduled benchmarks

Measure performance in the agentic-flow worker system, including trigger detection, agent selection, caching, and concurrent operations. Run a full suite or individual benchmarks to compare response times and resource usage against configured targets.

• Checking overall worker system performance
• Measuring trigger detection and agent selection speed
• Examining cache behavior and memory usage

Work with Waypoint models to analyze microbiome samples, the communities of microbes in a sample. Convert taxonomic abundance tables, generate numerical sample representations, configure further training for prediction tasks, and evaluate models on the Compass benchmark.

• Convert microbiome abundance tables into Waypoint inputs
• Generate sample vectors for clustering and visualization
• Configure further training with sample labels

Use PyTDC to discover therapeutic machine-learning datasets, create task-appropriate training and test splits, and evaluate predictions against curated benchmarks. Also score selected molecules with supported scoring functions, with explicit download plans and bounded outputs.

• Find datasets for therapeutic machine-learning tasks
• Plan and create task-appropriate data splits
• Evaluate predictions with benchmark-specific metrics

Create consistent records for evaluating AI agents, including task success, invalid tool calls, token use, and actual costs. Produce CSV tables, database definitions, and JSON validation schemas to support reproducible comparisons and checks for performance declines.

• Recording agent evaluation results in a shared format
• Comparing a candidate agent with a baseline
• Tracking actual cost per solved task

Build an Excel comparison of public companies using financial performance, valuation ratios, and statistical benchmarks. The spreadsheet includes formulas, source notes, medians, and quartiles to show how each company compares with its peers.

• Comparing public company valuations for M&A
• Benchmarking performance against industry peers
• Identifying unusually high or low valuation ratios

Design tests that assess an AI agent's capabilities and reliability. Covers repeated runs to measure variation, checks for required behavior, and adversarial testing to expose failures that ordinary successful examples may miss.

• Measuring variation across repeated runs of the same test
• Checking whether an agent follows required behavior
• Testing responses to unexpected or adversarial inputs

Guides iterative improvement of AI agent prompts, skills, and memory using A-Evolve. It explains how to evaluate agents against benchmark tasks, apply changes, validate results, and track or roll back revisions with Git.

• Improve an existing agent against benchmark tasks
• Build evaluations for domain-specific tasks
• Refine skills and memory from failure feedback

Find AI models for a specific task using official Hugging Face benchmark leaderboards. Compare scores, model sizes, and licenses, with memory-based estimates of device suitability and flags for models requiring quantization or API access.

• Compare coding or reasoning models by benchmark scores
• Find model candidates for a laptop's available memory
• Choose models for OCR or speech recognition

Quickly look up SaaS financial metrics, including recurring revenue and customer acquisition costs. Get definitions, formulas, benchmarks, and warning signs, plus reminders of which metrics matter when reviewing features, pricing, acquisition channels, or business health.

• Looking up NRR formulas and benchmarks
• Preparing a SaaS metrics sheet for a business review
• Checking metrics before expanding an acquisition channel

Evaluate local Codex skills and plugins, understand their scores, and identify fixes to prioritize. Get explanations of token estimates, set up benchmark scenarios, and prepare workflows for measuring actual token usage.

• Evaluate a local skill or plugin
• Understand scores and prioritize fixes
• Explain token estimates and prepare actual measurements

Diagnose problems in a PPT creation setup, including document parsing, browser rendering, PPTX export, and search or image tools. Receive a report showing which capabilities work, which remain unverified, and what configuration or dependencies are missing.

• Diagnosing failed HTML rendering or PPTX export
• Checking dependencies before using PPT tools
• Finding missing search and image API settings

Compare Google and Meta advertising results with industry benchmarks suited to your campaign type, budget, and account context. Identify strengths, likely causes of weak performance, and improvement opportunities ranked by potential impact.

• Finding improvement opportunities for a prospective client
• Adding context to quarterly or annual performance reviews
• Assessing whether customer acquisition costs are reasonable