871건 · 관련도순
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarksUse when "agent testing, agent evaluation, benchmark agents, agent reliability, test agent, testing, evaluation, benchmark, agents, reliability, quality" mentioned.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when benchmarking, profiling, or A/B-testing CUDA kernels or end-to-end perf in the imp inference engine on RTX 5090 (sm_120), including refreshing tests/perf_baseline.json or publishing numbers to docs/BENCHMARKS.md and the README. Triggers on "benchmark kernel", "profile cuda", "ncu", "nsys", "kernel timing", "kernel sum", "occupancy", "bandwidth bound", "compute bound", "roofline", "perf baseline", "is this regression real", "decode dropped", "aggregate throughput", "two-image A/B", "prefill kernel A/B". Do NOT use for writing/optimizing kernel code (sm120-cuda-expert) or output-quality checks (check-degeneration).
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Skippable Phase 0 of the Gaia evidence verification pipeline. Use only on declared need for higher-quality Stage-2 evidence: a promotion candidate, or a skill that gaia-meta-sweep flagged as under-evidenced. Given a skill id, source repo/context, and needed evidence types (benchmark-result, arxiv, peer-review, social-signal), it runs Firecrawl searches, scrapes top hits to markdown, and appends discovered sources into Phase 1 inputs. Missing benchmark evidence is recorded as benchmark-source candidates first, not direct scoring rows. Trigger phrases: "discover evidence", "find new evidence", "Phase 0", "run ev-discovery", "search for benchmarks/papers", "under-evidenced", "promotion candidate needs stronger proof". This is the ONLY ev-* skill that searches the web for NEW evidence; other ev-* skills aggregate or verify known sources. Requires Firecrawl to be usable (`firecrawl --status` authenticated OR FIRECRAWL_API_KEY set); skips gracefully when neither holds.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Align, run, audit, and publish GEODE benchmark comparisons against Agent-World arXiv 2604.18292v1 across MCP-Mark, BFCL V4, and tau2-bench. Use for Agent-World or AgentWorld comparison requests, cross-suite agent benchmark design, Tau2 comparator/profile diagnosis, paired thin-runtime-versus-GEODE experiments, mean-accuracy@8 replication, or benchmark comparability and artifact reviews.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Avalia LLMs em mais de 60 benchmarks acadêmicos (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use quando benchmarking de qualidade de modelo, comparação de modelos, relatório de resultados acadêmicos ou rastreamento de progresso de treinamento. Padrão da indústria usado por EleutherAI, HuggingFace e laboratórios principais. Suporta HuggingFace, vLLM, APIs.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Use when designing evaluation suites for agentic systems, choosing agent benchmarks, measuring multi-agent performance, or critiquing single-score evaluation of agents. Based on arXiv:2605.26112v1. Trigger phrases: "evaluate my agent", "benchmark agent", "measure agent reliability", "pass^k evaluation", "agent metrics beyond success rate", "trajectory quality", "memory hygiene metric", "longitudinal agent eval", "why my agent scores well on benchmarks but fails in production".
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
This skill details how to conduct cloud security audits using Center for Internet Security benchmarks for AWS, Azure, and GCP. It covers interpreting CIS Foundations Benchmark controls, running automated assessments with tools like Prowler and ScoutSuite, remediating failed controls, and maintaining continuous compliance monitoring against CIS v5 for AWS, v4 for Azure, and v4 for GCP.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
bu skill details how to conduct cloud security audits using Center for Internet Security benchmarks for AWS, Azure, and GCP. It covers interpreting CIS Foundations Benchmark controls, running automated assessments with tools like Prowler and ScoutSuite, remediating failed controls, and maintaining continuous compliance monitoring against CIS v5 for AWS, v4 for Azure, and v4 for GCP.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Autonomous model validation and benchmarking. Investigates any ML model (LLM, image gen, TTS, time series, etc.), runs it on GPU cloud, evaluates quality and performance, and generates HTML reports. Use when user asks to verify, benchmark, evaluate, or test a model. Triggers on "verify model", "benchmark", "evaluate model", "test model", "run benchmark", "model evaluation", "モデルを検証", "ベンチマーク", "モデルを試して".
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
This skill details how to conduct cloud security audits using Center for Internet Security benchmarks for AWS, Azure, and GCP. It covers interpreting CIS Foundations Benchmark controls, running automated assessments with tools like Prowler and ScoutSuite, remediating failed controls, and maintaining continuous compliance monitoring against CIS v5 for AWS, v4 for Azure, and v4 for GCP.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
Cross-model benchmark for gstack skills. Runs the same prompt through Claude, GPT (via Codex CLI), and Gemini side-by-side — compares latency, tokens, cost, and optionally quality via LLM judge. Answers "which model is actually best for this skill?" with data instead of vibes. Separate from /benchmark, which measures web page performance. Use when: "benchmark models", "compare models", "which model is best for X", "cross-model comparison", "model shootout". (gstack) Voice triggers (speech-to-text aliases): "compare models", "model shootout", "which model is best".
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
When the user requests information about academic benchmarks, datasets, or research papers, particularly in machine learning, deep learning, or logical reasoning domains. This skill enables systematic research of academic benchmarks by searching web sources, downloading and analyzing arXiv papers, extracting key metadata (number of tasks, training availability, difficulty levels), and compiling comparative summaries. It triggers on requests involving dataset comparisons, benchmark analysis, or academic paper research for table creation.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
This skill details how to conduct cloud security audits using Center for Internet Security benchmarks for AWS, Azure, and GCP. It covers interpreting CIS Foundations Benchmark controls, running automated assessments with tools like Prowler and ScoutSuite, remediating failed controls, and maintaining continuous compliance monitoring against CIS v5 for AWS, v4 for Azure, and v4 for GCP.
한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.
테스트를 먼저 작성하는 Go 개발을 돕습니다. 여러 사례 검증, HTTP 응답 확인, 성능 측정, 입력 자동 생성 검사와 커버리지 확인을 다룹니다.
현재 성능 측정, 가설별 변경안 비교, 정확성 검증으로 최적화를 진행합니다. 시간과 비용 한도 안에서 가장 빠르고 안전한 안을 고르고 재실행 절차를 남깁니다.
점수가 매겨진 경쟁사 자료를 시장 지도, 비교표, 차별화 여지와 우선 실행안을 담은 보고서로 정리합니다. 경영진 보고와 팀의 경쟁 전략 결정에 활용합니다.
Measure and optimize claude-flow v3 startup, search, attention processing, and memory use. Compare benchmarks against performance targets, track changes over time, and identify regressions to guide further optimization.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Measure throughput, response times, and resource use for systems and groups of agents. Use baseline comparisons, load tests, and performance criteria to identify slowdowns, assess behavior under pressure, and summarize results with recommendations.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Investigate failed GAIA benchmark questions by inspecting execution records and classifying causes such as missing tools, reasoning errors, or answer extraction bugs. Follow targeted fixes, verify them through another run, and save findings for future debugging.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Measure and optimize claude-flow v3 startup, search, attention processing, and memory use. Compare benchmarks against performance targets, track changes over time, and identify regressions to guide further optimization.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Benchmark how CodeGraph affects an agent investigating a real codebase. Choose a version, language, repository, and execution mode, then compare cost, tool calls, file reads, and answer correctness with and without CodeGraph.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Add a CUDA/C++ GPU operation to sgl-kernel using AOT, meaning compilation before use. Follow the implementation through PyTorch registration, Python API exposure, SGLang integration, correctness tests, and benchmarks that account for cache reuse.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.
Review SGLang Cookbook pull requests for configuration errors, documentation mismatches, benchmark issues, and broken links. Get findings for each file and an overall recommendation on whether to approve, request changes, or block the contribution.
한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.