「eval」の検索結果

Apply critical rationalist epistemology (Popper, Deutsch) to evaluate reasoning, identify errors, and refine understanding. Use when the user explicitly requests help with reasoning - phrases like "help me think this through", "does this make sense", "any flaws in this", "what am I missing", "critique this", "is this reasoning sound", "stress test this idea", "devil's advocate", or any request to evaluate arguments, identify logical problems, or improve thinking. Also use when errors in reasoning are significant enough to materially affect the user's goals, even if not explicitly requested.

日本語の概要は準備中です。原文の説明を表示しています。

Run a 3-leg evaluation against a newly released LLM. Leg 1 is trap questions with deterministic answers, leg 2 is an agentic orchestration task with a planted failure, leg 3 is a brownfield feature build in a real repo. Use when the user wants to vibe-check, test, evaluate, or compare a new frontier model, or mentions model-bench, model vibe check, or launch-day protocol.

日本語の概要は準備中です。原文の説明を表示しています。

Use this skill when implementing form submissions, data mutations, or any 'use server' function in Next.js 16. Trigger words: フォーム送信, mutation, 更新処理, 保存, Server Action, use server, useActionState, revalidatePath, データ更新.

Evaluate design from a UX perspective, assessing visual hierarchy, information architecture, emotional resonance, cognitive load, and overall quality with quantitative scoring, persona-based testing, and actionable feedback. Use when the user asks to review, critique, evaluate, or give feedback on a design or component.

日本語の概要は準備中です。原文の説明を表示しています。

Use when turning something that happened into something that makes the next run better - "write a retro", "record this paper cut", "this bug escaped verification", "make this an eval", "why do we keep hitting this", "did that change actually help", "scorecard", "what did we learn from that run", "lessons keep getting re-learned". Covers the six-step loop (capture, classify, convert, change, measure, retro), the repo artifact each step produces, and how escaped bugs become eval cases. Do NOT use to design or judge a single workflow experiment (see `workflow-experimentation`, which this loop feeds), to decide where to run work (see `dogfooding`), or to debug one failed execution in the moment.

日本語の概要は準備中です。原文の説明を表示しています。

Use when: 'intake this repo/skill', 'evaluate external skill', 'should we adopt X', a candidate repo URL plus an adoption question, or any external skill/workflow/tool surfaces as a possible fit for Agistra. Structured front-door evaluation of external skills, workflows, and similar capability artifacts before any install, promotion, or adaptation decision. Ends in exactly one classification: Install candidate, Adapt concept, or Reject. Never install first — intake first.

日本語の概要は準備中です。原文の説明を表示しています。

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

日本語の概要は準備中です。原文の説明を表示しています。

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

日本語の概要は準備中です。原文の説明を表示しています。

Memory is the cornerstone of intelligent agents. Without it, every interaction starts from zero. This skill covers the architecture of agent memory: short-term (context window), long-term (vector stores), and the cognitive architectures that organize them. Key insight: Memory isn't just storage - it's retrieval. A million stored facts mean nothing if you can't find the right one. Chunking, embedding, and retrieval strategies determine whether your agent remembers or forgets. The field is fragm

日本語の概要は準備中です。原文の説明を表示しています。

Azure Resource Manager SDK for Arize AI Observability and Evaluation (.NET).

日本語の概要は準備中です。原文の説明を表示しています。

You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.

日本語の概要は準備中です。原文の説明を表示しています。

Vendor (copy-import) a single external Claude Code skill into your own skills repo correctly — with license compliance and full provenance/attribution. Deterministic, idempotent primitive: given an upstream ref (owner/repo/path@sha or a GitHub URL) it fetches the skill, detects and gates on the license, copies the folder, injects a provenance frontmatter block (license + metadata: author/source/upstream_commit/vendored_at/provenance), ships the upstream LICENSE, adds a ## Credits section, and records a CREDITS.md ledger entry. Use when the user asks to "vendor a skill", "import a skill", "clone a skill", "borrow someone's skill", "add an external skill with attribution/credit", or スキルを取り込む / vendoring / 他人のスキルを自分のリポに取り込む. This is the import primitive that a skill-scout/curator orchestrator calls; it does NOT discover or evaluate — it correctly imports ONE skill you already chose.

Investigate model specifications, requirements, and evaluation strategy. Use when researching a model before benchmarking: reading HuggingFace model cards, estimating VRAM requirements, selecting GPU providers, and determining evaluation approach. Triggers on "model research", "investigate model", "model info", "VRAM estimate", "which provider", "model card".

日本語の概要は準備中です。原文の説明を表示しています。

Focused Signals scout for PostHog projects using AI observability. Rotates through a set of lenses — cost, latency, errors, volume, eval performance, eval/enrichment config, clusters, and tool usage — watching each for trends and spikes sliced by the dimensions it discovers over time. Leans on the sandbox's bundled `exploring-llm-*` deep-dive skills for the actual queries. Emits findings only when they clear the confidence bar; otherwise writes durable memory and closes out empty. Self-contained peer in the signals-scout-* fleet — no dependencies on other scouts.

日本語の概要は準備中です。原文の説明を表示しています。

Set up an LLM-judge evaluation that extracts canonical use cases for a PostHog feature at scale and streams the results to a Slack channel as a live feed. Use when someone wants to understand how users are actually using a specific AI/LLM-powered feature in production — what they're investigating, what questions they're trying to answer, and what patterns surface — without manually reading hundreds of traces. Assumes the feature emits `$ai_generation` and `$ai_evaluation` events with `$session_id` linkage to the trigger user's recording (the standard setup post the session-summary linkage PRs).

日本語の概要は準備中です。原文の説明を表示しています。

Diagnostic test / biomarker accuracy — sensitivity, specificity, PPV, NPV, likelihood ratios, accuracy from a 2x2 table; ROC curve, AUC, and the optimal cutoff (Youden) for a continuous biomarker; and post-test probability via Bayes. Use when you have test results vs a gold standard (binary 2x2, or a continuous score + true labels) and need to judge how good the test is, pick a threshold, or compute the probability of disease given a result. Emphasizes the prevalence-dependence of PPV/NPV.

日本語の概要は準備中です。原文の説明を表示しています。

Open-source embedding database for AI applications. Store embeddings and metadata, perform vector and full-text search, filter by metadata. Simple 4-function API. Scales from notebooks to production clusters. Use for semantic search, RAG applications, or document retrieval. Best for local development and open-source projects.

日本語の概要は準備中です。原文の説明を表示しています。

OpenAI's model connecting vision and language. Enables zero-shot image classification, image-text matching, and cross-modal retrieval. Trained on 400M image-text pairs. Use for image search, content moderation, or vision-language tasks without fine-tuning. Best for general-purpose image understanding.

日本語の概要は準備中です。原文の説明を表示しています。

Data framework for building LLM applications with RAG. Specializes in document ingestion (300+ connectors), indexing, and querying. Features vector indices, query engines, agents, and multi-modal support. Use for document Q&A, chatbots, knowledge retrieval, or building RAG pipelines. Best for data-centric LLM applications.

日本語の概要は準備中です。原文の説明を表示しています。

Builds and validates predictive time-to-event models on clinical and omics data with penalized Cox, random survival forests, gradient-boosted and deep survival models, and prediction-grade evaluation (Uno's C, time-dependent AUC, integrated Brier, calibration, competing risks). Use when building an individualized risk predictor or prognostic omics signature, choosing a survival model, or evaluating one beyond the C-index. For Kaplan-Meier, log-rank, and classical Cox hazard-ratio inference in a trial see clinical-biostatistics/survival-analysis.

日本語の概要は準備中です。原文の説明を表示しています。

Review Stellar Community Fund (SCF) Build Award applications. Use when evaluating SCF submissions, covering project fit assessment, technical architecture evaluation, team readiness, traction validation, budget analysis, and funding recommendations.

日本語の概要は準備中です。原文の説明を表示しています。

Theatre.js アニメーションツールキットリファレンス。 @theatre/core / @theatre/studio / @theatre/r3f / @theatre/dataverse。 getProject, Sheet, Sequence, Object, val, onChange, prop-types（number / rgba / compound / image）。 studio initialize / extend / transaction / scrub / createPane。 SheetProvider, editable, useCurrentSheet, useVal, usePrism, useAtom。 keyframe アニメーション、audio sync、state JSON、React Three Fiber 統合。

Windows アプリ開発 (Windows App SDK / WinRT) のファイル・設定・データ保存リファレンス。 Windows.Storage の StorageFile, StorageFolder, IStorageItem, FileIO, PathIO, CachedFileManager, DataReader, DataWriter。ApplicationData, ApplicationDataContainer, ApplicationDataCompositeValue によるローカル・ローミング設定。 FileOpenPicker, FileSavePicker, FolderPicker, StorageApplicationPermissions, KnownFolders によるファイルアクセスと権限。JsonObject, JsonArray, XmlDocument, SQLite, EF Core によるシリアライズと永続化。

Inngest イベント駆動 Durable Execution プラットフォームリファレンス。 サーバーレス環境での長時間ワークフロー、バックグラウンドジョブ、スケジュールタスク実装。 createFunction, step.run, step.sleep, step.sleepUntil, step.waitForEvent, step.waitForSignal, step.invoke, step.sendEvent, step.fetch, step.ai.infer, step.ai.wrap, inngest.send, serve, connect。 concurrency, throttle, debounce, rate-limit, batching, priority, idempotency, singleton フロー制御。 Realtime（channels, topics, publish, subscribe, useRealtime, subscription tokens）による進捗配信。 Agent Evals, scoring, group.experiment, MCP dev server などエージェント運用機能。 Next.js / Express / Hono / Cloudflare Workers / Vercel / Netlify / DigitalOcean / Render デプロイ対応。