Use when long-lived agents, project memories, system prompts, or playbooks are repeatedly summarized, rewritten, or optimized and risk losing domain detail, provenance, or execution learnings.
中文简介正在准备中。当前显示原文说明。
mouadja02/skills102026年10月21日 更新
免费在 GitHub 公开
rag-evaluation-matrix
Use when comparing basic, enhanced, GraphRAG, or agentic RAG designs, evaluating domain-specific RAG quality, tuning retrieval components, or deciding whether agentic RAG is worth its cost.
中文简介正在准备中。当前显示原文说明。
集成工具Hugging Face
查看安装方法包含的文件（3）
- SKILL.md3.0 KB
- references/rag-metrics.md990 B
- scripts/rag_eval_matrix.py1.6 KB
SKILL.md（原文）
安装前可以查看交给 AI 代理的指令内容。
RAG Evaluation Matrix
Do not choose agentic RAG by fashion. Compare retrieval designs against domain questions, answerability, correctness, cost, latency, and failure reasons.
Use When
- The user asks whether to use basic RAG, enhanced RAG, GraphRAG, or agentic RAG.
- A RAG system works on demos but fails on domain-specific questions.
- You need to compare embedding models, rerankers, chunking, query rewriting, or tool orchestration.
- LLM-as-judge scores need alignment with human review.
Evaluation Flow
- Build a domain question set with answerable, unanswerable, ambiguous, multi-hop, and adversarial cases.
- For each candidate pipeline, log retrieved evidence, final answer, citations, latency, token usage, and cost.
- Score answer correctness and answerability separately.
- Add evidence quality checks: citation support, contradiction handling, source freshness, and missing-source diagnosis.
- Segment results by question type instead of reporting only one aggregate score.
- Inspect low-correctness failures and map them to retrieval, synthesis, tool orchestration, or corpus gaps.
- Pick the simplest design that meets quality, latency, and cost constraints.
Matrix
| Dimension | Basic RAG | Enhanced RAG | Agentic RAG |
|---|---|---|---|
| Best for | Stable FAQ and narrow corpora | Noisy corpora, query mismatch, reranking | Multi-step, ambiguous, tool-rich tasks |
| Main risk | Weak recall and unsupported answers | Pipeline complexity | Cost, latency, loops, tool misuse |
| Eval focus | Retrieval recall and citation support | Component ablations | Trajectory, action choice, stopping behavior |
| Ship gate | Correctness and answerability meet threshold | Ablation proves each module helps | Agentic gains justify extra cost |
Script
Use the helper to combine per-run JSON metrics into a decision table:
python scripts/rag_eval_matrix.py results/*.json
Expected JSON fields: pipeline, question_type, correct, answerable_correct, latency_ms, cost_usd.
Common Mistakes
| Mistake | Fix |
|---|---|
| Optimizing average score only | Break down by question type and domain |
| Judging unanswerable questions as wrong by default | Score answerability separately |
| Skipping human calibration | Sample judge disagreements and tune rubrics |
| Choosing agentic RAG without ablation | Compare against enhanced RAG at equal budget |
| Ignoring failure reasons | Classify each miss before tuning |
References
- arXiv: Is Agentic RAG worth it? - https://arxiv.org/abs/2601.07711
- arXiv: RAGalyst - https://arxiv.org/abs/2511.04502
- Hugging Face Papers: RAGalyst - https://huggingface.co/papers/2511.04502
- Hugging Face dataset: RAGalyst QAC - https://huggingface.co/datasets/hoskerelab/ragalyst-qac
评价
还没有评价。欢迎分享你的使用感受。
评论
无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。
还没有评论。来写第一条吧。
同一仓库的技能
Use this skill when the user explicitly asks to map, document, or onboard into an existing codebase. Trigger for prompts like "map this codebase", "document this architecture", "onboard me to this repo", or "create codebase docs". Do not trigger for routine feature implementation, bug fixes, or narrow code edits unless the user asks for repository-level discovery.
中文简介正在准备中。当前显示原文说明。
mouadja02/skills102026年10月21日 更新
Adds educational comments to a code file so it becomes a learning resource, asking for the file (with a list of close matches) if none is given. Use when preparing code for learners, onboarding, or teaching.
中文简介正在准备中。当前显示原文说明。
mouadja02/skills102026年10月21日 更新
Write, debug, and optimize Adobe Illustrator automation scripts using ExtendScript (JavaScript/JSX). Use when creating or modifying scripts that manipulate documents, layers, paths, text frames, colors, symbols, artboards, or any Illustrator DOM objects. Covers the complete JavaScript object model, coordinate system, measurement units, export workflows, and scripting best practices.
中文简介正在准备中。当前显示原文说明。
mouadja02/skills102026年10月21日 更新
Patterns and techniques for adding governance, safety, and trust controls to AI agent systems. Use this skill when: - Building AI agents that call external tools (APIs, databases, file systems) - Implementing policy-based access controls for agent tool usage - Adding semantic intent classification to detect dangerous prompts - Creating trust scoring systems for multi-agent workflows - Building audit trails for agent actions and decisions - Enforcing rate limits, content filters, or tool restrictions on agents - Working with any agent framework (PydanticAI, CrewAI, OpenAI Agents, LangChain, AutoGen)
中文简介正在准备中。当前显示原文说明。
mouadja02/skills102026年10月21日 更新
Check any AI agent codebase against the OWASP Agentic Security Initiative (ASI) Top 10 risks. Use this skill when: - Evaluating an agent system's security posture before production deployment - Running a compliance check against OWASP ASI 2026 standards - Mapping existing security controls to the 10 agentic risks - Generating a compliance report for security review or audit - Comparing agent framework security features against the standard - Any request like "is my agent OWASP compliant?", "check ASI compliance", or "agentic security audit"
中文简介正在准备中。当前显示原文说明。
mouadja02/skills102026年10月21日 更新