跳到正文
cccskills
免费在 GitHub 公开

caio-review

Review an AI plan through six questions covering evaluation examples, error targets, fallback behavior, regulatory risk, model choices, costs, and hiring. Produce a structured verdict with gaps to address and three concrete next steps.

中文简介正在准备中。当前显示英语简介。

原文查看原文说明

/c-level-agents:caio-review [plan] — Eval-demanding Chief AI Officer interrogation of any plan that involves AI: model selection, risk classification, cost economics, or AI hiring. Use when shipping an AI feature without an eval set, choosing between API, fine-tune, and self-hosted, or classifying a use case under the EU AI Act.

查看安装方法

适用场景

  • Reviewing an AI feature before release
  • Comparing APIs, fine-tuning, and self-hosting
  • Checking AI regulatory risk before an EU launch
  • Assessing AI costs and the next team hire

包含的文件(1)

  • SKILL.md6.0 KB

SKILL.md(原文)

安装前可以查看交给 AI 代理的指令内容。

/c-level-agents:caio-review — CAIO Forcing Questions

Command: /c-level-agents:caio-review <plan>

The eval-demanding CAIO pressure-tests any plan that involves AI. Six questions before any AI feature ships, any multi-year vendor commitment, or any AI team expansion.

When to Run

  • Before shipping any new AI-powered feature
  • Before signing a multi-year AI vendor contract (API or self-hosted infra)
  • Before EU launch of any AI feature
  • Before a major AI team hire (especially ML engineer or research scientist)
  • Before a fine-tuning project commitment
  • Before adopting AI in a regulated domain (employment, credit, healthcare, education, etc.)
  • When the founder uses the word "AI" near "competitive advantage" or "moat"

The Six CAIO Questions

1. What does this AI need to be good at, and how would you measure it?

No eval set = no ship. Before any AI feature deploys, define the eval criteria.

  • 50-100 representative inputs minimum
  • Expected outputs OR rubric for grading
  • Edge cases: ambiguous, adversarial, format-edge
  • If you can't write down what "good" looks like, you don't have a feature; you have a vibe.

2. What's the SLO on hallucination / error rate, and what's the fallback?

Every AI feature has a failure mode. Plan for it.

  • Quantified SLO: "<5% hallucination on factual queries"
  • Detection mechanism: monitoring, sampling, customer feedback loop
  • Fallback: human-in-loop review, lower-risk default response, refuse-to-answer
  • Blast radius if SLO breached: how many users affected, what is the cost?

3. What's the risk tier under EU AI Act, and is conformity assessment required?

Run ai_risk_classifier.py if any EU residents are affected OR domain is regulated.

  • PROHIBITED → cannot launch in EU; re-scope
  • HIGH → conformity assessment + EU DB registration + 10 Articles of obligations (3-12 months, $50-200K)
  • LIMITED → transparency obligations (chatbot disclosure, AI-generated content marking)
  • MINIMAL → no specific obligations; NIST AI RMF voluntary

4. API, fine-tune, or build?

Run model_buildvsbuy_calculator.py for the specific use case.

  • 80% of B2B SaaS use cases: API
  • 15%: fine-tune (when domain-specific behavior + labeled data + ML team + high volume)
  • <1%: build from scratch
  • Decision must consider economic breakeven AND practical feasibility (data, team, compliance)

5. What's the 12-month cost trajectory at expected scale?

Run ai_cost_economics.py for the workload.

  • API: variable, scales linearly
  • Self-hosted: mostly fixed, breakeven typically 1-10B tokens/month for 70B-class
  • Hidden costs of self-hosted: ops, monitoring, model updates, capacity, failover, security
  • Hidden costs of API: vendor lock-in, capability drift, rate limits, data residency
  • Prompt caching is the most underrated lever; check provider support

6. What role unblocks this — and have we hired prerequisites first?

Map AI capability to specific role. Founders confuse AI engineer / ML engineer / research scientist.

  • AI engineer: applied + full-stack + prompts + evals + deployment (most startups need this)
  • ML engineer: fine-tuning + retraining infra (only after platform engineer + labeled data)
  • Research scientist: model invention (only if model IS the product)
  • Don't hire research scientist as first AI hire — they need infrastructure to be productive

Workflow

# 1. Model selection check
python ${CLAUDE_PLUGIN_ROOT}/vendor/c-level-advisor/chief-ai-officer-advisor/scripts/model_buildvsbuy_calculator.py use_case.json

# 2. Regulatory classification
python ${CLAUDE_PLUGIN_ROOT}/vendor/c-level-advisor/chief-ai-officer-advisor/scripts/ai_risk_classifier.py use_case.json

# 3. Cost projection
python ${CLAUDE_PLUGIN_ROOT}/vendor/c-level-advisor/chief-ai-officer-advisor/scripts/ai_cost_economics.py workload.json

Output Format

# CAIO Review: <plan>
**Date:** YYYY-MM-DD

## The Decision Being Made
[one sentence — which CAIO decision: model selection | risk classification | economics | next hire]

## Eval Discipline
- Eval set committed: yes/no
- SLO defined: <metric> < <threshold>
- Fallback behavior: <one line>

## Model Selection (if applicable)
- Recommended: API / FINE_TUNE / BUILD
- 3-year TCO: $X (chosen path) vs $Y (alternatives)
- Breakeven: <volume>

## Risk Classification (if applicable)
- EU AI Act tier: PROHIBITED / HIGH / LIMITED / MINIMAL
- Conformity assessment required: yes/no
- US state triggers: [list]
- Required controls open: N

## Cost Economics (if applicable)
- Monthly cost at current volume: $X
- Breakeven for self-hosted migration: <volume>
- Migration cost if applicable: $X (3-6 months)

## Org (if applicable)
- Next hire: <role>
- Why this, not the alternative: <one line>
- Prerequisite hires in place: yes/no

## Verdict
🟢 SHIP | 🟡 SHARPEN | 🔴 BLOCK

## Next Steps
[3 concrete actions]

Routing

  • /c-level-agents:cdo-review — for any training-data implications
  • /c-level-agents:gc-review — for AI vendor contracts, output liability, training-data licensing
  • /c-level-agents:ciso-review — for prompt injection / jailbreak / training-data poisoning threat model
  • /c-level-agents:cfo-review — for multi-year vendor or GPU commitment TCO
  • cs-chro-advisor agent — for AI team hires (comp, ladder, leveling)
  • /c-level-agents:decide — log the verdict
  • /c-level-agents:freeze 60 — on multi-year AI commitments

Related


Version: 1.0.0

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问,或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护,适用 Google 的隐私权政策和服务条款。

同一仓库的技能

Audit web applications for accessibility issues against WCAG 2.2 Level A and AA, the web accessibility standards. Get framework-specific code fixes, contrast checks, verification steps, and reports, with options for automated checks in development pipelines.

中文简介正在准备中。当前显示英语简介。

  • Audit a web application before release
  • Fix contrast and keyboard navigation issues
  • Prepare accessibility reports for stakeholders
alirezarezvani/claude-skills2.9万2026年10月21日 更新

Plan A/B tests that compare page variations, from headlines to pricing layouts. Define a hypothesis, choose success metrics, calculate sample size and duration, and prepare launch checks and results reports to support decisions.

中文简介正在准备中。当前显示英语简介。

  • Comparing landing page headlines
  • Testing signup button changes
  • Planning pricing page experiments
alirezarezvani/claude-skills2.9万2026年10月21日 更新

Create paid ad headlines and body copy tailored to each platform's character limits. Use existing performance data to develop variations, explore new messaging angles, and check copy for length, quality, and potential rejection risks.

中文简介正在准备中。当前显示英语简介。

  • Creating Google RSA headlines
  • Writing Meta and LinkedIn ad copy
  • Developing variations of successful ads
alirezarezvani/claude-skills2.9万2026年10月21日 更新

Review code changes from three critical perspectives: production failures, maintainability, and security. Get a consolidated report that ranks findings by severity and provides merge guidance, helping examine assumptions that an ordinary self-review may overlook.

中文简介正在准备中。当前显示英语简介。

  • Reviewing a pull request before merging
  • Getting a second opinion on AI-written code
  • Checking authentication or data access code
alirezarezvani/claude-skills2.9万2026年10月21日 更新

aeo

免费

Audit and improve content intended for citation in AI answers. Review evidence, author information, and structure, generate revised content and structured metadata, and keep a local record of observed citations across AI services.

中文简介正在准备中。当前显示英语简介。

  • Audit existing articles for evidence and credibility gaps
  • Revise article structure and references for AI answers
  • Record citations observed across AI services
alirezarezvani/claude-skills2.9万2026年10月21日 更新

Create signed records for consequential agent actions such as deployments, deletions, and payments. Decide which actions need evidence, record the agent, action, target, and policy, then verify the resulting certificate offline to detect later changes.

中文简介正在准备中。当前显示英语简介。

  • Creating signed records for production deployments
  • Preserving evidence of deletions and payments
  • Recording access grants for later review
alirezarezvani/claude-skills2.9万2026年10月21日 更新

查看 alirezarezvani 的全部技能

报告此技能的问题