본문으로 건너뛰기
cccskills
무료GitHub에 공개

caveman-manage

Inspect Caveman Cloud's eval-gated experiment lifecycle and block unsafe execution. Use when the user asks to start, approve, cancel, promote, or roll back a Caveman experiment, or asks what action an experiment's evidence supports. Read evidence first; do not execute lifecycle mutations until server-authoritative transition and evidence gates ship.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila
설치 방법 보기

포함된 파일(1)

  • SKILL.md4.0 KB

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

Manage eval-gated experiments

Treat every lifecycle change as a production control action. Read current state and results, then report one supported recommendation or block. Current agent MCP is intentionally read-only: control-api does not yet enforce a complete lifecycle transition table and evidence gate atomically.

Non-negotiable gates

  1. A request to review, inspect, explain, or recommend authorizes reads only.
  2. Never approve an experiment whose results are pending, whose required guardrails are absent, or whose evidence reports a breach.
  3. Never convert experiment lift into verified_savings. Only active real traffic plus provider-causal, provider-complete ledger evidence can do that.
  4. Never supply an organization id. Project and tenant scope come from the logged-in Caveman identity and server RBAC.
  5. Never execute a lifecycle mutation, even after user approval. Exact <action>:<experiment_id> strings are agent-generatable and are not proof of human intent.
  6. Unknown states and server errors fail closed. Report exact cave_snake_code.

Step 1 — Load project and experiment

Prefer MCP:

caveman_context {}
caveman_experiment_get {"action":"get","experiment_id":"<id>"}
caveman_experiment_get {"action":"results","experiment_id":"<id>"}

Use {"action":"list"} when the user has not named an id.

CLI fallback:

caveman cloud experiments list
caveman cloud experiments show <id>
caveman cloud experiments results <id>

Stop if login, project, experiment, or results are unavailable.

Step 2 — Evaluate evidence

Report:

  • current lifecycle state and safety class;
  • control and candidate sample sizes;
  • quality or eval result;
  • latency, error, cost, retry, drop, and escalation guardrails when present;
  • evidence cost;
  • rollback or hold reason;
  • whether result is pending, failed, promotable, or active.

Absence is not a pass. If a required field is absent, state evidence incomplete and do not propose approval.

Step 3 — Propose one action

Allowed actions:

  • start — only from a startable draft or queued state with configured graders;
  • approve — only with complete passing evidence and a safety class the current role may approve;
  • cancel — stop a non-active experiment the user no longer wants;
  • rollback — revert an active or harmful change through the server's linked policy path. Current deployments may reject this honestly with cave_not_implemented; never describe that response as a rollback.

Show recommendation and id:

Proposed action: approve experiment 7f...
Reason: candidate passed quality and every configured guardrail.
Execution: blocked until server-authoritative lifecycle and evidence gates ship.

Do not treat earlier generic statements such as "manage it" or "do what is best" as mutation approval.

Step 4 — Block unsafe execution

Do not emit or run an executable lifecycle command. Explain that current server does not yet enforce every evidence/state transition atomically. CLI and MCP agent surfaces therefore expose experiment reads only.

Step 5 — Re-read after external operator action

If operator says they executed command, read detail and results again. Report server-observed post-state, audit or result response, and any policy-delivery status returned. Never infer success from operator intent alone.

Use this close:

Action: <action> <experiment-id>
Before: <state>
Server response: <status and cave_snake_code if any>
After: <re-read state>
Basis: experiment evidence only. Verified savings unchanged unless the signed
ledger independently records active, provider-causal real-traffic savings.

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

같은 저장소의 스킬

agent-platform-alert-configuration

무료

Configures best-practice alerting policies for AI agents using OpenTelemetry (OTel) metrics, generating output as Terraform (.tf) configuration files. Use when analyzing, writing, or deploying alerting policies to monitor agent latency, error rates, token usage, and quality metrics. Don't use for standard infrastructure monitoring unrelated to AI agents, or when the agent is not instrumented with OpenTelemetry (for Reliability, Cost, Safety, Security alerts). NOTE: Reliability, Cost, Safety, and Security alerts use generic OTel metrics and work across runtimes (such as Cloud Run, Vertex AI). Quality alerts rely on Vertex AI Online Monitors and are strictly bound to Vertex AI deployments.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-deploy

무료

Deploy open models or custom weights from Model Garden to Agent Platform endpoints, check the status of an in-progress deployment operation, or clean up resources by undeploying models and deleting endpoints. Use when asked to actively deploy a model, list the Model Garden CATALOG of available models, check if a specific model is deployable (`gcloud ai model-garden models list-deployment-config`), query deployment cost, troubleshoot deployment errors (like quota limits), or undeploy/clean up endpoints. Also use when copying and deploying a 1P Tuned Model. Don't use for pure listing/discovery questions of the form "is X deployed?", "list my endpoints", or "which regions have models running?" — for those use `agent-platform-endpoint-management`. Don't use for public Vertex AI deployments (use the `vertex-deploy` skill) or for running model evaluations (use the `agent-platform-eval-flywheel` skill).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-endpoint-management

무료

Manages Agent Platform serving endpoints. Use when you need to create, list, describe, update, or delete serving endpoints for model deployment on Agent Platform. Also use when troubleshooting endpoint permission, quota, or resource busy errors. Don't use for deploying models to endpoints or for running model evaluations.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-eval-flywheel

무료

Measures and improves the quality of AI models and agents on Google Cloud using the Eval Quality Flywheel methodology. Use when evaluating an agent or model, building an eval dataset, picking or writing evaluation metrics, analyzing failures, comparing results before and after a fix, or when guidance is needed on Agent Platform eval methodology — including dataset schema, LLM-as-judge scoring, and common failure causes. For fine-tuning, use agent-platform-tuning. For general production deployment, use agent-platform-deploy.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-inference

무료

Connects to and performs inference with Google Cloud Agent Platform GenAI models, including First-Party Gemini models and Third-Party OpenMaaS models (Llama, DeepSeek, Qwen, etc.). Use when asked to perform inference, ask a model a question, run a test prompt, execute chat completions, or generate code for calling Gemini or OpenMaaS models, authenticate with GenAI SDK, OpenAI SDK, or legacy Agent Platform SDK, configure base URLs and global/regional endpoints, or troubleshoot 429 Resource Exhausted (DSQ), 400 User Validation, or 404 Not Found errors. Don't use for deploying models to endpoints or for running model evaluations.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

agent-platform-migrate-from-ai-studio

무료

Guides agents and users through migrating from Gemini API in Google AI Studio to Gemini Enterprise Agent Platform (formerly Vertex AI). Use this skill when moving applications to Google Cloud, to leverage Cloud credits, or to unify inferencing with other Cloud infrastructure (IAM, billing, telemetry).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

vaila-multimodaltoolbox/vaila212026년 10월 20일 업데이트

vaila-multimodaltoolbox의 스킬 모두 보기

이 스킬의 문제 신고