Compare actual AI session costs with estimates for using only Haiku, Sonnet, or Opus. The skill reports savings amounts and percentages, helping teams assess model selection costs while leaving differences in answer quality and retries outside the calculation.
Multi-baseline counterfactual cost analysis. Compares actual session spend to hypothetical always-haiku / always-sonnet / always-opus routing baselines. Answers "is the routing earning its keep?" Negative savings flag over-escalation; positive savings quantify the router's win.
Compare actual spending across different AI models with estimated spending if every task had used Haiku, Sonnet, or Opus. The skill reads session records stored in cost-tracking and includes input, output, cache writes, and cache reads in its calculations. It returns total costs, savings amounts, and savings percentages for each baseline model as a table or JSON.
Use it during regular cost reviews to explain savings from choosing different models for different tasks. It also helps check whether costs have worsened after changes to model selection settings. JSON output can support a CI check that fails when savings fall below a specified percentage. Comparisons with a cheaper model can also help investigate higher spending.
Comparisons require cost-tracking session records. Estimates assume the same token counts would produce the same results; they do not account for answer quality or retries after failures. Negative savings against a cheaper model do not establish that choosing a more expensive model was inappropriate.
This overview was written by AI (OpenAI Codex) from the public SKILL.md. Check the original below for the exact behavior.
AI rated this skill on four points using only the contents of its SKILL.md. Stars and popularity are not taken into account.
The total counts each point as A = 25, B = 15, C = 5. Rated by AI (OpenAI Codex) as of Oct 21, 2026. Treat it as a guide and read the original before installing.
Review the instructions the agent will receive before you install.
Multi-baseline counterfactual cost analysis. Pairs with the existing observability surface:
cost-budget-check — "have we crossed a threshold?" (reactive)cost-projection — "when will we cross a threshold?" (predictive)cost-counterfactual — "is the routing earning its keep?" (comparative) ← this onesession-* records from the cost-tracking namespace.--since window filter (default all-time).byModel[*] entries for each session.counterfactualUsd = (input × tier.input + output × tier.output + cache_write × tier.cache_write + cache_read × tier.cache_read) / 1Msavings = counterfactualUsd − actualUsd.| Sessions considered | 2 |
| Total input tokens | 100,000 |
| Actual spend | $0.162500 |
| Baseline | Hypothetical | Actual | Savings | % |
| `always-haiku` | $0.025000 | $0.162500 | -$0.137500 | -550.00% |
| `always-sonnet` | $0.300000 | $0.162500 | +$0.137500 | 45.83% |
| `always-opus` | $1.500000 | $0.162500 | +$1.337500 | 89.17% |
A negative always-haiku result means the router chose more-expensive models than haiku on tasks haiku could have handled. That's an over-escalation signal:
cost optimize (or inspect specific sessions via cost conversation) to investigatePositive savings quantify the router's win against that baseline. The most informative number is usually always-sonnet — it's the standard "safe default" baseline most teams would pick if they didn't have routing.
cost counterfactual --format json | jq '.baselines[1].savingsPct > 30' — fail builds if routing isn't saving ≥30% vs sonnet baseline (workload-shift detector).Like all counterfactual analyses, this assumes the same tokens at the same complexity would have produced the same outcome from the baseline model. That's an upper bound — the baseline might have failed and required retries, which the math doesn't capture. Treat the numbers as a quality-blind ceiling.
No reviews yet. Tell others what you thought.
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.
Compare models available through the HuggingFace router with chat-ui configuration, research missing models, and write short English descriptions of their architecture and capabilities. Add entries to both production and development configuration files and commit the changes.
Create an Architecture Decision Record, a document for recording important design choices, with the next sequential number. It generates a Markdown template, registers the record in AgentDB, and links related decisions found through search.
Build a searchable index and relationship graph from ADRs, records of architectural decisions. Import two supported document formats, refresh existing metadata without duplicate relationships, and review summaries of statuses, missing references, and inconsistencies.
Rebuild the index of ADRs, records of architectural decisions, after files or relationship lines are deleted. It clears stored records and relationships, indexes the files still present, and checks that the final record count matches.
Review code changes against accepted Architecture Decision Records, documents that capture agreed design choices. Report violations, outdated references, compliant changes, and areas without ADR coverage, then suggest code fixes or a new decision record where appropriate.
Check saved architecture decision records (ADRs) for broken references, circular replacement relationships, and inconsistent statuses. Use the results to verify imported records or stop a CI check when selected integrity rules fail.