News Reaction Failure Analyzer
Overview
Implements step 2 of Jason Shapiro's COT contrarian process: once a market
is flagged as crowded (cot-contrarian-detector, step 1), check whether it
FAILED to react to news that should have rewarded the crowd. A crowded-long
market that doesn't rally on genuinely bullish news, or a crowded-short
market that doesn't sell off on genuinely bearish news, is the core
behavioral tell that the crowd has run out of buying/selling power — this
is the confirmation step that turns "crowded" into a contrarian setup
candidate (steps 3-5, still manual: price-action confirmation, entry, exit).
Why this isn't a naive failure-ratio check: an earlier design flagged
"news failure" whenever fewer than half the relevant events "responded" —
but under pure noise, roughly 69% of individual events fail to respond by
chance, so that rule would CONFIRM on random noise 48-83% of the time
depending on sample size. This skill instead requires the market to have
moved significantly against the crowd's favorable news (a drift-
significance test with a Monte-Carlo-verified null false-positive bound),
never merely "didn't respond enough." See
references/news-failure-patterns.md for the full statistical rationale.
When to Use This Skill
English:
- "Did the market shrug off [event] even though [asset] is crowded long/short?"
- "Run a news-failure check on [symbol]"
- "Is [symbol] confirmed for a Shapiro-style contrarian setup?"
- After
cot-contrarian-detector flags a market CROWDED_LONG / CROWDED_SHORT
and the user wants to move to step 2
Japanese:
- 「この市場は好材料に反応しなかった？」
- 「COTで偏っているこの銘柄のニュース失敗を確認して」
Do NOT use when:
- The market isn't crowded (NEUTRAL classification) — this skill refuses
fail-closed without an explicit
--direction override
- No curated events JSON exists yet — WebSearch must run first (Phase 2
below); never fabricate events or URLs to get a verdict
Prerequisites
- FMP API Key: Required. Set
FMP_API_KEY or pass --api-key. Used
for price data only (stable/historical-price-eod/light) — coverage
varies by symbol; see references/price-source-map.md.
- Python 3.9+ with
requests installed.
- WebSearch access to curate the events JSON (Phase 2). Skill degrades
gracefully without it (states the limitation; never fabricates events).
- Optional: a
cot-contrarian-detector JSON report (--detector-json)
to auto-resolve symbol + direction, or supply --direction explicitly.
Workflow
Phase 1: Obtain symbol + direction
From a cot-contrarian-detector report (--detector-json, symbol looked
up in markets[]) or directly from the user (--symbol + --direction).
A NEUTRAL classification, a symbol missing from the report, or a report
older than --max-detector-age-days (default 10) all refuse fail-closed
with a specific reason — only an explicit --direction overrides.
Phase 2: Curate the events JSON via WebSearch
Search news in the evaluation window (--window-days, default 10) using
the 4-tier source hierarchy (issuer/primary → SEC/official stats → wire →
portal — see references/news-failure-patterns.md). Write findings into
an events JSON from references/news-failure-patterns.md's template —
event, event_time (ISO8601 with explicit UTC offset), source_url,
source_tier, expected_impact (BULLISH/BEARISH) per event.
Never fabricate events or URLs. WebSearch unavailable → state it
explicitly; proceed without an events JSON only if the user accepts an
INSUFFICIENT_EVIDENCE result (reason no_events_provided) — the CLI
never raises an exception for a missing events file, it always exits 0
with a documented reason.
Phase 3: Run the CLI
python3 skills/news-reaction-failure-analyzer/scripts/analyze_news_reaction.py \
--symbol B6 --detector-json reports/cot_crowding_2026-07-12.json \
--events-json reports/nrf_events_B6_2026-07-12.json \
--output-dir reports/
The script fetches the price series (documented fallback chain — futures
symbol first, ETF proxy if 402/restricted or rows == 0; see
references/price-source-map.md), computes effective dates / returns /
z-scores per event, clusters events whose 3-trading-day windows overlap
(independence guard), and synthesizes the verdict.
Phase 4: Present verdict + handoff
Present the verdict, aggregate stats (drift_stat, responded_ratio), and
the evidence table (per-event returns/z-scores/reaction labels, with any
dropped_events reasons shown — never silently hidden). If a proxy
(run_context.proxy_used) was used, note the tracking-error caveat.
Emit a handoff block for contrarian-setup-gate (#241, not yet built):
{"news_failure": {"verdict": "CONFIRMED", "confidence": "HIGH", "report_path": "reports/nrf_B6_2026-07-12.json"}}
Output
- JSON:
reports/nrf_<symbol>_<as-of-date>.json — schema_version,
symbol, direction, expected_direction, actual_reaction
(FAILED_TO_RALLY/FAILED_TO_SELL_OFF/RALLIED/SOLD_OFF/
MIXED_REACTION/NO_DATA), verdict, confidence,
relevant_events_used, aggregate (mean_z3/drift_stat/responded_ratio),
evidence[], dropped_events[], run_context.
- Markdown:
reports/nrf_<symbol>_<as-of-date>.md — human-readable
verdict, aggregate stats, evidence table, dropped-events table, proxy
caveat (if used), and methodology footnote.
Guardrails
- CONFIRMED is not a trade signal. It confirms step 2 of 5 — price-
action confirmation (step 3), entry (step 4), and exit (step 5) are still
manual and still required before any position.
- INSUFFICIENT_EVIDENCE never advances the pipeline. Fewer than
--min-events (default 3) usable relevant event clusters, a missing
detector report, or a detector vintage (data_date) that's missing,
unparsable, dated after --as-of, or older than
--max-detector-age-days (stale), a NEUTRAL classification without an
explicit override, or no working price source all produce this verdict
— never a crash, never a forced call on inadequate data.
- COT publication lag. COT data is 3-9 days old by the time it's read
(see
cot-contrarian-detector); news-failure evidence should be read in
that context, not as same-day confirmation.
- Counter-direction events are context only — shown in the evidence
table but excluded from the verdict (only events whose
expected_impact
matches the crowd's expected_direction count).
- Proxy-based prices are noted, not hidden. When an ETF proxy was used
(
run_context.proxy_used), the report says so — tracking error, expense
drag, and roll-timing differences make the reaction-direction read
approximate, not exact.
- Residual statistical risk under extreme correlation. The verdict's
null false-CONFIRMED rate is hard-verified under i.i.d. noise (<8%) and
under a realistic residual-correlation stress (AR(1) ρ=0.1, <10%). Under
an intentionally extreme correlation stress (lag-1 ρ=0.3 across
non-clustered event windows — roughly 10x liquid-futures empirical
autocorrelation), the measured null rate rises to ~11-13%. This is a
documented v1 limitation, not a silent gap — see
references/news-failure-patterns.md for the full numbers. Users who
want the stricter <10% margin even under that stress can pass
--drift-z 1.75 (at the cost of missing some genuine news-failure
signals, not just noise).
- Not investment advice. Research/educational purposes only.
Resources
references/news-failure-patterns.md
Full methodology: what qualifies as a relevant event, the 4-tier source
hierarchy, worked examples, the events-JSON curation guide + template, and
the verdict-threshold rationale (why drift-significance, not a naive
ratio; the Monte-Carlo-verified null bounds).
references/price-source-map.md
Per-market price-source fallback chain, verified/402/0-rows status (live-
probed at implementation time), ETF-proxy caveats, and markets with no
viable source (documented no_price_source cases: VX, ZQ, HO, all agri on
this key).
When to Load References
- First use / explaining the methodology: Load
references/news-failure-patterns.md
- Explaining why a market has no verdict (no_price_source): Load
references/price-source-map.md
- Regular execution: References not needed for the CLI itself — needed
for Phase 2 (events curation) and for explaining results to the user