DAAF Contribution Community

Complex survey analysis: strata/PSU/weights, variance estimation (Taylor, BRR, jackknife, bootstrap), survey GLM, domain analysis, calibration, survey data I/O (SAS, SPSS, Stata formats). Polars-native. Use for any complex-sample survey: NHANES, CPS, ACS PUMS, BRFSS, DHS, ECLS-K, MEPS-HC. CRITICAL: statsmodels WLS and pyfixest clustered SEs are NOT substitutes for proper survey-weighted analysis — they ignore stratification and unequal probability sampling. Non-survey regression: statsmodels/pyfixest. R equivalent: survey-r (use when execution language is R).

日本語の概要は準備中です。原文の説明を表示しています。

Privacy-preserving workflow for building synthetic datasets and data source skills when raw data must never enter the DAAF container. The user profiles their sensitive data locally with a disclosure-controlled script; only a summary profile report crosses the boundary; DAAF builds a synthetic dataset and skill from the report alone. Use whenever data is sensitive, proprietary, PII-bearing, HIPAA/FERPA-governed, held in a secure enclave, or the user says the data cannot leave their environment, they cannot upload it, or asks to profile it locally. Covers a four-tier disclosure ladder (T1 schema, T2 marginals, T3 relationships, T4 local high-fidelity synthesis), profile-only generation with simstudy (R) / NumPy-SciPy copulas (Python), and three-part QA (disclosure-safety, report consistency, synthetic-vs-profile validation). Not for synthetic control, the causal-inference method; for that see data-scientist. Synthetic data here is a code-development scaffold, not an analytic substitute.

日本語の概要は準備中です。原文の説明を表示しています。

R data manipulation with dplyr, tidyr, readr, purrr, stringr, forcats, lubridate. Verb pipelines, reshaping, joins, string/date ops, parquet via arrow. Includes data.table for performance. Use when execution language is R. Python equivalent: polars.

日本語の概要は準備中です。原文の説明を表示しています。

Complex survey analysis in R with survey (Lumley): svydesign, svymean/svytotal, svyglm, svyby domains, replicate weights (BRR, jackknife, bootstrap). Use when execution language is R. Python equivalent: svy. lm(weights=) is NOT survey analysis.

日本語の概要は準備中です。原文の説明を表示しています。

R machine learning with tidymodels: recipes (preprocessing), parsnip (model specs), workflows (fit pipelines), tune (hyperparameters), rsample (resampling). Engines: ranger (RF), glmnet (lasso/ridge), xgboost, kknn. Model interpretation (permutation importance, PDP/ICE, SHAP via DALEX/iml/kernelshap) and fairness assessment (fairmodels). UMAP via uwot. Use when execution language is R. Python equivalent: scikit-learn. For econometric regression use fixest or r-stats.

日本語の概要は準備中です。原文の説明を表示しています。

DAAF's protocol for retrieving web documents that an agent needs to READ (docs pages, articles, API references, issue threads). Retrieval runs through the framework CLI `bash /daaf/scripts/web_fetch.sh URL DEST_DIR`, which saves the raw response, a deterministic extract, and a provenance manifest to disk — the built-in WebFetch is blocked because it returns an AI paraphrase, not source text. Covers when to fetch at all (check local skills first), tool syntax and flags, the two-artifact + MANIFEST provenance model, the exit-code failure table, the untrusted-content rule, thin-extract diagnosis, extract size rules, and destination conventions. Use when a permitted context (search-agent, data-ingest, debugger, orchestrator) must pull a specific web page into context to read or quote it. NOT for discovery — WebSearch stays built-in for finding URLs. NOT for pipeline data acquisition — Stage 5 / onboarding scripts fetching datasets into parquet are a separate, unchanged activity for any pipeline agent.

日本語の概要は準備中です。原文の説明を表示しています。

Mirror-first discovery for Urban Institute Education Data: maps a research question to the mirror dataset FILES that answer it — canonical path, entity grain, year coverage, key variables, and join keys — for CCD, IPEDS, CRDC, College Scorecard, SAIPE, EDFacts, FSA, MEPS, NHGIS, PSEO, NACUBO, NCCS, EADA, and Campus Safety. Discovery output is the mirror path (plus grain, years, variables, caveats), the currency planning and acquisition consume. A pure readable-reference skill for read-only discovery agents — no code, no live HTTP probes; all facts keyed to pinned mirror vintage 0.26.1. Load BEFORE education-data-query — routing (which file, what grain) here, retrieval (fetch code, mirror config) there.

日本語の概要は準備中です。原文の説明を表示しています。

Interpretation guidance for Urban Institute Education Data Portal datasets. The Portal is a curation layer over federal data: lowercase variable names, integer-encoded categoricals, standardized missing codes (-1 missing, -2 not applicable, -3 suppressed). Covers year definitions, grade encoding (grade=-1 is Pre-K, not missing), suppression rates, ODC-By licensing, and cross-source join identifiers. Load before analyzing any Portal data. Routes to source-specific deep-dive skills for individual datasets.

日本語の概要は準備中です。原文の説明を表示しています。

In-situ smoke testing that verifies a LIVE DAAF deployment functions end-to-end in whatever provider route it is configured for. Auto-detects the active install route (anthropic-subscription, openrouter, chatgpt-subscription shim, openai-api shim) and runs tiered probes: Tier 0 free preflight (route/env coherence, hooks, statuslines, shim health); Tier 1 one live round-trip (~cents); Tier 2 a six-probe functional battery (dispatch, coding, web, skill loading, isolation-strip, nested-dispatch deny); Tier D a zero-cost deterministic battery (bats, Pester, lint, safety-hook tests). Use after install, after a rebuild or update, after any provider/config change (new API key, model remap, shim change), for pre-release validation, or when asked "is my install working" or to "smoke test the deployment/config." Requires the DAAF_DEV=1 dev image; a contributor tool, not end-user-facing. NOT DAAFBench (model behavioral-adherence evaluation under benchmarks/) and NOT scripts/smoke_tests/ (R/Python library-package smoke tests).

日本語の概要は準備中です。原文の説明を表示しています。

CRDC — OCR civil rights collection for U.S. public schools; Portal families have topic-specific coverage from 2011 through 2022. Discipline, course access, harassment, restraint/seclusion by race/sex/disability/EL. Use for civil rights and equity analysis. Official evidence identifies 2013-14 as a universe collection; 2020-21 is COVID-impacted.

日本語の概要は準備中です。原文の説明を表示しています。

EADA — college athletics gender equity (~2,000+ institutions, 2002-2021). Participation, coaching, salaries, expenses, revenues, athletic aid by gender. Not Title IX compliance data. No sector column; join IPEDS on unitid for institution type.

日本語の概要は準備中です。原文の説明を表示しています。

EDFacts — K-12 outcomes: assessment proficiency, ACGR graduation rates, ESSA accountability at school/district level (2009-2020). Within-state trends and subgroup gaps. Complements CCD with outcome data. Cannot compare across states — use NAEP.

日本語の概要は準備中です。原文の説明を表示しています。

FSA — Title IV aid at institution level (~5,500 institutions, 1999-2021). Pell Grants, Direct/PLUS loans, campus-based aid, financial responsibility scores, 90/10 metrics. Use for aid distribution, loan volume, or for-profit analysis. By unitid.

日本語の概要は準備中です。原文の説明を表示しています。

IPEDS — primary federal postsecondary data (~6,500 institutions, 1979-present in the mirror; the IPEDS program dates from 1980): enrollment, completions, graduation rates, finance, aid, admissions, HR. For college/university analysis. Grad rates = first-time full-time; finance needs GASB/FASB care.

日本語の概要は準備中です。原文の説明を表示しています。

CCD — federal universe of all U.S. public K-12 schools (~100K) and districts (~18K). Enrollment, staffing, finance, directory data (1986-present). Use for public school analysis by grade/race/sex. Public only; excludes private and postsecondary.

日本語の概要は準備中です。原文の説明を表示しています。

Data science methodology and method-selection routing for quantitative research. Covers EDA, data validation, descriptive analysis, causal inference (IV, DiD, RD, synthetic control), clustering/PCA/UMAP, supervised ML, geospatial analysis, network analysis, and visualization design. Contains the canonical method-to-library routing tree, routed by execution language — Python: statsmodels (OLS/GLM/time series), pyfixest (FE/DiD), linearmodels (RE/GMM/SUR), svy (complex surveys), scikit-learn (clustering/prediction ML), geopandas (spatial), igraph (network analysis); R: r-stats (OLS/GLM/time series), fixest (FE/DiD), plm (panel/RE/IV), survey-r (complex surveys), tidymodels (ML), sf-terra (spatial), igraph-r (network analysis). Load the routed tool-specific skill before giving tool-specific advice or writing code — library skills encode environment constraints and curated caveats absent from general knowledge.

日本語の概要は準備中です。原文の説明を表示しています。

NHGIS — census geography crosswalks via Portal: links schools (ncessch) and colleges (unitid) to census tracts, block groups, CBSAs, and regions (census-geography vintages 1990-2020; mirror file-year/linkage coverage 1980-2023). Portal provides geography linkage tables ONLY; browse census demographics in the public NHGIS Data Finder, then sign in with a free IPUMS NHGIS account to submit, manage, or download extracts. API extract requests also need an API key. Use for linking education or institutional data to census geography.

日本語の概要は準備中です。原文の説明を表示しています。

NCCS — Form 990 data for private nonprofit colleges (Portal: IPEDS-matched, 1993-2016). Revenue, expenses, assets, endowment, governance beyond IPEDS. Use when IRS financial depth needed. Portal ends 2016; public institutions excluded (no Form 990).

日本語の概要は準備中です。原文の説明を表示しています。

PSEO — Census data linking graduates to employment via LEHD wage records. Earnings percentiles at 1/5/10 years post-graduation by institution, degree, and CIP. Use for graduate earnings analysis. Coverage changes by release; Census V4.13.0 (2025Q4) lists 952 institutions and partners in 32 states plus D.C.

日本語の概要は準備中です。原文の説明を表示しています。

SAIPE — annual Census poverty estimates for school districts (Portal; county/state not in Portal). Use for district poverty, Title I context, or trends. ~18-month lag. No race/ethnicity disaggregation at district level — use ACS 5-year for that.

日本語の概要は準備中です。原文の説明を表示しています。

College Scorecard — post-enrollment outcomes linking aid records to IRS/Treasury earnings. Earnings, loan repayment, debt via six Portal sub-datasets. Use when tax-record-based earnings needed. Tracks only Title IV aid recipients, not all students.

日本語の概要は準備中です。原文の説明を表示しています。

County Presidential Returns 2000-2024 (MIT MEDSL). Vote shares, party trends, turnout by county_fips (joins census/education data). Requires HARVARD_DATAVERSE_API_KEY set via environment_settings.txt. Critical: naive mode='TOTAL' filtering silently drops ~1,000 counties in post-2020 data where states report by vote mode (absentee, election-day, provisional) instead of totals — use 3-pattern reconstruction (TOTAL-present rows kept, breakdown-only counties summed across modes, empty-string mode rows reclassified). Categorical variables use uppercase strings, not Portal integer codes.

日本語の概要は準備中です。原文の説明を表示しています。

Fast high-dimensional fixed effects in R: feols/fepois/feglm/fenegbin with multi-way FE; IV estimation; DiD (TWFE, Sun-Abraham via sunab); clustered/ heteroskedasticity-robust SEs; etable/coefplot/iplot for reporting. Use when execution language is R. Python equivalent: pyfixest. For panel RE/between use plm; for GLM without FE use r-stats.

日本語の概要は準備中です。原文の説明を表示しています。

CSS — annual Clery Act crime/fire safety for Title IV institutions. Portal: hate crimes only (2005-2021); primary offenses, VAWA, arrests, fire safety need ope.ed.gov directly. Use for campus crime analysis. Identified by IPEDS unitid.

日本語の概要は準備中です。原文の説明を表示しています。