evolve
Evolve this harness with Darwin Mode — frozen model, evolving harness (real, sandboxed, safety-gated).
The English summary is being prepared. Showing the original description.
See how to installFiles included (1)
- SKILL.md2.7 KB
SKILL.md (original)
Review the instructions the agent will receive before you install.
evolve — Darwin Mode self-improvement
group-field-theory-harness ships with Darwin Mode (@metaharness/darwin, ADR-070…146): the model
is frozen; the harness evolves. Each generation mutates ONE of the 7 surface files
(planner, contextBuilder, reviewer, retry/tool/memory/score policy), sandboxes each
child, scores it, and keeps only variants that measurably improve — building an
archive of successful descendants.
Run it
npm run evolve # real substrate: runs your test command per variant (deterministic mutator — no API key, no network)
npm run evolve:dry # mock substrate: fast, fully offline, no test execution
Or directly:
npx metaharness-darwin evolve . --sandbox real --generations 3 --children 4
Safety (secure by default)
- Deterministic mutator is the default — no network, no API key, air-gapped.
- Every mutation passes the
validateGeneratedCodegate: no new imports, network, filesystem, shell, env access, or dependencies — pure refactor/tuning only. - Mutations run in a sandbox; only variants that pass your tests are archived.
- Nothing is promoted without measured improvement (guard against Goodharting).
See @metaharness/darwin for selection strategies (--selection, --crossover,
--curriculum), statistical gates (--fdr, --bench), and the real-LLM mutator (library API).
What the benchmarks taught us (measured, full SWE-bench Lite 300)
Defaults worth carrying into how you evolve and run this harness (full evidence + CIs in
@metaharness/darwin's LEARNINGS.md / bench/results/RESULTS.md):
- Closed-loop repair is the #1 lever (~2×). Feeding test/compiler failure back and retrying took resolve-rate 7.7% → 15.3% on the same cheap model. Iterate against ground truth, don't single-shot.
- Cheap-first + cost-aware routing. Track $/resolve, not just resolve-rate; a cheap model resolved 31× cheaper per fix than a frontier one. Reserve frontier for measured capability gaps.
- Tier the models (Barbarian & Scholar). Cheap sweep + frontier on only the residual = 33.3% at ~6× lower cost than running frontier everywhere.
- Put the output-format contract in a system message + example, and size prompts to the model's real context window — this alone took a weak local model from 0% to ~50% valid output.
- Only trust batch evaluation of the final artifact — in-loop counters drift 1.5–5×.
- The harness multiplies the model; it can't rescue one below the task's reasoning floor. Pick the smallest model above the floor, then let evolution do the rest.
Reviews
No reviews yet. Tell others what you thought.
Comments
No login needed. Share how it worked for you, ask a question, or note where it did not run.
No comments yet. Be the first to write one.