Skip to content
cccskills
FreePublished on GitHub

rl-env-use

Build an RL environment / eval that trains or measures a skill in any domain (finance, medicine, law, math, science, a niche industry) from real cases that Browser Use agents mine from the web, then baseline and hillclimb a model on it. Use when the user knows a domain and what a model should be able to do in it, but not how to get the data or build the environment.

The English summary is being prepared. Showing the original description.

browser-use/rl-env-use
See how to install

Files included (200)

  • SKILL.md6.4 KB
  • .env.example41 B
  • .gitignore284 B
  • envs/sec-credit/audit.json18.6 KB
  • envs/sec-credit/climb-haiku/_state.json13.3 KB
  • envs/sec-credit/climb-haiku/baseline/change.md9 B
  • envs/sec-credit/climb-haiku/baseline/dropped.jsonl40.7 KB
  • envs/sec-credit/climb-haiku/baseline/prompt.txt0 B
  • envs/sec-credit/climb-haiku/baseline/results.jsonl195.2 KB
  • envs/sec-credit/climb-haiku/baseline/summary.json72 B
  • envs/sec-credit/climb-haiku/report.html120.3 KB
  • envs/sec-credit/climb-haiku/v1/change.md482 B
  • envs/sec-credit/climb-haiku/v1/dropped.jsonl34.9 KB
  • envs/sec-credit/climb-haiku/v1/prompt.txt1.2 KB
  • envs/sec-credit/climb-haiku/v1/results.jsonl164.4 KB
  • envs/sec-credit/climb-haiku/v1/summary.json545 B
  • envs/sec-credit/climb-haiku/v2/change.md305 B
  • envs/sec-credit/climb-haiku/v2/dropped.jsonl8.8 KB
  • envs/sec-credit/climb-haiku/v2/prompt.txt628 B
  • envs/sec-credit/climb-haiku/v2/results.jsonl40.3 KB
  • envs/sec-credit/climb-haiku/v2/summary.json368 B
  • envs/sec-credit/climb-haiku/v3/change.md234 B
  • envs/sec-credit/climb-haiku/v3/dropped.jsonl8.8 KB
  • envs/sec-credit/climb-haiku/v3/prompt.txt472 B
  • envs/sec-credit/climb-haiku/v3/results.jsonl40.3 KB
  • envs/sec-credit/climb-haiku/v3/summary.json297 B
  • envs/sec-credit/climb-haiku/v4/change.md489 B
  • envs/sec-credit/climb-haiku/v4/dropped.jsonl49.5 KB
  • envs/sec-credit/climb-haiku/v4/prompt.txt1.3 KB
  • envs/sec-credit/climb-haiku/v4/results.jsonl235.2 KB
  • envs/sec-credit/climb-haiku/v4/summary.json552 B
  • envs/sec-credit/climb-haiku/v5/change.md261 B
  • envs/sec-credit/climb-haiku/v5/dropped.jsonl8.7 KB
  • envs/sec-credit/climb-haiku/v5/prompt.txt1.4 KB
  • envs/sec-credit/climb-haiku/v5/results.jsonl40.5 KB
  • envs/sec-credit/climb-haiku/v5/summary.json324 B
  • envs/sec-credit/climb-haiku/v6/change.md393 B
  • envs/sec-credit/climb-haiku/v6/dropped.jsonl35.0 KB
  • envs/sec-credit/climb-haiku/v6/prompt.txt2.1 KB
  • envs/sec-credit/climb-haiku/v6/results.jsonl164.4 KB
  • envs/sec-credit/climb-haiku/v6/summary.json456 B
  • envs/sec-credit/climb-haiku/v7/change.md361 B
  • envs/sec-credit/climb-haiku/v7/dropped.jsonl34.9 KB
  • envs/sec-credit/climb-haiku/v7/prompt.txt1.9 KB
  • envs/sec-credit/climb-haiku/v7/results.jsonl164.3 KB
  • envs/sec-credit/climb-haiku/v7/summary.json424 B
  • envs/sec-credit/climb-haiku/v8/change.md405 B
  • envs/sec-credit/climb-haiku/v8/dropped.jsonl34.9 KB
  • envs/sec-credit/climb-haiku/v8/prompt.txt2.1 KB
  • envs/sec-credit/climb-haiku/v8/results.jsonl164.5 KB
  • envs/sec-credit/climb-haiku/v8/summary.json468 B
  • envs/sec-credit/climb-qwen-35b-verified/_state.json15.5 KB
  • envs/sec-credit/climb-qwen-35b-verified/baseline/change.md9 B
  • envs/sec-credit/climb-qwen-35b-verified/baseline/prompt.txt0 B
  • envs/sec-credit/climb-qwen-35b-verified/baseline/results.jsonl731.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/baseline/summary.json75 B
  • envs/sec-credit/climb-qwen-35b-verified/report.html525.9 KB
  • envs/sec-credit/climb-qwen-35b-verified/v1/change.md413 B
  • envs/sec-credit/climb-qwen-35b-verified/v1/errors.jsonl122 B
  • envs/sec-credit/climb-qwen-35b-verified/v1/prompt.txt1.5 KB
  • envs/sec-credit/climb-qwen-35b-verified/v1/results.jsonl158.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v1/summary.json479 B
  • envs/sec-credit/climb-qwen-35b-verified/v10/change.md247 B
  • envs/sec-credit/climb-qwen-35b-verified/v10/prompt.txt1.4 KB
  • envs/sec-credit/climb-qwen-35b-verified/v10/results.jsonl159.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v10/summary.json313 B
  • envs/sec-credit/climb-qwen-35b-verified/v11/change.md423 B
  • envs/sec-credit/climb-qwen-35b-verified/v11/prompt.txt1.7 KB
  • envs/sec-credit/climb-qwen-35b-verified/v11/results.jsonl159.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v11/summary.json495 B
  • envs/sec-credit/climb-qwen-35b-verified/v12/change.md321 B
  • envs/sec-credit/climb-qwen-35b-verified/v12/prompt.txt1.5 KB
  • envs/sec-credit/climb-qwen-35b-verified/v12/results.jsonl159.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v12/summary.json387 B
  • envs/sec-credit/climb-qwen-35b-verified/v13/change.md425 B
  • envs/sec-credit/climb-qwen-35b-verified/v13/prompt.txt1.5 KB
  • envs/sec-credit/climb-qwen-35b-verified/v13/results.jsonl159.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v13/summary.json497 B
  • envs/sec-credit/climb-qwen-35b-verified/v14/change.md330 B
  • envs/sec-credit/climb-qwen-35b-verified/v14/errors.jsonl122 B
  • envs/sec-credit/climb-qwen-35b-verified/v14/prompt.txt1.4 KB
  • envs/sec-credit/climb-qwen-35b-verified/v14/results.jsonl158.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v14/summary.json396 B
  • envs/sec-credit/climb-qwen-35b-verified/v15/change.md274 B
  • envs/sec-credit/climb-qwen-35b-verified/v15/errors.jsonl122 B
  • envs/sec-credit/climb-qwen-35b-verified/v15/prompt.txt1.6 KB
  • envs/sec-credit/climb-qwen-35b-verified/v15/results.jsonl643.9 KB
  • envs/sec-credit/climb-qwen-35b-verified/v15/summary.json340 B
  • envs/sec-credit/climb-qwen-35b-verified/v2/change.md357 B
  • envs/sec-credit/climb-qwen-35b-verified/v2/prompt.txt1.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v2/results.jsonl159.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v2/summary.json423 B
  • envs/sec-credit/climb-qwen-35b-verified/v3/change.md600 B
  • envs/sec-credit/climb-qwen-35b-verified/v3/prompt.txt1.1 KB
  • envs/sec-credit/climb-qwen-35b-verified/v3/results.jsonl889.9 KB
  • envs/sec-credit/climb-qwen-35b-verified/v3/summary.json666 B
  • envs/sec-credit/climb-qwen-35b-verified/v4/change.md360 B
  • envs/sec-credit/climb-qwen-35b-verified/v4/prompt.txt2.0 KB
  • envs/sec-credit/climb-qwen-35b-verified/v4/results.jsonl159.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v4/summary.json432 B
  • envs/sec-credit/climb-qwen-35b-verified/v5/change.md283 B
  • envs/sec-credit/climb-qwen-35b-verified/v5/errors.jsonl122 B
  • envs/sec-credit/climb-qwen-35b-verified/v5/prompt.txt1.7 KB
  • envs/sec-credit/climb-qwen-35b-verified/v5/results.jsonl554.4 KB
  • envs/sec-credit/climb-qwen-35b-verified/v5/summary.json349 B
  • envs/sec-credit/climb-qwen-35b-verified/v6/change.md358 B
  • envs/sec-credit/climb-qwen-35b-verified/v6/prompt.txt1.7 KB
  • envs/sec-credit/climb-qwen-35b-verified/v6/results.jsonl159.8 KB
  • envs/sec-credit/climb-qwen-35b-verified/v6/summary.json424 B
  • envs/sec-credit/climb-qwen-35b-verified/v7/change.md585 B
  • envs/sec-credit/climb-qwen-35b-verified/v7/prompt.txt2.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v7/results.jsonl75.1 KB
  • envs/sec-credit/climb-qwen-35b-verified/v7/summary.json657 B
  • envs/sec-credit/climb-qwen-35b-verified/v8/change.md425 B
  • envs/sec-credit/climb-qwen-35b-verified/v8/prompt.txt1.8 KB
  • envs/sec-credit/climb-qwen-35b-verified/v8/results.jsonl11.8 KB
  • envs/sec-credit/climb-qwen-35b-verified/v8/summary.json500 B
  • envs/sec-credit/climb-qwen-35b-verified/v9/change.md372 B
  • envs/sec-credit/climb-qwen-35b-verified/v9/errors.jsonl1.1 KB
  • envs/sec-credit/climb-qwen-35b-verified/v9/prompt.txt1.8 KB
  • envs/sec-credit/climb-qwen-35b-verified/v9/results.jsonl150.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v9/summary.json444 B
  • envs/sec-credit/kb.json998.7 KB
  • envs/sec-credit/models/_state.json393 B
  • envs/sec-credit/models/baseline/change.md9 B
  • envs/sec-credit/models/baseline/dropped.jsonl0 B
  • envs/sec-credit/models/baseline/errors.jsonl1.3 KB
  • envs/sec-credit/models/baseline/results.jsonl326.6 KB
  • envs/sec-credit/models/report.html388.2 KB
  • envs/sec-credit/models/v1/change.md5 B
  • envs/sec-credit/models/v1/dropped.jsonl0 B
  • envs/sec-credit/models/v1/results.jsonl322.2 KB
  • envs/sec-credit/models/v2/change.md9 B
  • envs/sec-credit/models/v2/errors.jsonl732 B
  • envs/sec-credit/models/v2/results.jsonl317.6 KB
  • envs/sec-credit/models/v3/change.md9 B
  • envs/sec-credit/models/v3/results.jsonl323.1 KB
  • envs/sec-credit/models/v4/change.md6 B
  • envs/sec-credit/models/v4/results.jsonl324.5 KB
  • envs/sec-credit/models/v5/change.md9 B
  • envs/sec-credit/models/v5/results.jsonl325.6 KB
  • envs/sec-credit/models/v6/change.md13 B
  • envs/sec-credit/models/v6/results.jsonl320.9 KB
  • envs/sec-credit/models/v7/change.md3 B
  • envs/sec-credit/models/v7/results.jsonl308.5 KB
  • envs/sec-credit/models/v8/change.md8 B
  • envs/sec-credit/models/v8/results.jsonl350.8 KB
  • envs/sec-credit/spec.json1002 B
  • envs/sec-credit/tasks.jsonl7.7 MB
  • envs/stats.json19.1 KB
  • envs/trial-forecast/audit.json8.4 KB
  • envs/trial-forecast/climb-qwen-35b/_state.json13.4 KB
  • envs/trial-forecast/climb-qwen-35b/baseline/change.md9 B
  • envs/trial-forecast/climb-qwen-35b/baseline/dropped.jsonl37.9 KB
  • envs/trial-forecast/climb-qwen-35b/baseline/prompt.txt0 B
  • envs/trial-forecast/climb-qwen-35b/baseline/results.jsonl248.0 KB
  • envs/trial-forecast/climb-qwen-35b/baseline/summary.json75 B
  • envs/trial-forecast/climb-qwen-35b/report.html140.6 KB
  • envs/trial-forecast/climb-qwen-35b/v1/change.md289 B
  • envs/trial-forecast/climb-qwen-35b/v1/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v1/prompt.txt1.1 KB
  • envs/trial-forecast/climb-qwen-35b/v1/results.jsonl68.6 KB
  • envs/trial-forecast/climb-qwen-35b/v1/summary.json355 B
  • envs/trial-forecast/climb-qwen-35b/v10/change.md320 B
  • envs/trial-forecast/climb-qwen-35b/v10/dropped.jsonl30.3 KB
  • envs/trial-forecast/climb-qwen-35b/v10/prompt.txt1.6 KB
  • envs/trial-forecast/climb-qwen-35b/v10/results.jsonl238.6 KB
  • envs/trial-forecast/climb-qwen-35b/v10/summary.json390 B
  • envs/trial-forecast/climb-qwen-35b/v11/change.md342 B
  • envs/trial-forecast/climb-qwen-35b/v11/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v11/prompt.txt1.6 KB
  • envs/trial-forecast/climb-qwen-35b/v11/results.jsonl68.5 KB
  • envs/trial-forecast/climb-qwen-35b/v11/summary.json411 B
  • envs/trial-forecast/climb-qwen-35b/v12/change.md290 B
  • envs/trial-forecast/climb-qwen-35b/v12/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v12/prompt.txt1.7 KB
  • envs/trial-forecast/climb-qwen-35b/v12/results.jsonl68.6 KB
  • envs/trial-forecast/climb-qwen-35b/v12/summary.json356 B
  • envs/trial-forecast/climb-qwen-35b/v2/change.md300 B
  • envs/trial-forecast/climb-qwen-35b/v2/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v2/prompt.txt861 B
  • envs/trial-forecast/climb-qwen-35b/v2/results.jsonl68.3 KB
  • envs/trial-forecast/climb-qwen-35b/v2/summary.json366 B
  • envs/trial-forecast/climb-qwen-35b/v3/change.md289 B
  • envs/trial-forecast/climb-qwen-35b/v3/dropped.jsonl44.8 KB
  • envs/trial-forecast/climb-qwen-35b/v3/prompt.txt1.3 KB
  • envs/trial-forecast/climb-qwen-35b/v3/results.jsonl317.3 KB
  • envs/trial-forecast/climb-qwen-35b/v3/summary.json355 B
  • envs/trial-forecast/climb-qwen-35b/v4/change.md446 B
  • envs/trial-forecast/climb-qwen-35b/v4/dropped.jsonl30.3 KB
  • envs/trial-forecast/climb-qwen-35b/v4/prompt.txt1.7 KB
  • envs/trial-forecast/climb-qwen-35b/v4/results.jsonl238.6 KB
  • envs/trial-forecast/climb-qwen-35b/v4/summary.json512 B
  • envs/trial-forecast/climb-qwen-35b/v5/change.md403 B
  • envs/trial-forecast/climb-qwen-35b/v5/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v5/prompt.txt1.9 KB
  • envs/trial-forecast/climb-qwen-35b/v5/results.jsonl68.6 KB
  • envs/trial-forecast/climb-qwen-35b/v5/summary.json469 B
  • envs/trial-forecast/climb-qwen-35b/v6/change.md403 B
  • envs/trial-forecast/climb-qwen-35b/v6/dropped.jsonl6.7 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

rl-env-use

Anthropic's build-eval -> hillclimb workflow with real inputs. If the claude-api skill is available, its shared/evals/build-eval.md and shared/evals/eval-hillclimb.md are the method. A case = the materials the model gets + its true outcome, published elsewhere or later. Browser agents mine both. The model under test reads the materials and answers without a browser.

Commands: rl-env-use ... (or python -m rl_env_use ...). Every env is one folder, ./envs/<name>/.

Where things are saved

envs/<name>/
  spec.json        topic, skill, sources, conventions, budget; `create` adds spend and result
  kb.json          what the agents mined: materials, quote-verified outcome facts, missions
  tasks.jsonl      every drafted task with gold and checks (the env serves the ones that pass)
  audit.json       latest audit (also printed by `report`)
  reviews.jsonl    your verdicts and gold fixes from the review app
  models/          model comparisons, one slot per model (named in <slot>/change.md)
  climb-<model>/   a hillclimb: baseline, v1..vN (change.md = the edit, prompt.txt = full prompt), _state.json, report.html
  hillclimb/       runs by hand; ablation/: runs with materials withheld (--blind)
  <flow>/<slot>/   results.jsonl (graded rows), errors.jsonl (infra failures), traces/ (transcripts)

Slots are baseline, v1, v2, ...; change.md's first line is the report's column title. Traces, trajectory/ and logs are git-ignored.

Autonomous (default when the user wants it hands-off)

  1. create "<domain, skill, sources, constraints in one line>" [--tasks 150] [--budget 40]. A planner writes the spec (skill, sources, conventions, an outcome-blind sampling rule). Mining rounds then run until the task target, the budget, or two rounds with no new tasks. Each round: mine -> write -> verify -> panel -> harden -> verify -> panel.
    • Derived numbers are computed by code from an expression over numbers in the materials.
    • The frontier panel (opus55, gpt-astra, gemini) measures difficulty. All-pass tasks go back to the drafter and come back harder on the same case, up to two levels. Forecasts must move with the materials and must not be answerable without them. At most 60% of served forecasts resolve one way.
    • When the panel saturates derive tasks (on SEC filings it solves nearly every computation), later rounds write forecasts. Derive tasks stay useful for smaller policies: pass --panel with the policy family.
    • Mining is fastest when the line names a URL that lists records (a registry API query, a search page).
    • Ctrl-C stops cleanly (cloud browsers are released); create --name <env> resumes with its spend so far.
  2. climb envs/<name> --model M: the hillclimb loop in climb-<model>/. Split by case, A/A noise from rep halves, refuses saturated or underpowered setups, K edits per round screened on train, kept only on a paired case-bootstrap test gain, failure audit on stalls, final baseline-vs-best on fresh reps. Report the final line and climb-<model>/report.html as they are: "within noise, do not merge" is a result.

On a terminal both show a live board (stage bar with ETA, spend vs budget, kept tasks; for climb: baseline, best, rounds). In a log they print status lines.

Step by step (when the user wants to review)

Ask every decision with AskUserQuestion, your pick first as "(Recommended)". Wait for a clear yes at each sign-off.

Step 0: what is evaluated. Ask: (1) the skill as input -> outcome ("given a trial's protocol, predict whether it met its primary endpoint"), one per env; (2) where inputs and outcomes live (2-6 URLs), ideally separated in place or time; (3) logins (Browser Use profile id) or a proxy country. Then new <name> --topic "<domain>" --skill "<input -> outcome>" --sources URL.. [--profile ID] [--proxy-country CC]. Without --skill it is a browse-task env (the model must find answers on the live web).

Step 1: mine the cases (sign-off 1). build envs/<name> mines (~$0.15-0.40 per mission) and stops before any task exists. Show the look output (build envs/<name> --only look reprints it). The user checks review envs/<name> -> Data tab. Too few cases: build --only scout. Tasks that compute from tables need them in the materials: build --only scout --enrich. SIGN-OFF: "Are these the cases the environment should be built from?"

Step 2: tasks and grading (sign-off 2). build envs/<name> again writes and checks tasks: gold from outcome quotes, no leak into prompt or materials, closed-book memory check, live evidence, a critic. Grading is programmatic (number / date / set / exact / contains / prob), self-tested against null, foreign and perturbed golds. Show 5 tasks (Tasks tab). SIGN-OFF: "Would you grade any of these differently?"

Step 3: pilot and cost (sign-off 3). run envs/<name> --model M --limit 5, then run ... --dry-run prices the full run from measured cost per episode. SIGN-OFF on that number.

Step 4: baseline. run envs/<name> --model M --flow models per model (each gets its own slot). report envs/<name> regrades, audits and writes each report.html. inverted cases usually mean a task or grader bug. Headroom is relative to the policy: frontier-saturated derive tasks can still train a smaller model; for frontier headroom use forecasts on lesser-known records. Ask whether to commit the env.

Step 5: hillclimb. climb envs/<name> --model M, or by hand: split envs/<name>, then per round run envs/<name> --model M --variant vN --system prompt.txt. Label failures in the Traces tab, codify, then judges (trusted once they agree with the user's labels). export envs/<name> writes the dataset for an RL trainer; from rl_env_use import Env serves it (reset, reward).

Optional: measure the edge

compare envs/<name> --missions 8 reruns the mining with vanilla Claude Code (WebSearch + WebFetch, same model and instructions) and pushes both arms through the same synth + verify.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

Report a problem with this skill