본문으로 건너뛰기
cccskills
무료GitHub에 공개

rl-env-use

Build an RL environment / eval that trains or measures a skill in any domain (finance, medicine, law, math, science, a niche industry) from real cases that Browser Use agents mine from the web, then baseline and hillclimb a model on it. Use when the user knows a domain and what a model should be able to do in it, but not how to get the data or build the environment.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

browser-use/rl-env-use
설치 방법 보기

포함된 파일(200)

  • SKILL.md6.4 KB
  • .env.example41 B
  • .gitignore284 B
  • envs/sec-credit/audit.json18.6 KB
  • envs/sec-credit/climb-haiku/_state.json13.3 KB
  • envs/sec-credit/climb-haiku/baseline/change.md9 B
  • envs/sec-credit/climb-haiku/baseline/dropped.jsonl40.7 KB
  • envs/sec-credit/climb-haiku/baseline/prompt.txt0 B
  • envs/sec-credit/climb-haiku/baseline/results.jsonl195.2 KB
  • envs/sec-credit/climb-haiku/baseline/summary.json72 B
  • envs/sec-credit/climb-haiku/report.html120.3 KB
  • envs/sec-credit/climb-haiku/v1/change.md482 B
  • envs/sec-credit/climb-haiku/v1/dropped.jsonl34.9 KB
  • envs/sec-credit/climb-haiku/v1/prompt.txt1.2 KB
  • envs/sec-credit/climb-haiku/v1/results.jsonl164.4 KB
  • envs/sec-credit/climb-haiku/v1/summary.json545 B
  • envs/sec-credit/climb-haiku/v2/change.md305 B
  • envs/sec-credit/climb-haiku/v2/dropped.jsonl8.8 KB
  • envs/sec-credit/climb-haiku/v2/prompt.txt628 B
  • envs/sec-credit/climb-haiku/v2/results.jsonl40.3 KB
  • envs/sec-credit/climb-haiku/v2/summary.json368 B
  • envs/sec-credit/climb-haiku/v3/change.md234 B
  • envs/sec-credit/climb-haiku/v3/dropped.jsonl8.8 KB
  • envs/sec-credit/climb-haiku/v3/prompt.txt472 B
  • envs/sec-credit/climb-haiku/v3/results.jsonl40.3 KB
  • envs/sec-credit/climb-haiku/v3/summary.json297 B
  • envs/sec-credit/climb-haiku/v4/change.md489 B
  • envs/sec-credit/climb-haiku/v4/dropped.jsonl49.5 KB
  • envs/sec-credit/climb-haiku/v4/prompt.txt1.3 KB
  • envs/sec-credit/climb-haiku/v4/results.jsonl235.2 KB
  • envs/sec-credit/climb-haiku/v4/summary.json552 B
  • envs/sec-credit/climb-haiku/v5/change.md261 B
  • envs/sec-credit/climb-haiku/v5/dropped.jsonl8.7 KB
  • envs/sec-credit/climb-haiku/v5/prompt.txt1.4 KB
  • envs/sec-credit/climb-haiku/v5/results.jsonl40.5 KB
  • envs/sec-credit/climb-haiku/v5/summary.json324 B
  • envs/sec-credit/climb-haiku/v6/change.md393 B
  • envs/sec-credit/climb-haiku/v6/dropped.jsonl35.0 KB
  • envs/sec-credit/climb-haiku/v6/prompt.txt2.1 KB
  • envs/sec-credit/climb-haiku/v6/results.jsonl164.4 KB
  • envs/sec-credit/climb-haiku/v6/summary.json456 B
  • envs/sec-credit/climb-haiku/v7/change.md361 B
  • envs/sec-credit/climb-haiku/v7/dropped.jsonl34.9 KB
  • envs/sec-credit/climb-haiku/v7/prompt.txt1.9 KB
  • envs/sec-credit/climb-haiku/v7/results.jsonl164.3 KB
  • envs/sec-credit/climb-haiku/v7/summary.json424 B
  • envs/sec-credit/climb-haiku/v8/change.md405 B
  • envs/sec-credit/climb-haiku/v8/dropped.jsonl34.9 KB
  • envs/sec-credit/climb-haiku/v8/prompt.txt2.1 KB
  • envs/sec-credit/climb-haiku/v8/results.jsonl164.5 KB
  • envs/sec-credit/climb-haiku/v8/summary.json468 B
  • envs/sec-credit/climb-qwen-35b-verified/_state.json15.5 KB
  • envs/sec-credit/climb-qwen-35b-verified/baseline/change.md9 B
  • envs/sec-credit/climb-qwen-35b-verified/baseline/prompt.txt0 B
  • envs/sec-credit/climb-qwen-35b-verified/baseline/results.jsonl731.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/baseline/summary.json75 B
  • envs/sec-credit/climb-qwen-35b-verified/report.html525.9 KB
  • envs/sec-credit/climb-qwen-35b-verified/v1/change.md413 B
  • envs/sec-credit/climb-qwen-35b-verified/v1/errors.jsonl122 B
  • envs/sec-credit/climb-qwen-35b-verified/v1/prompt.txt1.5 KB
  • envs/sec-credit/climb-qwen-35b-verified/v1/results.jsonl158.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v1/summary.json479 B
  • envs/sec-credit/climb-qwen-35b-verified/v10/change.md247 B
  • envs/sec-credit/climb-qwen-35b-verified/v10/prompt.txt1.4 KB
  • envs/sec-credit/climb-qwen-35b-verified/v10/results.jsonl159.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v10/summary.json313 B
  • envs/sec-credit/climb-qwen-35b-verified/v11/change.md423 B
  • envs/sec-credit/climb-qwen-35b-verified/v11/prompt.txt1.7 KB
  • envs/sec-credit/climb-qwen-35b-verified/v11/results.jsonl159.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v11/summary.json495 B
  • envs/sec-credit/climb-qwen-35b-verified/v12/change.md321 B
  • envs/sec-credit/climb-qwen-35b-verified/v12/prompt.txt1.5 KB
  • envs/sec-credit/climb-qwen-35b-verified/v12/results.jsonl159.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v12/summary.json387 B
  • envs/sec-credit/climb-qwen-35b-verified/v13/change.md425 B
  • envs/sec-credit/climb-qwen-35b-verified/v13/prompt.txt1.5 KB
  • envs/sec-credit/climb-qwen-35b-verified/v13/results.jsonl159.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v13/summary.json497 B
  • envs/sec-credit/climb-qwen-35b-verified/v14/change.md330 B
  • envs/sec-credit/climb-qwen-35b-verified/v14/errors.jsonl122 B
  • envs/sec-credit/climb-qwen-35b-verified/v14/prompt.txt1.4 KB
  • envs/sec-credit/climb-qwen-35b-verified/v14/results.jsonl158.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v14/summary.json396 B
  • envs/sec-credit/climb-qwen-35b-verified/v15/change.md274 B
  • envs/sec-credit/climb-qwen-35b-verified/v15/errors.jsonl122 B
  • envs/sec-credit/climb-qwen-35b-verified/v15/prompt.txt1.6 KB
  • envs/sec-credit/climb-qwen-35b-verified/v15/results.jsonl643.9 KB
  • envs/sec-credit/climb-qwen-35b-verified/v15/summary.json340 B
  • envs/sec-credit/climb-qwen-35b-verified/v2/change.md357 B
  • envs/sec-credit/climb-qwen-35b-verified/v2/prompt.txt1.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v2/results.jsonl159.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v2/summary.json423 B
  • envs/sec-credit/climb-qwen-35b-verified/v3/change.md600 B
  • envs/sec-credit/climb-qwen-35b-verified/v3/prompt.txt1.1 KB
  • envs/sec-credit/climb-qwen-35b-verified/v3/results.jsonl889.9 KB
  • envs/sec-credit/climb-qwen-35b-verified/v3/summary.json666 B
  • envs/sec-credit/climb-qwen-35b-verified/v4/change.md360 B
  • envs/sec-credit/climb-qwen-35b-verified/v4/prompt.txt2.0 KB
  • envs/sec-credit/climb-qwen-35b-verified/v4/results.jsonl159.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v4/summary.json432 B
  • envs/sec-credit/climb-qwen-35b-verified/v5/change.md283 B
  • envs/sec-credit/climb-qwen-35b-verified/v5/errors.jsonl122 B
  • envs/sec-credit/climb-qwen-35b-verified/v5/prompt.txt1.7 KB
  • envs/sec-credit/climb-qwen-35b-verified/v5/results.jsonl554.4 KB
  • envs/sec-credit/climb-qwen-35b-verified/v5/summary.json349 B
  • envs/sec-credit/climb-qwen-35b-verified/v6/change.md358 B
  • envs/sec-credit/climb-qwen-35b-verified/v6/prompt.txt1.7 KB
  • envs/sec-credit/climb-qwen-35b-verified/v6/results.jsonl159.8 KB
  • envs/sec-credit/climb-qwen-35b-verified/v6/summary.json424 B
  • envs/sec-credit/climb-qwen-35b-verified/v7/change.md585 B
  • envs/sec-credit/climb-qwen-35b-verified/v7/prompt.txt2.2 KB
  • envs/sec-credit/climb-qwen-35b-verified/v7/results.jsonl75.1 KB
  • envs/sec-credit/climb-qwen-35b-verified/v7/summary.json657 B
  • envs/sec-credit/climb-qwen-35b-verified/v8/change.md425 B
  • envs/sec-credit/climb-qwen-35b-verified/v8/prompt.txt1.8 KB
  • envs/sec-credit/climb-qwen-35b-verified/v8/results.jsonl11.8 KB
  • envs/sec-credit/climb-qwen-35b-verified/v8/summary.json500 B
  • envs/sec-credit/climb-qwen-35b-verified/v9/change.md372 B
  • envs/sec-credit/climb-qwen-35b-verified/v9/errors.jsonl1.1 KB
  • envs/sec-credit/climb-qwen-35b-verified/v9/prompt.txt1.8 KB
  • envs/sec-credit/climb-qwen-35b-verified/v9/results.jsonl150.3 KB
  • envs/sec-credit/climb-qwen-35b-verified/v9/summary.json444 B
  • envs/sec-credit/kb.json998.7 KB
  • envs/sec-credit/models/_state.json393 B
  • envs/sec-credit/models/baseline/change.md9 B
  • envs/sec-credit/models/baseline/dropped.jsonl0 B
  • envs/sec-credit/models/baseline/errors.jsonl1.3 KB
  • envs/sec-credit/models/baseline/results.jsonl326.6 KB
  • envs/sec-credit/models/report.html388.2 KB
  • envs/sec-credit/models/v1/change.md5 B
  • envs/sec-credit/models/v1/dropped.jsonl0 B
  • envs/sec-credit/models/v1/results.jsonl322.2 KB
  • envs/sec-credit/models/v2/change.md9 B
  • envs/sec-credit/models/v2/errors.jsonl732 B
  • envs/sec-credit/models/v2/results.jsonl317.6 KB
  • envs/sec-credit/models/v3/change.md9 B
  • envs/sec-credit/models/v3/results.jsonl323.1 KB
  • envs/sec-credit/models/v4/change.md6 B
  • envs/sec-credit/models/v4/results.jsonl324.5 KB
  • envs/sec-credit/models/v5/change.md9 B
  • envs/sec-credit/models/v5/results.jsonl325.6 KB
  • envs/sec-credit/models/v6/change.md13 B
  • envs/sec-credit/models/v6/results.jsonl320.9 KB
  • envs/sec-credit/models/v7/change.md3 B
  • envs/sec-credit/models/v7/results.jsonl308.5 KB
  • envs/sec-credit/models/v8/change.md8 B
  • envs/sec-credit/models/v8/results.jsonl350.8 KB
  • envs/sec-credit/spec.json1002 B
  • envs/sec-credit/tasks.jsonl7.7 MB
  • envs/stats.json19.1 KB
  • envs/trial-forecast/audit.json8.4 KB
  • envs/trial-forecast/climb-qwen-35b/_state.json13.4 KB
  • envs/trial-forecast/climb-qwen-35b/baseline/change.md9 B
  • envs/trial-forecast/climb-qwen-35b/baseline/dropped.jsonl37.9 KB
  • envs/trial-forecast/climb-qwen-35b/baseline/prompt.txt0 B
  • envs/trial-forecast/climb-qwen-35b/baseline/results.jsonl248.0 KB
  • envs/trial-forecast/climb-qwen-35b/baseline/summary.json75 B
  • envs/trial-forecast/climb-qwen-35b/report.html140.6 KB
  • envs/trial-forecast/climb-qwen-35b/v1/change.md289 B
  • envs/trial-forecast/climb-qwen-35b/v1/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v1/prompt.txt1.1 KB
  • envs/trial-forecast/climb-qwen-35b/v1/results.jsonl68.6 KB
  • envs/trial-forecast/climb-qwen-35b/v1/summary.json355 B
  • envs/trial-forecast/climb-qwen-35b/v10/change.md320 B
  • envs/trial-forecast/climb-qwen-35b/v10/dropped.jsonl30.3 KB
  • envs/trial-forecast/climb-qwen-35b/v10/prompt.txt1.6 KB
  • envs/trial-forecast/climb-qwen-35b/v10/results.jsonl238.6 KB
  • envs/trial-forecast/climb-qwen-35b/v10/summary.json390 B
  • envs/trial-forecast/climb-qwen-35b/v11/change.md342 B
  • envs/trial-forecast/climb-qwen-35b/v11/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v11/prompt.txt1.6 KB
  • envs/trial-forecast/climb-qwen-35b/v11/results.jsonl68.5 KB
  • envs/trial-forecast/climb-qwen-35b/v11/summary.json411 B
  • envs/trial-forecast/climb-qwen-35b/v12/change.md290 B
  • envs/trial-forecast/climb-qwen-35b/v12/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v12/prompt.txt1.7 KB
  • envs/trial-forecast/climb-qwen-35b/v12/results.jsonl68.6 KB
  • envs/trial-forecast/climb-qwen-35b/v12/summary.json356 B
  • envs/trial-forecast/climb-qwen-35b/v2/change.md300 B
  • envs/trial-forecast/climb-qwen-35b/v2/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v2/prompt.txt861 B
  • envs/trial-forecast/climb-qwen-35b/v2/results.jsonl68.3 KB
  • envs/trial-forecast/climb-qwen-35b/v2/summary.json366 B
  • envs/trial-forecast/climb-qwen-35b/v3/change.md289 B
  • envs/trial-forecast/climb-qwen-35b/v3/dropped.jsonl44.8 KB
  • envs/trial-forecast/climb-qwen-35b/v3/prompt.txt1.3 KB
  • envs/trial-forecast/climb-qwen-35b/v3/results.jsonl317.3 KB
  • envs/trial-forecast/climb-qwen-35b/v3/summary.json355 B
  • envs/trial-forecast/climb-qwen-35b/v4/change.md446 B
  • envs/trial-forecast/climb-qwen-35b/v4/dropped.jsonl30.3 KB
  • envs/trial-forecast/climb-qwen-35b/v4/prompt.txt1.7 KB
  • envs/trial-forecast/climb-qwen-35b/v4/results.jsonl238.6 KB
  • envs/trial-forecast/climb-qwen-35b/v4/summary.json512 B
  • envs/trial-forecast/climb-qwen-35b/v5/change.md403 B
  • envs/trial-forecast/climb-qwen-35b/v5/dropped.jsonl6.8 KB
  • envs/trial-forecast/climb-qwen-35b/v5/prompt.txt1.9 KB
  • envs/trial-forecast/climb-qwen-35b/v5/results.jsonl68.6 KB
  • envs/trial-forecast/climb-qwen-35b/v5/summary.json469 B
  • envs/trial-forecast/climb-qwen-35b/v6/change.md403 B
  • envs/trial-forecast/climb-qwen-35b/v6/dropped.jsonl6.7 KB

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

rl-env-use

Anthropic's build-eval -> hillclimb workflow with real inputs. If the claude-api skill is available, its shared/evals/build-eval.md and shared/evals/eval-hillclimb.md are the method. A case = the materials the model gets + its true outcome, published elsewhere or later. Browser agents mine both. The model under test reads the materials and answers without a browser.

Commands: rl-env-use ... (or python -m rl_env_use ...). Every env is one folder, ./envs/<name>/.

Where things are saved

envs/<name>/
  spec.json        topic, skill, sources, conventions, budget; `create` adds spend and result
  kb.json          what the agents mined: materials, quote-verified outcome facts, missions
  tasks.jsonl      every drafted task with gold and checks (the env serves the ones that pass)
  audit.json       latest audit (also printed by `report`)
  reviews.jsonl    your verdicts and gold fixes from the review app
  models/          model comparisons, one slot per model (named in <slot>/change.md)
  climb-<model>/   a hillclimb: baseline, v1..vN (change.md = the edit, prompt.txt = full prompt), _state.json, report.html
  hillclimb/       runs by hand; ablation/: runs with materials withheld (--blind)
  <flow>/<slot>/   results.jsonl (graded rows), errors.jsonl (infra failures), traces/ (transcripts)

Slots are baseline, v1, v2, ...; change.md's first line is the report's column title. Traces, trajectory/ and logs are git-ignored.

Autonomous (default when the user wants it hands-off)

  1. create "<domain, skill, sources, constraints in one line>" [--tasks 150] [--budget 40]. A planner writes the spec (skill, sources, conventions, an outcome-blind sampling rule). Mining rounds then run until the task target, the budget, or two rounds with no new tasks. Each round: mine -> write -> verify -> panel -> harden -> verify -> panel.
    • Derived numbers are computed by code from an expression over numbers in the materials.
    • The frontier panel (opus55, gpt-astra, gemini) measures difficulty. All-pass tasks go back to the drafter and come back harder on the same case, up to two levels. Forecasts must move with the materials and must not be answerable without them. At most 60% of served forecasts resolve one way.
    • When the panel saturates derive tasks (on SEC filings it solves nearly every computation), later rounds write forecasts. Derive tasks stay useful for smaller policies: pass --panel with the policy family.
    • Mining is fastest when the line names a URL that lists records (a registry API query, a search page).
    • Ctrl-C stops cleanly (cloud browsers are released); create --name <env> resumes with its spend so far.
  2. climb envs/<name> --model M: the hillclimb loop in climb-<model>/. Split by case, A/A noise from rep halves, refuses saturated or underpowered setups, K edits per round screened on train, kept only on a paired case-bootstrap test gain, failure audit on stalls, final baseline-vs-best on fresh reps. Report the final line and climb-<model>/report.html as they are: "within noise, do not merge" is a result.

On a terminal both show a live board (stage bar with ETA, spend vs budget, kept tasks; for climb: baseline, best, rounds). In a log they print status lines.

Step by step (when the user wants to review)

Ask every decision with AskUserQuestion, your pick first as "(Recommended)". Wait for a clear yes at each sign-off.

Step 0: what is evaluated. Ask: (1) the skill as input -> outcome ("given a trial's protocol, predict whether it met its primary endpoint"), one per env; (2) where inputs and outcomes live (2-6 URLs), ideally separated in place or time; (3) logins (Browser Use profile id) or a proxy country. Then new <name> --topic "<domain>" --skill "<input -> outcome>" --sources URL.. [--profile ID] [--proxy-country CC]. Without --skill it is a browse-task env (the model must find answers on the live web).

Step 1: mine the cases (sign-off 1). build envs/<name> mines (~$0.15-0.40 per mission) and stops before any task exists. Show the look output (build envs/<name> --only look reprints it). The user checks review envs/<name> -> Data tab. Too few cases: build --only scout. Tasks that compute from tables need them in the materials: build --only scout --enrich. SIGN-OFF: "Are these the cases the environment should be built from?"

Step 2: tasks and grading (sign-off 2). build envs/<name> again writes and checks tasks: gold from outcome quotes, no leak into prompt or materials, closed-book memory check, live evidence, a critic. Grading is programmatic (number / date / set / exact / contains / prob), self-tested against null, foreign and perturbed golds. Show 5 tasks (Tasks tab). SIGN-OFF: "Would you grade any of these differently?"

Step 3: pilot and cost (sign-off 3). run envs/<name> --model M --limit 5, then run ... --dry-run prices the full run from measured cost per episode. SIGN-OFF on that number.

Step 4: baseline. run envs/<name> --model M --flow models per model (each gets its own slot). report envs/<name> regrades, audits and writes each report.html. inverted cases usually mean a task or grader bug. Headroom is relative to the policy: frontier-saturated derive tasks can still train a smaller model; for frontier headroom use forecasts on lesser-known records. Ask whether to commit the env.

Step 5: hillclimb. climb envs/<name> --model M, or by hand: split envs/<name>, then per round run envs/<name> --model M --variant vN --system prompt.txt. Label failures in the Traces tab, codify, then judges (trusted once they agree with the user's labels). export envs/<name> writes the dataset for an RL trainer; from rl_env_use import Env serves it (reset, reward).

Optional: measure the edge

compare envs/<name> --missions 8 reruns the mining with vanilla Claude Code (WebSearch + WebFetch, same model and instructions) and pushes both arms through the same synth + verify.

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

이 스킬의 문제 신고