Skip to content
cccskills
FreePublished on GitHub

jevaluate-harness

Use when running a jevaluate rating round, re-rating after a rubric change, changing the jevaluate rubric, or publishing ratings to the jev-mode ratings folder.

The English summary is being prepared. Showing the original description.

tiffygk/jev-mode
See how to install

Files included (23)

  • SKILL.md2.4 KB
  • fixtures/bad-rating-evidence/files/src__x.py103 B
  • fixtures/bad-rating.md1.5 KB
  • fixtures/bad-rating.md.steps.json517 B
  • fixtures/good-rating-evidence/files/src__x.py103 B
  • fixtures/good-rating.md1.5 KB
  • fixtures/good-rating.md.steps.json517 B
  • freeze_tag_check.py2.6 KB
  • images/skills-dark.png407.6 KB
  • images/skills-light.png364.7 KB
  • publish.md900 B
  • rater-brief.md2.1 KB
  • rating-round.md4.3 KB
  • README.md3.7 KB
  • rubric-change.md3.3 KB
  • scan_transcripts.py2.8 KB
  • scripts/check_release.py3.3 KB
  • scripts/dispute_packet.py3.6 KB
  • scripts/harness_status.py2.9 KB
  • test_dispute_packet.py2.3 KB
  • test_release.py2.1 KB
  • test_scan.py4.9 KB
  • test_status.py2.8 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Jevaluate Harness

The maintainer's process for jevaluate. One rating is jevaluate/read.md; testing the rubric is the jevaluate-eval skill. If $JEVALUATE_OVERLAY is set, read that file first; its rules override the defaults here.

Settings (yours, not in this skill)

  • $JEVALUATE_LIBRARY: the private ratings library (default ~/.claude/jevaluate-library if it exists, else ~/.jevaluate-library).
  • $PRIVATE_TERMS: your never-publish regexes, one per line. library.py export reads only $JEVALUATE_LIBRARY/private-terms.txt, so keep the file there (or symlink it) and have your pre-push hook read the same file.
  • $JEVALUATE_OVERLAY: a markdown note of your own rules for these workflows, read first; no required format.

Two principles

  • Code checks, text explains: every rule has a row in jevaluate/enforcement.md naming the check that refuses a wrong answer, or why it stays judgment.
  • Ask for what is observed: raters record type, the calls-Jev line, each decision's stakes as read and the verdict; library.py add derives kind and every n.a.

Workflows

Open the file for the workflow and follow it in order; each step names the gate that must pass.

WorkflowFileDone when
Change the rubricrubric-change.mdjevaluate-eval passes and the owner approves the version
Rate or re-rate projectsrating-round.md (rater prompt: rater-brief.md)every rating logged by library.py add, scan clean
Publish ratingspublish.mdone pull request, reviewed

To see where you are: python3 jevaluate-harness/scripts/harness_status.py <ledger> --workflow rubric|round (round runs through publishing). It reruns the scripted gates it has (the materials check), reads the other steps from the ledger, and names the file to open next. Ledger lines: done: <step> <YYYY-MM-DD> <evidence>. Steps: rubric = rules-written, materials-checked, quiz-passed, eval-passed, owner-approved, frozen; round = picked, sized, costs-approved, rated, scanned, adjudicated, final-gate, release-check, published.

Keep a ledger per workflow: each step, each ruling with its cost if wrong, and tokens per agent. Record rulings and handoffs where your overlay says, or in the ledger.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

jev-lens

Free

Legacy. Use only when images (photos, illustrations, screenshots) must reach TypeSafe's Jev (System One) model as text, because the request can't send the `TypeSafe-Danger-Zone-Images` header that lets Jev read images directly. Triggers on "turn this photo into a Jev state", "describe these images for Jev", "decode images for System One", or a Jev project whose decision depends on what a picture shows. Handles 1 to about 100 images; produces a neutral, complete decode and a JSON state, checked against Jev.

The English summary is being prepared. Showing the original description.

tiffygk/jev-mode12Updated Oct 16, 2026

jev-sources

Free

Finds and quotes TypeSafe's docs on Jev, including Noul, Choice, Score and Box questions, models and limits. Use when a question, review, design or claim involves Jev or System One, when briefing a subagent on Jev work, or when relaying a subagent's Jev finding.

The English summary is being prepared. Showing the original description.

tiffygk/jev-mode12Updated Oct 16, 2026

jevaluate

Free

Use when rating whether a project, repo, skill, workflow, plugin, agent product or post uses TypeSafe's Jev (System One) model well -- "is this a good use of Jev?", "rate this Jev integration", "jevaluate <url>", or a link to something named after Jev; also when checking whether something that calls itself Jev actually uses it.

The English summary is being prepared. Showing the original description.

tiffygk/jev-mode12Updated Oct 16, 2026

jevaluate-eval

Free

Use when testing whether the jevaluate rubric and instructions lead graders to the right answers: after a rubric or instruction change, before freezing a rubric, before publishing ratings, or when a grader's calls look wrong.

The English summary is being prepared. Showing the original description.

tiffygk/jev-mode12Updated Oct 16, 2026

jevaluate-private

Free

Rates a project with Jevaluate and keeps the rating private, with the same rubric and checks, a private library that export refuses, and nothing pushed or published. Use when a rating must stay private, such as "/jevaluate-private" followed by a link or folder, "rate this privately", "evaluate my own project", a client's or employer's code, a design doc or a local folder that isn't public, or any rating that must not reach the public ratings or the repo.

The English summary is being prepared. Showing the original description.

tiffygk/jev-mode12Updated Oct 16, 2026

See all skills by tiffygk

Report a problem with this skill