/jury
Score a website the way the Awwwards jury would: from what a visitor sees and what the browser
measures, never from the code, the design notes or what the owner meant. Hand back a score card
with evidence for every number and a ranked list of fixes.
Not affiliated with Awwwards. The criteria and weights are theirs and public
(awwwards.com/about-evaluation); the scores are an estimate. Say so on every card.
Skill root: the directory containing this file ($SKILL; substitute the absolute path in
commands). The scripts need Node 20+ and Playwright, plus axe-core for the accessibility pass.
references/quality-bar.md has the system, the rubric and the critic brief.
references/calibration.md has the measured gap between this critic and real juries. It is for
you, for the score card. Never give it to the critic.
Why blind
Anyone who knows what a site is meant to do scores the intent instead of the screen. A juror
gets a URL and a few minutes. So the critic gets only screenshots and measurements: no source,
no DESIGN.md, no README, no pitch from the owner. If the user explains the idea, hold it until
the score is in, then compare (step 5).
Workflow
1. Target
- A URL: score what is deployed.
- A local project, no URL: build it and serve the production output. Never score a dev
server: unminified JS and dev overlays wreck WPO and add console noise.
- Next.js:
npm run build, then npx next start -p 4400 in the background.
- Static output (Astro, Vite, Next
output: 'export'):
node $SKILL/scripts/serve.mjs <dist|out> --port 4400 in the background.
- Anything else: its own production build and start commands.
- A dev URL only (localhost with hot reload): build production instead if the project is
here. If it isn't, score it and say the WPO numbers are pessimistic.
- Several URLs ("mine vs theirs"): run steps 3 to 5 for each, each with its own fresh
critic, then put the cards side by side.
- Behind a login or password: it can't be scored. Ask for a public URL or a local build.
Evidence goes in .jury/<host>/ in the current directory (<host> such as example.com or
127.0.0.1-4400). Use absolute paths for it below.
2. Tools
The scripts resolve Playwright and axe-core from their working directory.
3. Evidence
node $SKILL/scripts/shoot.mjs <url> --crawl --nav --out <abs>/.jury/<host>/shots
node $SKILL/scripts/audit.mjs <url> > <abs>/.jury/<host>/audit.txt
shoot.mjs --crawl takes one page of every type (home, each top-level page, one item per
collection) plus the 404. For each: desktop at 1440, a phone at 390 throttled to a mid-range
device (4x CPU, slow 4G), reduced motion, keyboard focus, rest/hover pairs for pointer
targets, idle motion, and page-transition frames. Allow a minute or two per page.audit.mjs exits with code 1 when it finds a FAIL. That is a finding, not a broken run.- For a quick look at one page, drop
--crawl and pass --max 10 to the audit. Say on the
card that it covers one page.
- Check the evidence is real before scoring. Open
shots/home/d-01-hero.png and
shots/summary.json. A bot challenge, a login wall, an error page, or a blank frame with
errors in report.json means there is nothing to score: stop and tell the user. A cookie
banner is not a reason to stop. Visitors see it too, so it counts.
- Crawling is light: at most 40 HTML pages for the audit and 8 page types for screenshots.
Don't raise those limits on a site the user doesn't own.
4. Score
- With subagents: give a fresh one the critic brief from
references/quality-bar.md
("Critic brief"), with its three paths filled in (the rubric, .jury/<host>/shots/,
.jury/<host>/audit.txt). Give it nothing else, and never calibration.md. It reads every
screenshot, so your own context stays small.
- Without subagents: score it yourself under the same brief, from the evidence alone, and
mark the card as a self-score.
- If you built or edited this site in this conversation, a self-score isn't blind. Use a
subagent, or say on the card that the score is not independent.
5. Score card
Write .jury/<host>/scorecard.md and give the same card in chat:
# <site>: jury score
Estimate, not affiliated with Awwwards. <Independent critic | Self-score>, from <n> pages on <date>.
**The idea, as the critic read it:** one sentence
**Critic overall: 6.2** (0.4 Design + 0.3 Usability + 0.2 Creativity + 0.1 Content, on the critic's strict scale)
**Jury estimate: about 7.3** (critic + 1.1, measured on two recent Sites of the Day)
**Band, from the estimate:** below Honorable Mention (under 6.5) | Honorable Mention range (6.5 to 7.1) | Site of the Day range (7.2 and up)
| Criterion | Score | Evidence |
|---|---|---|
| Design (40%) | | file or report field, and what it shows |
| Usability (30%) | | |
| Creativity (20%) | | |
| Content (10%) | | |
| Developer | Score | Evidence |
|---|---|---|
| Semantics/SEO | | |
| Animations/Transitions | | |
| Accessibility | | |
| WPO | | |
| Responsive | | |
| Markup/Meta-data | | |
## Fixes, most score first
1. [Usability] m-03: what is wrong, and what to change
2. ...
## Measured
Mobile LCP and CLS, loader time, FPS, serious axe violations, overflow, `noHoverChange`,
`colorOnly`, idle motion, audit FAIL and WARN counts (from `summary.json` and the audit).
Not scored: anything stills can't show (how motion feels, sound).
Take the offset for the jury estimate from references/calibration.md (the 1.1 above is the
offset when this was written) and follow its rules: the critic's own numbers stay as scored, and
the developer areas get no jury estimate.
Then, in chat only:
- If the user told you what the site is meant to say, one line: does the critic's reading
match? If not, the site isn't saying it, and that is the first Creativity fix.
- The two fixes that would move the overall most.
- An offer: fix the list in this project if it's here, or run
/awwwards in it for a redesign.
Re-scoring after fixes
Run steps 3 to 5 again with a new critic, never the earlier one, and never show it the
earlier card: it would anchor on the old numbers. Before writing the new card, rename the old
one to scorecard-<n>.md, then show the change per category.
Rules
- No evidence, no points. Every score cites a screenshot file or a report field. Start each
category at 6 and make the evidence earn every point above it.
- Don't argue a score up. Dispute one only by pointing at a screenshot the critic misread.
- Judge the site on screen. Don't propose a new concept; fix what's there.
- Specific, not cruel: "m-03: the title wraps to five lines at 390 px; drop it to 14vw", never
"the typography needs work". Someone made this.