jev-use

Keep the decision layer above Cua Driver. Driver supplies observations and executes actions; the application constructs complete candidates; TypeSafe Jev returns one candidate ID. Never let Jev invent tool names, coordinates, refs, targets, delivery modes, or other arguments.

Use the example at libs/cua-driver/examples/jev-use/ as the runnable reference. Keep TypeSafe request construction in the external Jev adapter rather than in Driver or a Driver extension. The Python and TypeScript adapters must expose equivalent mock and live behavior. For a process boundary, use cua.jev_choice_request_v1 on stdin and require cua.jev_choice_v1 on stdout. The request contains only a goal, capture ID, compact regions, bounded history, and candidate IDs with descriptions; the response contains only the selected ID, model identity, confidence, and probabilities. Invoke the Python interpreter and absolute chooser path directly without a shell. For native desktop applications, use NativeAccessibilitySource and cua.jev_choice_request_v2, which adds a per-candidate source (page, ax, or visual), compact value-free elements, and optional progress counted from the runner's own performed actions. Browser tasks keep sending v1. Prefer browser DOM and semantic evidence. The optional visual adapter consumes the public cua.visual_regions_v1 result only when Driver advertises both parse_visual_regions and the capture-bound click.capture_id input. Use the checked-in fixtures for deterministic development; do not add a model, extension artifact, or Driver implementation detail to the recipe.

Decision loop

1. State the goal and obtain a fresh Cua Driver observation through one persistent CLI or MCP session.
2. Prefer an unambiguous fresh accessibility or browser DOM token.
3. If visual grounding is needed, discover parse_visual_regions through the current MCP tool inventory. Validate its versioned result, capture ID, screenshot reference and dimensions, coordinate mapping, unique region IDs, bounds, content, confidence, and ambiguity. Build a pixel action only with the exact capture ID in the same click call. Otherwise reobserve or abstain.
4. Construct a bounded candidate table. Each executable candidate contains the complete Driver tool and arguments. Include reobserve and abstain when evidence can be stale, incomplete, or ambiguous.
5. Send Jev only the goal, compact observation, recent history, and candidate IDs with descriptions. Include typed visual regions and their capture_id when the current observation has validated visual evidence; do not send extension internals or screenshot bytes.
6. Resolve the returned ID against the original immutable table. Reject an unknown, duplicate, malformed, denied, stale, or capture-mismatched choice, or a result below the caller's stated confidence policy.
7. Execute at most one Driver action. Use background delivery by default; foreground delivery is an explicit escalation subject to the active Driver contract and user authorization.
8. Reobserve and verify the postcondition before building another table.

Freshness and visual evidence

• Treat Driver page refs, accessibility tokens, screenshot IDs, and visual region IDs as observation-local. Never reuse them after the UI changes.
• Require visual bounds and centers to remain inside the exact screenshot coordinate space and tied to the same target and snapshot.
• If semantic and visual evidence disagree, or multiple regions are plausible, offer reobserve and abstain without inventing a mutation.
• Never remove capture_id or retry an expired, stale, or mismatched capture as an unbound coordinate action.
• Use semantic evidence as authority when it is available. A visual label does not prove editability or interactivity.

Native accessibility candidates

• Build native candidates from one get_window_state call that returns the tree and the screenshot together, so element tokens and capture_id describe the same moment.
• Map raw AX, UIA, and AT-SPI roles through the role-class table in native_roles.py / native_roles.ts, keyed by Driver's normalized_role. Do not normalize roles in Driver.
• Offer only enabled, on-screen, labeled, native elements; exclude in_web_content elements, window chrome, and labels equal to the value.
• Derive candidate IDs from role class, label, and actionable-ancestor path, never element_index. Cap at 24 action candidates plus reobserve and abstain, and log how many were dropped.
• Exclude delete, send, purchase, and close actions unless the task spec allows that risk. Text comes only from task parameters.
• A stale token or truncated tree leads to a reobserve, never to an unbound coordinate action.
• Verify completion through the task's independent oracle, such as the harness task-state file, not the accessibility tree the model saw.

Credentials and proof

The deterministic mock path must work without TYPESAFE_API_KEY. For live Jev, read the key from the process environment or a secure interactive prompt; never put it in source, command arguments, logs, artifacts, or messages. Verify task completion from an independent application postcondition rather than a model answer, action response, or screenshot alone.