Operate apps and websites by inspecting screenshots, clicking controls, typing, and dragging. Use it to fill forms, check layouts, and test complete user flows, with recorded actions available for replay and review.
中文简介正在准备中。当前显示英语简介。
Use when you need to visually interact with a GUI: test buttons, fill forms, verify visual layouts, fuzz web pages, automate user flows, take screenshots, or perform end-to-end QA on any application. Works on cloud VMs, Docker containers, local machines, and sandboxes. Needs the `cua` CLI (curl -fsSL https://cua.ai/install.sh | sh).
安装前可以查看交给 AI 代理的指令内容。
CUA gives you eyes and hands on a real computer: see the screen, move the mouse, click, type, drag, and manage windows, like a human at the keyboard.
Use this skill for visual interaction that can't be done via shell or API.
cua --version # check install; if missing: curl -fsSL https://cua.ai/install.sh | sh
# Connect to target (pick one)
cua do switch my-sandbox # any sandbox from `cua sb ls`
cua do switch relay:<machine-id> # a relay Space from `cua spaces ls`
cua do switch url http://127.0.0.1:3211 --as dev # a cua-spacesd by URL
cua do-host-consent && cua do switch host # local machine (one-time consent)
ANTHROPIC_API_KEYis optional. With it,cua do snapshotreturns an AI-annotated screen with element coordinates. Without it, usescreenshotand read the image yourself.
Look → Act → Verify: repeat until done, then share:
cua do screenshot # look
cua do click 450 280 # act
cua do screenshot # verify
cua trajectory view # open the replay for the user
Re-screenshot after every UI change: coordinates go stale when the screen changes.
cua do screenshot
cua do click 450 280
cua do screenshot
cua do screenshot
cua do click 400 200 && cua do type "Jane Doe"
cua do key tab && cua do type "jane@example.com"
cua do key tab && cua do type "SecureP@ss123"
cua do click 400 500
cua do screenshot
cua do click 350 400 # "Choose File"
cua do type "/home/user/report.pdf"
cua do key enter
cua do screenshot
When clicking small or dense UI elements, especially on the host machine, zoom into the target window first. Coordinates become window-relative and screenshots show only that window, giving you higher effective resolution.
cua do zoom "Google Chrome" # crop to Chrome window; coords are now window-relative
cua do screenshot # zoomed view, easier to locate small elements
cua do click 112 44 # precise click on a small tab or button
cua do screenshot # verify
cua do unzoom # restore full-screen coords when done
cua do screenshot # back to full desktop view
Use
zoomany time click accuracy is uncertain.unzoombefore switching windows or when you need to see the full desktop again.
cua do window ls # list open windows
cua do drag 150 300 650 400 # source → destination
cua do screenshot
cua do screenshot
cua do click 400 200
cua do type "<script>alert(1)</script>"
cua do key tab && cua do type "'; DROP TABLE users; --"
cua do key tab && cua do type "AAAAAAAAAAAAAAAAAAAAAAA"
cua do click 400 500
cua do screenshot # check for errors, crashes, unexpected behavior
Every action is auto-recorded to ~/.cua/trajectories/{machine}/{session}/.
cua trajectory view # open the replay viewer (do this at the end)
cua trajectory ls # list sessions
cua trajectory clean # delete sessions
cua do --no-record click 100 200 # disable recording for a single action
Tell the user where the trajectory is (cua trajectory ls prints the path).
| Action | Command |
|---|---|
| Connect to target | cua do switch <sandbox> |
| Screenshot | cua do screenshot |
| AI-annotated screen | cua do snapshot ["instructions"] |
| Click | cua do click <x> <y> [left|right|middle] |
| Double-click | cua do dclick <x> <y> |
| Type text | cua do type "text" |
| Press key | cua do key <key> |
| Hotkey | cua do hotkey <combo> (e.g. ctrl+c) |
| Scroll | cua do scroll <direction> [amount] |
| Drag | cua do drag <x1> <y1> <x2> <y2> |
| Move cursor | cua do move <x> <y> |
| Shell command | cua do shell "command" |
| Open URL/file | cua do open <url|path> |
| List windows | cua do window ls [app] |
| Focus window | cua do window focus <id> |
| Zoom to window | cua do zoom "App Name" |
| Unzoom | cua do unzoom |
| View trajectory | cua trajectory view |
| Target | Example |
|---|---|
| a sandbox | cua do switch my-sandbox (see cua sb ls) |
| a relay Space | cua do switch relay:<machine-id> (cua spaces ls) |
| a spacesd URL | cua do switch url http://host:3211 --as x |
| this machine | cua do switch host |
Legacy <provider> <name> pairs (cloud, docker, lume, ...) still work.
See references/command-reference.md for full argument syntax.
还没有评价。欢迎分享你的使用感受。
无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。
还没有评论。来写第一条吧。
Operate native apps on macOS, Windows, and Linux using identified controls, menu paths, or screen coordinates. Target a specific window, verify the result after each action, and consult available recent activity when resuming work.
中文简介正在准备中。当前显示英语简介。
Create disposable Linux or macOS computers for running code, testing apps, operating desktop interfaces, and browsing websites. Use isolated environments locally or in the Cua cloud, then delete them when the work is complete.
中文简介正在准备中。当前显示英语简介。
Work in a local or remote computer environment whose screen you can watch. Run commands, transfer files, launch coding agents, move signed-in app sessions, and share the host computer’s network when needed.
中文简介正在准备中。当前显示英语简介。
Save agent memory and outputs in Cua Volume, shared storage that persists beyond a Space, the working environment. Hand files to the user, read shared references, and check synchronization, version history, and conflicting edits.
中文简介正在准备中。当前显示英语简介。
Build a computer-control loop where Cua Driver observes and acts, while TypeSafe Jev chooses from predefined actions. Validate each choice, execute one action at a time, and verify the result before continuing.
中文简介正在准备中。当前显示英语简介。
Review open GitHub issues, proposals, and pull requests to identify useful maintainer work. Get ranked candidates with evidence, dependencies, and validation needs, then begin work only on an item you explicitly select.
中文简介正在准备中。当前显示英语简介。