本文へ移動
cccskills
無料GitHub で公開

crawlberg-automating-the-browser

Use when extracting a page needs scripted interaction first — click, type, press a key, scroll, wait, screenshot, or run JS before capturing the DOM. Covers `crawlberg interact URL --actions` with the real action schema, result shape, limits, and external-CDP options.

日本語の概要は準備中です。原文の説明を表示しています。

xberg-io/crawlbergスター 192(GitHub 192 + cccskills 0)

実行スコア 未検証

インストール方法を見る

含まれるファイル(2)

  • SKILL.md6.4 KB
  • index.md565 B

SKILL.md(原文)

インストールする前に、エージェントに与えられる指示の中身を確認できます。

<!-- AI-RULEZ :: GENERATED FILE — DO NOT EDIT Content-Hash: blake3:a333d7ec586948ca3e195c02441494ee99272b227c25bf6037f920dc15d80b5b Source-Hash: blake3:4b18f96f8bf0da5fa02f0e2c5c6ad57db3a523b7eb72186f504977e42807760d Schema-Version: v1 -->

Automating the browser

crawlberg interact <url> --actions '[...]' drives a real headless browser through an ordered list of actions, then captures the resulting page. Reach for it when a static scrape is not enough: the content lives behind a "Load more" button, an infinite scroll, a form submission, or JS that only runs on interaction.

Quick recipe

crawlberg interact https://example.com \
  --actions '[{"type":"click","selector":"#load-more"},
              {"type":"wait","milliseconds":500},
              {"type":"scrape"}]'

Actions run in order. The result wraps the final page state under interaction (see Output below).

Flag surface

FlagDefaultPurpose
--actions—Required. JSON array of action objects (see below).
--formatjsonjson (full result) or markdown (final HTML only).
--timeout30000Per-request timeout in ms.
--browser-modeautoauto, always, never. Interaction needs a browser.
--browser-endpoint—External CDP ws:// or wss:// URL.
--config—Inline JSON or @file.json for the full CrawlConfig.

There is no --respect-robots-txt flag on interact; it targets the one URL you point it at.

Action schema

Each action is a JSON object tagged by type (camelCase). Unknown fields are rejected, so match the shapes exactly:

typeFieldsNotes
clickselectorClick the element matching the CSS selector.
typeselector, textType text into the input matching selector.
presskeyPress a key, e.g. "Enter", "Tab", "Escape".
scrolldirection ("up"/"down"), selector?, amount?Scroll the page, or a scrollable element if selector is set.
waitmilliseconds?, selector?Wait a fixed time, or until selector appears (selector wins).
screenshotfullPage?Capture the viewport, or the full scrollable page if true.
executeJsscriptRun arbitrary JS in the page context. Trusted scripts only.
scrape—Capture the current page HTML into the result.

There is no select action and no wait_for_selector action: to wait for an element, use wait with a selector field. Scroll direction is up or down only.

Limits (enforced — exceeding them errors)

  • Max 100 actions per call.
  • Single wait: max 300000 ms; total wait across all wait actions: max 300 s.
  • selector max 4096 bytes; text and executeJs script max 1 MB each.
  • scroll amount max 100000 px (absolute).

Output

JSON mode (default)

The result is wrapped under an interaction key. It carries the final page HTML (interaction.final_html) and a per-action record under interaction.action_results[...] — including screenshot data and executeJs return values where applicable.

crawlberg interact https://example.com \
  --actions '[{"type":"type","selector":"#q","text":"rust"},
              {"type":"press","key":"Enter"},
              {"type":"wait","selector":".results"},
              {"type":"scrape"}]' \
  --format json | jq '.interaction.final_html | length'

Markdown mode

Prints interaction.final_html directly — the raw final DOM, not converted Markdown. Use JSON mode and feed final_html to a scrape step when you need clean Markdown.

Patterns

Expand lazy content, then capture

crawlberg interact https://example.com/feed \
  --actions '[{"type":"scroll","direction":"down"},
              {"type":"wait","milliseconds":800},
              {"type":"scroll","direction":"down"},
              {"type":"wait","milliseconds":800},
              {"type":"scrape"}]'

Capture a full-page screenshot after login

crawlberg interact https://app.example.com \
  --actions '[{"type":"type","selector":"#email","text":"me@example.com"},
              {"type":"type","selector":"#password","text":"..."},
              {"type":"click","selector":"button[type=submit]"},
              {"type":"wait","selector":".dashboard"},
              {"type":"screenshot","fullPage":true}]'

Never hardcode credentials — pass them from the environment or a secrets store.

Point at an external Chrome

crawlberg interact https://example.com \
  --browser-endpoint ws://browser.internal:9222/devtools/browser/<id> \
  --actions '[{"type":"click","selector":"#go"},{"type":"scrape"}]'

See the crawlberg-headless-fallback skill for CDP endpoints, wait strategies, and persistent browser profiles set via --config.

When to reach for scrape or crawl instead

If the page yields its content on a plain fetch (or via --browser-mode always) with no clicking or typing, use crawlberg scrape — see crawlberg-scraping-html-to-markdown. Use interact only when ordered actions must run before the content exists.

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

コメント

ログインなしで書き込めます。使ってみた感想、質問、動かなかった環境などを気軽にどうぞ。

まだコメントはありません。最初のコメントを書いてみませんか。

送信するとすぐに公開されます。メールアドレスや電話番号などの個人情報は書かないでください。URL は 2 つまでです。

0 / 1000 文字

このフォームは reCAPTCHA で保護されており、Google のプライバシーポリシーと利用規約が適用されます。

同じリポジトリのスキル

api

無料

Use crawlberg api reference documentation.

日本語の概要は準備中です。原文の説明を表示しています。

xberg-io/crawlberg1922026年10月26日 更新

cli

無料

Use crawlberg cli reference documentation.

日本語の概要は準備中です。原文の説明を表示しています。

xberg-io/crawlberg1922026年10月26日 更新

crawlberg

無料

Crawl, scrape, and convert websites to Markdown using the local crawlberg CLI and its MCP server. Use when the user wants to fetch a page, follow links across a domain, enumerate URLs, or drive a real browser. Covers installation, the subcommands (scrape, crawl, map, interact, batch-scrape, batch-crawl, download, citations, version, mcp, serve), output formats (JSON + Markdown), browser fallback, and when to prefer the MCP server over shelling out.

日本語の概要は準備中です。原文の説明を表示しています。

xberg-io/crawlberg1922026年10月26日 更新

Use when the user wants to follow links across a domain and capture every reachable page as Markdown. Covers `crawlberg crawl` with depth, page caps, concurrency, rate limiting, domain scoping, robots, and output selection.

日本語の概要は準備中です。原文の説明を表示しています。

xberg-io/crawlberg1922026年10月26日 更新

Use when a static fetch returns nothing useful and the page needs a real browser. Covers `--browser-mode auto|always|never`, external CDP via `--browser-endpoint`, symptoms of JS-only pages and WAF blocks, and the performance cost.

日本語の概要は準備中です。原文の説明を表示しています。

xberg-io/crawlberg1922026年10月26日 更新

Use when the user wants the list of URLs on a site rather than the page content — sitemap analysis, link planning, or seeding another tool. Covers `crawlberg map URL` with `--limit`, `--search`, robots, output, and how it differs from a full crawl.

日本語の概要は準備中です。原文の説明を表示しています。

xberg-io/crawlberg1922026年10月26日 更新

xberg-io のスキルをすべて見る

このスキルの問題を報告する