「web scraping」の検索結果

Scrape web pages using Scrapling with anti-bot bypass (like Cloudflare Turnstile), stealth headless browsing, spiders framework, adaptive scraping, and JavaScript rendering. This skill should be used when asked to scrape, crawl, or extract data from websites; web_fetch fails; the site has anti-bot protections; write Python code to scrape/crawl; write spiders.

日本語の概要は準備中です。原文の説明を表示しています。

Use when scraping JavaScript-heavy pages or SPAs, crawling multiple URLs concurrently, extracting structured data with reusable CSS/JSON schemas, or building automated web data pipelines. Wraps the Crawl4AI library (`crwl` CLI and Python SDK) with schema-generation patterns for LLM-free extraction. Triggers on crawl4ai, crwl, scrape JS-heavy site, scrape SPA, headless browser scrape, schema-based extraction, batch crawl, sitemap crawl, web data pipeline. SKIP when a static HTML page can be read with `defuddle` / `fetch-web` — those are faster cold-start and don't need a browser.

日本語の概要は準備中です。原文の説明を表示しています。

Scrape web pages to clean markdown using Firecrawl v2 — handles JS-heavy pages, site crawls, URL mapping, document parsing (PDF/DOCX/XLSX), LLM-powered extraction, autonomous agent scraping, and post-scrape browser interaction (Interact API). Free keyless mode available for search/scrape/parse/interact (no API key needed). Prefer over WebFetch for quality and completeness. Triggers on scrape URL, fetch page, crawl site, extract content, parse document, web to markdown, DeepWiki, Firecrawl.

日本語の概要は準備中です。原文の説明を表示しています。

Cloudflare platform management via Wrangler CLI, Agents SDK (v0.13+), and Browser Rendering API. Deploy Pages, Workers, KV, R2, D1, Queues, Vectorize, Workflows, Hyperdrive. Build stateful AI agents with the full SDK stack — Agent, AIChatAgent, Think, Voice, sub-agents, fibers, HITL, scheduling, observability, Code Mode (Dynamic Worker sandboxes). Budget web scraping via cf_browser.py. Use when deploying to Cloudflare, managing CF infrastructure, configuring wrangler.toml/wrangler.jsonc, building AI agents or chat agents on Workers, adding voice, orchestrating sub-agents, or for cheap web scraping. Triggers: Cloudflare, wrangler, Pages, KV, R2, D1, Workers, Vectorize, Queues, Workflows, Hyperdrive, cf_browser, Browser Rendering, Agents SDK, Code Mode, codemode, MCP, Think, AIChatAgent, Voice agent, sub-agents, fibers, HITL, Dynamic Workers.

日本語の概要は準備中です。原文の説明を表示しています。

Direct browser control via CDP — use for any task involving a website or web app: browsing, scraping and data extraction, filling forms, testing sites, taking screenshots, automating web workflows.

日本語の概要は準備中です。原文の説明を表示しています。

Browser automation for AI agents via inference.sh. Navigate web pages, interact with elements using @e refs, take screenshots, record video. Capabilities: web scraping, form filling, clicking, typing, drag-drop, file upload, JavaScript execution. Use for: E2E testing, web research, automat...

日本語の概要は準備中です。原文の説明を表示しています。

Computer Use Agent — autonomous browser control via Steel, Browser Use, and Browserbase cloud providers. Supports headful CDP sessions, CAPTCHA solving, proxy rotation, live view, session recording/replay, Playwright/Puppeteer integration, and LLM-driven visual browsing. Use when asked about computer use, browser automation, web scraping, Steel sessions, cloud browsers, headful browsing, CAPTCHA solving, or autonomous web navigation.

日本語の概要は準備中です。原文の説明を表示しています。

Scrape web pages using Scrapling with anti-bot bypass (like Cloudflare Turnstile), stealth headless browsing, spiders framework, adaptive scraping, and JavaScript rendering. Use when asked to scrape, crawl, or extract data from websites; web_fetch fails; the site has anti-bot protections; write Python code to scrape/crawl; or write spiders.

日本語の概要は準備中です。原文の説明を表示しています。

Non-testing browser automation - web scraping, form filling, screenshot capture, PDF generation, workflow automation. For TESTING with Playwright, use e2e-playwright skill instead. Activates for web scraping, form automation, screenshot, PDF, headless browser, Puppeteer, Selenium, automation scripts, data extraction.

日本語の概要は準備中です。原文の説明を表示しています。

Use this skill when the user asks to test, verify, interact with, or automate web pages and browsers. Trigger for requests involving Chrome automation, browser testing, web scraping, screenshot capture, element selection, or checking web applications. Also trigger when user mentions "browser tools" explicitly.

日本語の概要は準備中です。原文の説明を表示しています。

Web scraping with Scrapling - HTTP fetching, stealth browser automation, Cloudflare bypass, and spider crawling via CLI and Python.

日本語の概要は準備中です。原文の説明を表示しています。

Use this skill when a user needs a screenshot, website thumbnail, full-page capture, or PDF of a public HTTP(S) webpage saved as a local artifact through Latchshot, including report, QA, archive, and social-preview workflows. Do not use it for private or authenticated pages, raw HTML, scraping or extraction, arbitrary browser actions, CAPTCHA or anti-bot bypass, or local-file capture.

日本語の概要は準備中です。原文の説明を表示しています。

Scrapes social platforms, business data, and e-commerce via Apify actors — Instagram, LinkedIn, TikTok, YouTube, Facebook, Google Maps, Amazon, and web crawls — filtering in code. USE WHEN scrape Instagram, scrape LinkedIn, scrape TikTok, scrape YouTube, scrape Facebook, Google Maps leads, Amazon reviews, business intelligence, multi-platform social listening, competitive analysis, lead generation, social monitoring, Apify actors, web crawl, extract contacts. NOT FOR X/Twitter account operations like posting, threads, or bookmarks (those need a dedicated X API client), 4-tier progressive scraping with proxy escalation (use BrightData), or real-Chrome bot bypass and computer use (use Interceptor).

日本語の概要は準備中です。原文の説明を表示しています。

Real Chrome/Brave + macOS Computer Use from inside the browser — zero CDP fingerprint, real sessions; mandatory for visual deploy verification. Drive clickwork yourself. USE WHEN verify deploy, confirm UI, screenshot verification, computer use, macos automation, debug web, troubleshoot, visual check, motion/animation bug, jank, transition stutter, scrub a flow, console logs/errors, runtime/JS/react errors, mismatch warning, network traffic/log, HAR/pcapng export, hydration/blank-page debug, flash then blank/page broken, why is this not working/what's happening on the page, authenticated page, bot detection bypass, reproduce bug, drive native app, about to ask the operator to click/navigate/fill a form/log in/approve OAuth, OAuth consent flow, complete a web login, do this in your browser. NOT FOR residential-proxy crawling (BrightData) or social actor scraping (Apify).

日本語の概要は準備中です。原文の説明を表示しています。

Playwright-based browser automation patterns for autonomous web interaction

日本語の概要は準備中です。原文の説明を表示しています。

Turns one-shot external recon into a continuous monitoring program. Covers the scheduled re-scan-and-diff loop (baseline snapshot -> interval sleep -> re-scan -> asset/finding delta -> threshold-gated webhook alert), the scan-to-scan diff engine (new/removed/changed assets by a tracked-attribute table, new/resolved findings by a stable cross-scan fingerprint), adversary CTI / chatter monitoring across six public feeds (ransomwatch, ransomware.live, HackerNews Algolia search, Reddit security-subreddit RSS, GitHub Gist code-search, public Telegram channel scraping) with a source-kind-aware severity engine (leak-site/forum/telegram/paste tiers, CRITICAL through INFO), literal/glob/regex watchlist pattern matching, full-corpus capture with retroactive rescan on new watchlist entries, infrastructure-tracking-over-time discipline (certificate-transparency, passive-DNS, port/service, and typosquat re-enumeration cadence, and what a genuine 'perimeter drift' event looks like in the diff output), a five-state finding-lifecycle state machine (open/triaged/risk_accepted/resolved/false_positive) with per-severity SLA and fingerprint-based cross-scan dedup and auto-resolve/reopen rules, the alert-fatigue trap where a lifecycle-unaware rule re-fires on an already-accepted finding, a durable retry/backoff alert-outbox pattern ('queued is not delivered'), and copy-paste bash-cron plus PowerShell-Scheduled-Task recipes for a re-scan+diff loop with the Slack-compatible webhook payload shape. Passive OSINT and analysis only -- no new active-intrusion technique. Use when setting up ongoing monitoring for a retainer or MSSP engagement, tuning alert thresholds to avoid fatigue, triaging a finding's lifecycle status, investigating adversary chatter about a brand, building a 'what changed on the perimeter since last week' report, or deciding whether a persisting finding should re-alert.

日本語の概要は準備中です。原文の説明を表示しています。

Web data extraction using 55+ Apify Actors for AI-driven scraping. Supports Instagram, Facebook, TikTok, YouTube, Google, and more. Auto-selects best Actor for the task. Structured output in JSON/CSV with rate limiting and ethical scraping guidelines.

日本語の概要は準備中です。原文の説明を表示しています。

Web scraping with structured data extraction - define your output schema

日本語の概要は準備中です。原文の説明を表示しています。

Extract speaker names, titles, companies, and bios from conference websites. Supports direct HTML scraping and Apify web scraper fallback for JS-heavy sites. Use for pre-event research and outreach targeting.

日本語の概要は準備中です。原文の説明を表示しています。

Parallel web scraping and data collection from multiple sources

日本語の概要は準備中です。原文の説明を表示しています。

Use the ax CLI instead of curl + throwaway parsing scripts whenever you fetch a URL, explore an unknown web page, or extract structured data from HTML. Trigger whenever you are about to write an inline script (python3 heredoc, node -e, regex over HTML) or a bare curl for one-off web fetching, scraping, or page exploration.

日本語の概要は準備中です。原文の説明を表示しています。

Browser Use is an open-source Python library that lets an LLM agent drive a real browser to navigate sites, fill forms, click and extract data from a plain-English task. Use when the user wants an AI agent to automate a website, scrape pages that need clicking or login, return structured data from the web, add browser tools to a coding agent, or run browser agents locally or in the cloud.

日本語の概要は準備中です。原文の説明を表示しています。

AI Overview (AI Mode) scraping via Google Search. Returns AI-summarized key points for a single keyword, ideal for deep research, technical Q&A, long-tail product selection, and cross-border consumer preference analysis using the latest web information. Single-round only; follow-ups require the agent to summarize context and issue a new request. Triggered by: Google AI, AI Overview, AI Mode, Google AI search, AI search, deep research, consumer preference analysis, web summary, long-tail product research, cross-border market insights.

日本語の概要は準備中です。原文の説明を表示しています。

Use when writing asyncio Python code — async/await coroutines, concurrent I/O with asyncio.gather, task creation and cancellation, semaphore rate limiting, producer-consumer queues, async context managers, async generators, WebSocket servers, aiohttp web scraping, async database operations, run_in_executor for blocking calls, or testing async code with pytest-asyncio. Covers FastAPI and aiohttp patterns, synchronization primitives, timeout handling, and common pitfalls like event loop blocking and missing await.

日本語の概要は準備中です。原文の説明を表示しています。