'scraping' 검색 결과

Browser automation powers web testing, scraping, and AI agent interactions. The difference between a flaky script and a reliable system comes down to understanding selectors, waiting strategies, and anti-detection patterns. This skill covers Playwright (recommended) and Puppeteer, with patterns for testing, scraping, and agentic browser control. Key insight: Playwright won the framework war. Unless you need Puppeteer's stealth ecosystem or are Chrome-only, Playwright is the better choice in 2025. Critical distinction: Testing automation (predictable apps you control) vs scraping/agent automation (unpredictable sites that fight back). Different problems, different solutions. Use when "playwright, puppeteer, browser automation, headless, web scraping, e2e test, end-to-end, selenium, chromium, browser test, page.click, locator, playwright, puppeteer, browser, automation, testing, scraping, headless, e2e, selenium, web-testing" mentioned.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Complete knowledge domain for Firecrawl v2 API - web scraping and crawling that converts websites into LLM-ready markdown or structured data. Use when: scraping websites, crawling entire sites, extracting web content, converting HTML to markdown, building web scrapers, handling dynamic JavaScript content, bypassing anti-bot protection, extracting structured data from web pages, or when encountering "content not loading", "JavaScript rendering issues", or "blocked by bot detection". Keywords: firecrawl, firecrawl api, web scraping, web crawler, scrape website, crawl website, extract content, html to markdown, site crawler, content extraction, web automation, firecrawl-py, firecrawl-js, llm ready data, structured data extraction, bot bypass, javascript rendering, scraping api, crawling api, map urls, batch scraping

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Collect posts from X using keywords, user handles, or direct URLs. Get structured data containing post text, author details, engagement counts, and media information to support topic research and comparisons across posts.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Collecting posts about a keyword or hashtag
• Comparing posts from specific accounts
• Gathering posts within a date or language filter

Escalate blocked scraping campaigns with provider-neutral proxy rotation — decide between datacenter, residential, and mobile pools, integrate rotation with scrapling, configure session stickiness for stateful flows, monitor cost and health, and stay inside legal and ethical boundaries. Use as the next step after `headless-web-scraping` client-side stealth (StealthyFetcher, rate limiting, robots.txt) is insufficient and traffic is legitimate.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Extract web page content as Markdown from a single URL or a site crawl. When access is blocked, progressively switch from basic fetching to Chrome and Bright Data proxy services, which route requests through other IP addresses.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Converting a single web page to Markdown
• Extracting content from JavaScript-rendered pages
• Retrieving pages blocked by bot detection

Web scraping with anti-bot bypass, content extraction, undocumented APIs and poison pill detection. Use when extracting content from websites, handling paywalls, implementing scraping cascades or processing social media. Covers requests, trafilatura, Playwright with stealth mode, yt-dlp and instaloader patterns.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Deploy Apify Actors and integrate scraping into external applications. Use when deploying an Actor to the platform, integrating Actor results into a web app (Next.js, Express), wiring webhooks, or scheduling scraping pipelines. Trigger with "deploy apify actor", "apify Vercel integration", "apify production deploy", "integrate apify results", "apify API endpoint".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Production-grade architecture patterns for Apify-powered applications. Use when designing scraping infrastructure, building multi-Actor pipelines, or integrating Apify into a larger system architecture. Trigger with "apify architecture", "apify best practices", "apify project structure", "scraping architecture", "apify system design".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Build a complete web scraping Actor with Crawlee and deploy to Apify. Use when you need end-to-end web scraping on Apify: defining an input schema, building a router-based Crawlee crawler, extracting structured data, storing results in a dataset, testing locally, and deploying the Actor to the platform. Trigger with "apify scrape website", "build apify actor", "crawlee scraper", "apify main workflow".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Assesses lawful basis for AI training data processing per EDPB April 2025 report on LLMs and general-purpose AI. Covers legitimate interest balancing tests, consent challenges for ML training, public dataset assessment, and web scraping lawfulness. Keywords: AI training data, lawful basis, EDPB LLM, legitimate interest, consent, web scraping.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Build reliable web scrapers and crawlers with Crawlee — Apify's open-source framework for structured web scraping. Use when someone asks to "scrape a website", "build a crawler", "Crawlee", "web scraping at scale", "scrape JavaScript-rendered pages", "crawl with Playwright/Puppeteer", or "extract data from websites reliably". Covers HTTP crawling, browser crawling, request queues, proxy rotation, and data export.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Bright Data Web MCP gives MCP-compatible agents live web search, scraping, and optional browser-automation access through Bright Data’s web-access platform. It is built for agents that need current information, anti-bot resilience, and structured extraction without maintaining their own scraping stack.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Stealth headless browser with CDP server for anti-fingerprint browsing, parallel scraping, and low-memory CDP automation. This skill should be used when stealth browsing, anti-detection scraping, headless CDP automation with fingerprint protection, or tracker-free page fetching is needed. Triggers on 'stealth browse', 'anti-fingerprint', 'obscura', 'stealth CDP', 'undetectable scraping', 'headless stealth', 'tracker-free browsing'.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

CLI tool for AI-powered web scraping, data extraction, search, and crawling via ScrapeGraph AI. Use when the user needs to scrape websites, extract structured data from URLs, convert pages to markdown, crawl multi-page sites, search the web for information, automate browser interactions (login, click, fill forms), get raw HTML, discover sitemaps, or generate JSON schemas. Triggers on tasks involving: (1) extracting data from websites, (2) web scraping or crawling, (3) converting webpages to markdown, (4) AI-powered web search with extraction, (5) browser automation, (6) generating output schemas for scraping. The CLI is just-scrape (npm package just-scrape).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Firecrawl의 `/interact`를 앱에 통합하여 클릭, 양식 입력, 페이지 이동이 필요한 웹 콘텐츠를 가져오는 기능 구현을 돕습니다.

• 클릭 후 나타나는 콘텐츠 가져오기
• 검색 양식과 필터 조작하기
• 여러 페이지에 나뉜 결과 가져오기

앱이나 에이전트에 Firecrawl의 웹 검색, 페이지 내용 수집, 구조화된 정보 추출, 브라우저 조작 기능을 추가하도록 선택부터 설정, 구현, 동작 확인까지 안내합니다.

• 앱에 웹 검색 추가
• 알고 있는 URL에서 구조화된 데이터 추출
• 클릭과 양식 입력을 처리 흐름에 추가

Automate forms, clicks, screenshots, and website checks in a browser where you are already signed in. A separately launched browser also supports scraping and network capture, while login or verification steps can be handed to a person.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Fill forms on websites where you are already signed in
• Capture screenshots and check website behavior
• Record network traffic in a separately launched browser

Build and debug Apify Actors, cloud programs for web scraping, automation, and data processing. Get guidance on language templates, input and output schemas, runtime code, local testing, and deployment, with practices for handling credentials and external data.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Creating an Actor for web scraping
• Defining Actor inputs and structured outputs
• Debugging an Actor with local test data

Collect product details, prices, stock information, and customer reviews from supported online stores using Apify. Compare competitors, examine recurring quality complaints, discover sellers through Google Shopping, and export results as CSV or JSON.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Monitor prices and compare competing products
• Analyze reviews for recurring quality issues
• Find sellers across stores through Google Shopping

Collect an X conversation from its root post ID, including replies and nested replies visible to the logged-in account. Return structured text, author details, engagement counts, and reply relationships for reviewing discussions or preparing further analysis.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Collect replies to a widely discussed post
• Review question-and-answer discussions
• Reconstruct reply relationships in long conversations

Collect product details from public shopping pages using a URL, keyword, or product identifier. Extract prices, stock status, images, ratings, and variants into structured data for product research, comparisons, and catalog collection.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing product prices and stock across shopping sites
• Finding product details by ASIN or barcode
• Collecting images and descriptions for a product catalog

Implement Apify webhooks for Actor run notifications and event-driven pipelines. Use when setting up run completion alerts, building event-driven scraping pipelines, or configuring ad-hoc webhooks for individual runs. Trigger with "apify webhook", "apify events", "actor run notification", "apify run succeeded webhook", "apify ad-hoc webhook".

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Web data extraction using 55+ Apify Actors for AI-driven scraping. Supports Instagram, Facebook, TikTok, YouTube, Google, and more. Auto-selects best Actor for the task. Structured output in JSON/CSV with rate limiting and ethical scraping guidelines.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Compatibility alias for the original ScrapeCreators social scraping skill. Use scrapecreators-api for current endpoint guidance and the task-specific Brand Growth research skills for finished analysis.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.