Results for “crawl”

Search, scrape, and interact with the web via the Firecrawl CLI. Use this skill whenever the user wants to search the web, find articles, research a topic, look something up online, scrape a webpage, grab content from a URL, get data from a website, crawl documentation, download a site, or interact with pages that need clicks or logins. Also use when they say "fetch this page", "pull the content from", "get the page at https://", or reference external websites. This provides real-time web search with full page content and interact capabilities — beyond what Grok can do natively with built-in tools. Do NOT trigger for local file operations, git commands, deployments, or code editing tasks.

The English summary is being prepared. Showing the original description.

Full-site crawling, scraping, and site mapping via Firecrawl MCP. Use when user says "crawl site", "map site", "full crawl", "find all pages", "broken links", "site structure", "discover pages", "JS rendering", or needs site-wide analysis.

The English summary is being prepared. Showing the original description.

Extract website content into Markdown, HTML, or plain text with Scrapling. Handle JavaScript-rendered pages and anti-bot challenges, or write Python crawlers that follow links, collect structured data, and resume interrupted runs.

• Save article text as Markdown
• Extract content loaded by JavaScript
• Retrieve pages with anti-bot protections

Audit XML sitemaps by checking their structure and listed URLs against actual page conditions. Find errors, redirects, pages excluded from indexing, missing eligible pages, and inaccurate modification dates to identify sitemap problems affecting indexing.

• Investigating sitemap errors in Search Console
• Finding URLs that should be removed from a sitemap
• Checking for indexable pages missing from a sitemap

Build a complete web scraping Actor with Crawlee and deploy to Apify. Use when you need end-to-end web scraping on Apify: defining an input schema, building a router-based Crawlee crawler, extracting structured data, storing results in a dataset, testing locally, and deploying the Actor to the platform. Trigger with "apify scrape website", "build apify actor", "crawlee scraper", "apify main workflow".

The English summary is being prepared. Showing the original description.

Bulk-extract many pages from one site or section. Use for "crawl", "everything under /docs", or content spanning linked pages.

The English summary is being prepared. Showing the original description.

Crawl websites and extract content from multiple pages via the Tavily CLI. Use this skill when the user wants to crawl a site, download documentation, extract an entire docs section, bulk-extract pages, save a site as local markdown files, or says "crawl", "get all the pages", "download the docs", "extract everything under /docs", "bulk extract", or needs content from many pages on the same domain. Supports depth/breadth control, path filtering, semantic instructions, and saving each page as a local markdown file.

The English summary is being prepared. Showing the original description.

Search the web, read source pages, and retrieve structured records with Firecrawl. Use for current information, finding sources, reading a supplied URL, or looking up data-provider capabilities.

The English summary is being prepared. Showing the original description.

Export documentation portals, help centers and API reference sites into local markdown or JSON with Firecrawl browser, including pages that render with JavaScript, hide content behind collapsible navigation or paginate. Use when the user wants a docs site, help center or support knowledge base saved as files, even without an explicit request to scrape.

The English summary is being prepared. Showing the original description.

Build a knowledge base from web content with Firecrawl. Use for local reference docs, RAG-ready chunks, fine-tuning datasets, documentation mirrors, topic corpora, or LLM-ready markdown organized from web sources.

The English summary is being prepared. Showing the original description.

Walk through a product's key flows with Firecrawl browser and produce a structured UX/product walkthrough. Use for signup, onboarding, pricing, docs, dashboard, product demo prep, UX teardown, and first-run experience analysis.

The English summary is being prepared. Showing the original description.

Test a live website with Firecrawl browser and write up what's broken: links, forms, mobile layout, performance and console errors. Use for pre-launch checks, bug hunts or confirming a deployed site works.

The English summary is being prepared. Showing the original description.

Monitor competitor pricing, features, changelogs, dashboards, and product changes with Firecrawl. Use for recurring competitive intelligence, pricing tier extraction, feature change tracking, or structured competitor alerts.

The English summary is being prepared. Showing the original description.

Extract structured company lists with Firecrawl from any page or site that lists companies: accelerator batches, VC portfolios, member and sponsor lists, marketplaces and app directories, YC, Crunchbase, Product Hunt or G2. Use when the user wants those companies as a CSV, JSON, CRM-ready list or research table, even without an explicit request to scrape.

The English summary is being prepared. Showing the original description.

Researches products across retailers and review sites with Firecrawl and recommends what to buy within the user's budget and preferences, with current prices and sources. Use when the user wants to compare products, find the best option or check reviews before buying, since prices and models change faster than training data. Adds to a cart only when asked and a signed-in browser profile exists.

The English summary is being prepared. Showing the original description.

Run focused Firecrawl workflows for research, documentation lookup, SEO audits, QA, lead discovery and enrichment, business contacts, current jobs, fantasy decisions, brand assets, knowledge bases and website design systems. Use when the user wants a sourced answer or deliverable from a workflow rather than raw extraction or product-code integration.

The English summary is being prepared. Showing the original description.

Research markets and companies with Firecrawl using current, cited figures: market size, growth, trends, financials and earnings. Use when the user asks for market research, an industry overview or a comparison of companies by the numbers, since these figures change faster than training data.

The English summary is being prepared. Showing the original description.

Read metrics off web dashboards with Firecrawl browser: public stats, status and transparency pages as well as analytics tools the user is signed in to. Use when the user wants numbers, KPIs or charts from a dashboard turned into a report, summary or date-range table, even without an explicit request to scrape.

The English summary is being prepared. Showing the original description.

Bulk-extract many pages from one site or section. Use for "crawl", "everything under /docs", or content spanning linked pages.

The English summary is being prepared. Showing the original description.

Find and qualify new prospects with Firecrawl Alexandria people and company data: people by role, seniority, company, industry, size, location or technology, target accounts that match criteria, or the right contacts at companies, returned as a ranked, sourced table or CSV. Also extracts lead lists from a database or directory the user names, and refreshes a prospect list with only new matches. Use when the user wants a lead list, prospects, target accounts or the people to reach out to at companies that fit criteria. Not for enriching people or companies the user already supplied (use firecrawl-lead-research).

The English summary is being prepared. Showing the original description.

Produce an intensive, cited analytical report: executive summary, multi-angle findings, contrarian views, open questions, and full sources. Use only when the user needs rigorous synthesis of a complex topic (scientific, technical, policy, or market-analytical) that cannot be answered with a short search, and wants a formal written report, not a recommendation list. Do not use for product picks, top-N lists, quick lookups, or routine "find out about X" tasks. If the request does not clearly need this kind of report, do not use this skill. Do not use for a literature review over published papers. This skill collects evidence from the open web. A request for the literature on a biomedical, clinical, life-science, or other scientific topic — papers, studies, trials, preprints — belongs to firecrawl-research-papers, which queries Firecrawl's paper index (PubMed, bioRxiv, medRxiv, arXiv) instead of searching websites.

The English summary is being prepared. Showing the original description.

Find and synthesize research papers, whitepapers, PDFs, technical reports, and academic sources with Firecrawl Research, using semantic paper search, related-paper expansion, and in-body verification over Firecrawl's paper index — largely biomedical and life-science literature from PubMed, bioRxiv, and medRxiv, plus arXiv preprints in CS, physics, and math. Use when the user wants a literature review, systematic review, survey of studies, paper summary, research landscape, or sourced synthesis from scholarly and industry publications, including clinical, drug, gene, disease, epidemiology, and public-health topics. Prefer this over a general web-research workflow whenever the evidence base is published papers rather than web pages.

The English summary is being prepared. Showing the original description.

Smart agentic web data extraction with multi-strategy scraping (Crawl4AI v4, Firecrawl), LLM extraction loops, anti-bot bypass, and structured export / Ekstraksi data web cerdas dan agentic dengan scraping multi-strategi (Crawl4AI v4, Firecrawl), ekstraksi LLM, bypass anti-bot, dan ekspor terstruktur.

The English summary is being prepared. Showing the original description.

Web scraping, search, crawling, and browser automation via the Firecrawl CLI. Use this skill whenever the user wants to search the web, find articles, research a topic, look something up online, scrape a webpage, grab content from a URL, extract data from a website, crawl documentation, download a site, or interact with pages that need clicks or logins. Also use when they say "fetch this page", "pull the content from", "get the page at https://", or reference scraping external websites. This provides real-time web search with full page content extraction and cloud browser automation — capabilities beyond what Claude can do natively with built-in tools. Do NOT trigger for local file operations, git commands, deployments, or code editing tasks.

The English summary is being prepared. Showing the original description.