511 skills · sorted by relevance
Set up Firecrawl in an application, from browser authentication and API key configuration to choosing an SDK, a library for calling the service. Add the first request and verify that the integration works.
Build web search, page content retrieval, structured data extraction, and browser interactions into applications with Firecrawl. The skill guides endpoint selection, credentials, SDK or REST integration, and verification with a real request.
Build web discovery into applications and agent workflows with Firecrawl search. Turn a query into candidate URLs, select useful sources, and decide whether to retrieve page content for further extraction or answers with cited sources.
Integrate Firecrawl browser actions into application code to retrieve content that requires clicks, form entry, or pagination. Use it for multi-step page interactions and workflows that need to preserve the browser session after scraping.
Integrate single-page extraction into an application using Firecrawl scrape. Retrieve Markdown, HTML, links, screenshots, metadata, or structured data from a known URL, and configure how recently cached content may be reused.
Crawl websites through Firecrawl to collect URLs, page content, metadata, and links. Extract content that appears through JavaScript, map site structure, and gather material for broken link checks and broader SEO analysis.
Crawl websites (news, blogs, papers, GitHub, product docs, RSS feeds) into a fixed-schema JSONL file, then create a dataset and a searchable application in Viking AI Search. Supports one-time crawl and scheduled recurring crawl with automatic incremental sync.
The English summary is being prepared. Showing the original description.
Complete knowledge domain for Firecrawl v2 API - web scraping and crawling that converts websites into LLM-ready markdown or structured data. Use when: scraping websites, crawling entire sites, extracting web content, converting HTML to markdown, building web scrapers, handling dynamic JavaScript content, bypassing anti-bot protection, extracting structured data from web pages, or when encountering "content not loading", "JavaScript rendering issues", or "blocked by bot detection". Keywords: firecrawl, firecrawl api, web scraping, web crawler, scrape website, crawl website, extract content, html to markdown, site crawler, content extraction, web automation, firecrawl-py, firecrawl-js, llm ready data, structured data extraction, bot bypass, javascript rendering, scraping api, crawling api, map urls, batch scraping
The English summary is being prepared. Showing the original description.
Crawl linked pages within a chosen website scope and collect clean text, Markdown, or HTML with page metadata. Use the resulting records as source material for AI search systems or chatbot knowledge bases.
Build reliable web scrapers and crawlers with Crawlee — Apify's open-source framework for structured web scraping. Use when someone asks to "scrape a website", "build a crawler", "Crawlee", "web scraping at scale", "scrape JavaScript-rendered pages", "crawl with Playwright/Puppeteer", or "extract data from websites reliably". Covers HTTP crawling, browser crawling, request queues, proxy rotation, and data export.
The English summary is being prepared. Showing the original description.
When the user wants to improve crawlability, fix orphan pages, or optimize site structure for search engines. Also use when the user mentions "crawlability," "crawl budget," "orphan pages," "internal links," "site structure," "site crawlability," "infinite scroll," "pagination," "masonry SEO," "AI crawler optimization," "GPTBot crawlability," "ClaudeBot crawlability," or "content not indexed." For internal links, use internal-links.
The English summary is being prepared. Showing the original description.
Check whether AI crawlers, the bots that collect web content for AI services, can access a website. Get a crawler access table, an assessment score, and recommendations based on robots.txt, page tags, HTTP headers, and rendering requirements.
Any live-web task via the Firecrawl CLI — including ordinary web research: searching the web, reading or extracting pages, gathering sources, discovering site URLs, bulk extraction, downloading a site, change alerts, or pages needing clicks/login — web only; local files route to firecrawl-parse. For papers use firecrawl-research-index; for library, API, error, or bug questions use firecrawl-developer-index.
The English summary is being prepared. Showing the original description.
Find the papers that answer a research query in Firecrawl's research paper index — a corpus of paper abstracts whose largest share is biomedical and life-science literature (PubMed, bioRxiv, medRxiv), alongside arXiv preprints in CS, physics, and math — using semantic search, semantic and structural expansion, and in-body verification. Use this skill for literature-finding and paper-retrieval tasks of any kind, including clinical, biomedical, drug, gene, disease, and other life-science questions, whether the answer is a single paper or a full multi-paper set. The index is reached only through the `firecrawl_research_*` MCP tools or the `firecrawl research` CLI subcommands. Calling `firecrawl_search` with its `categories` option set to `["research"]` is a different feature — it filters ordinary web search to research-affiliated websites (the list includes PubMed, bioRxiv, medRxiv, arXiv, and publisher sites) and returns page results from them, without querying the paper records in this index.
The English summary is being prepared. Showing the original description.
Use when scraping JavaScript-heavy pages or SPAs, crawling multiple URLs concurrently, extracting structured data with reusable CSS/JSON schemas, or building automated web data pipelines. Wraps the Crawl4AI library (`crwl` CLI and Python SDK) with schema-generation patterns for LLM-free extraction. Triggers on crawl4ai, crwl, scrape JS-heavy site, scrape SPA, headless browser scrape, schema-based extraction, batch crawl, sitemap crawl, web data pipeline. SKIP when a static HTML page can be read with `defuddle` / `fetch-web` — those are faster cold-start and don't need a browser.
The English summary is being prepared. Showing the original description.
Curate Gaia bot crawler branches into real-skill catalog entries and clean up processed bot/* branches. Use when the user says: "process bot branches", "curate the crawl output", "clean up bot proposals", "run bot curation", "merge the crawler results", "review bot/* branches", "ingest crawler data", or explicitly types /gaia-bot-curate. Also triggers when the user asks to review pending `bot/*` branches, turn crawler JSON proposals into registry nodes, decide which crawled skills to accept vs reject, or batch-add skills from an automated crawl run. This skill handles the full pipeline: fetch remote bot branches, triage proposals, run `gaia dev` mutations, validate, open a PR, and delete consumed bot branches.
The English summary is being prepared. Showing the original description.
Use when a request needs external sources, current facts, listings, a supplied URL, papers, or library and API documentation. When using Firecrawl for the request, use firecrawl_search to find sources and firecrawl_scrape to read a supplied or discovered URL. Firecrawl also retrieves documents, collects structured data, and tracks website changes. Respect explicit source and tool preferences.
The English summary is being prepared. Showing the original description.
Scrape and crawl websites with Firecrawl: scrape a page, start a crawl, check crawl status, map a site, search the web. Trigger phrases: firecrawl, scrape website.
The English summary is being prepared. Showing the original description.
Any live-web task via the Firecrawl CLI — including ordinary web research: searching the web, reading or extracting pages, gathering sources, discovering site URLs, bulk extraction, downloading a site, change alerts, or pages needing clicks/login — web only; local files route to firecrawl-parse. For US legal or regulatory questions use gov; for papers use firecrawl-research-index; for library, API, error, or bug questions use firecrawl-developer-index.
The English summary is being prepared. Showing the original description.
Find the papers that answer a research query in Firecrawl's research paper index — a corpus of paper abstracts whose largest share is biomedical and life-science literature (PubMed, bioRxiv, medRxiv), alongside arXiv preprints in CS, physics, and math — using semantic search, semantic and structural expansion, and in-body verification. Use this skill for literature-finding and paper-retrieval tasks of any kind, including clinical, biomedical, drug, gene, disease, and other life-science questions, whether the answer is a single paper or a full multi-paper set. The index is reached only through the `firecrawl_research_*` MCP tools or the `firecrawl research` CLI subcommands. Calling `firecrawl_search` with its `categories` option set to `["research"]` is a different feature — it filters ordinary web search to research-affiliated websites (the list includes PubMed, bioRxiv, medRxiv, arXiv, and publisher sites) and returns page results from them, without querying the paper records in this index.
The English summary is being prepared. Showing the original description.
Fix rsky-pds createRecord/putRecord hanging indefinitely with no error logs. Use when: (1) createSession succeeds but createRecord never returns, (2) No error appears in PDS logs despite request hanging for 60+ seconds, (3) PDS health endpoint works but write operations hang, (4) readiness probes intermittently fail on rsky-pds pod. Root cause: PDS_CRAWLERS env var configures crawler notification URLs that are called via reqwest HTTP client WITHOUT any timeout in crawlers.rs. If the crawler URL is unreachable from the cluster, every write hangs forever. Also: setting PDS_CRAWLERS="" produces [""] (one empty string element) which causes a "builder error" instead — must fully unset the env var.
The English summary is being prepared. Showing the original description.
Find the published business contacts of companies with Firecrawl, whether the user names them or describes them by type and place: press and media, sales, partnerships and support inboxes, shared business phone numbers and official contact forms, taken from the companies' own websites. Use when the user asks how to contact a company or wants published business emails, phone numbers or press, sales, support or partnership contacts, since contact pages change and an address must come from the page, not memory. Not for named people's details (use firecrawl-lead-research) or finding people to reach out to (use firecrawl-lead-gen).
The English summary is being prepared. Showing the original description.
Enrich and research the people and companies the user names or supplies with Firecrawl Alexandria data: background on a person, a professional profile URL to look up, whether someone still holds a role, missing columns in a table or CSV of people or companies, firmographics for company domains (industry, headcount, funding, location), and pre-call or meeting briefs. Use whenever the request is about specific named people or companies, even without the word enrich, since roles, headcount and funding change faster than training data. Not for finding new prospects (use firecrawl-lead-gen) or a company's published press, sales or support contacts (use firecrawl-business-contacts).
The English summary is being prepared. Showing the original description.
Any live-web task via the Firecrawl CLI — including ordinary web research: searching the web, reading or extracting pages, gathering sources, discovering site URLs, bulk extraction, downloading a site, change alerts, or pages needing clicks/login — web only; local files route to firecrawl-parse. For US legal or regulatory questions use gov; for papers use firecrawl-research-index; for library, API, error, or bug questions use firecrawl-developer-index.
The English summary is being prepared. Showing the original description.