Skip to content
cccskills
FreePublished on GitHub

firecrawl-crawl

Bulk extract content from an entire website or site section. Use this skill when the user wants to crawl a site, extract all pages from a docs section, bulk-scrape multiple pages following links, or says "crawl", "get all the pages", "extract everything under /docs", "bulk extract", or needs content from many pages on the same site. Handles depth limits, path filtering, and concurrent extraction.

The English summary is being prepared. Showing the original description.

See how to install

Files included (1)

  • SKILL.md2.6 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

firecrawl crawl

Bulk extract content from a website. Crawls pages following links up to a depth/limit.

When to use

  • You need content from many pages on a site (e.g., all /docs/)
  • You want to extract an entire site section
  • Step 4 in the workflow escalation pattern: search → scrape → map → crawl → interact

Quick start

# Crawl a docs section
firecrawl crawl "<url>" --include-paths /docs --limit 50 --wait -o .firecrawl/crawl.json

# Full crawl with depth limit
firecrawl crawl "<url>" --max-depth 3 --wait --progress -o .firecrawl/crawl.json

# Check status of a running crawl
firecrawl crawl <job-id>

Options

OptionDescription
--waitWait for crawl to complete before returning
--progressShow progress while waiting
--limit <n>Max pages to crawl
--max-depth <n>Max link depth to follow
--include-paths <paths>Only crawl URLs matching these paths
--exclude-paths <paths>Skip URLs matching these paths
--delay <ms>Delay between requests
--max-concurrency <n>Max parallel crawl workers
--prettyPretty print JSON output
-o, --output <path>Output file path

Tips

  • Always use --wait when you need the results immediately. Without it, crawl returns a job ID for async polling.
  • Use --include-paths to scope the crawl — don't crawl an entire site when you only need one section.
  • Crawl consumes credits per page. Check firecrawl credit-usage before large crawls.

See also

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Creating algorithmic art using p5.js with seeded randomness and interactive parameter exploration. Use this when users request creating art using code, generative art, algorithmic art, flow fields, or particle systems. Create original algorithmic art rather than copying existing artists' work to avoid copyright violations.

The English summary is being prepared. Showing the original description.

Scoheart/agentskills2Updated Aug 4, 2026

蚂蚁银行开户

The English summary is being prepared. Showing the original description.

Scoheart/agentskills2Updated Aug 4, 2026

Applies Anthropic's official brand colors and typography to any sort of artifact that may benefit from having Anthropic's look-and-feel. Use it when brand colors or style guidelines, visual formatting, or company design standards apply.

The English summary is being prepared. Showing the original description.

Scoheart/agentskills2Updated Aug 4, 2026

Automates browser interactions for web testing, form filling, screenshots, and data extraction. Use when the user needs to navigate websites, interact with web pages, fill forms, take screenshots, or extract information from web pages.

The English summary is being prepared. Showing the original description.

Scoheart/agentskills2Updated Aug 4, 2026

Create beautiful visual art in .png and .pdf documents using design philosophy. You should use this skill when the user asks to create a poster, piece of art, design, or other static piece. Create original visual designs, never copying existing artists' work to avoid copyright violations.

The English summary is being prepared. Showing the original description.

Scoheart/agentskills2Updated Aug 4, 2026

仅使用 AWS CLI 操作 AWS。Agent 直接执行 aws 命令(无需脚本)。仅在用户提到 AWS、S3、EC2、Lambda 等时激活。

The English summary is being prepared. Showing the original description.

Scoheart/agentskills2Updated Aug 4, 2026

See all skills by Scoheart

Report a problem with this skill