Skip to content
cccskills
FreePublished on GitHub

web-scraper

Scrape web pages, crawl sites, extract structured data, and capture screenshots through SandBase. Use when asked to extract content from URLs, crawl websites, get structured data from pages, take screenshots, or convert web pages to markdown.

The English summary is being prepared. Showing the original description.

See how to install

Files included (2)

  • SKILL.md2.9 KB
  • references/sandbase-api-map.md1.8 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Web Scraper

Universal web scraping and data extraction through SandBase. Scrape individual pages, crawl entire sites, extract structured data, and capture visual snapshots. Read the API map before selecting a capability.

Call SandBase capabilities

Use the capability identifiers below as discovery hints, not MCP tool names. Find the matching endpoint with sandbase_discover(q: "<provider and capability>"); use its returned name in sandbase_inspect(name: "<returned name>"). Read inputSchema, pricing, and execute_as, then call sandbase_run using execute_as.arguments.name and schema-defined arguments. If a run_id is returned, poll sandbase_run_get(run_id: "<returned run_id>") within the task budget until completed or failed; report pending or failed runs without resubmitting them automatically.

Operating principles

  • Respect robots.txt and rate limits — these tools handle that automatically.
  • Use the lightest tool for the job: markdown scrape before full crawl.
  • Structured data extraction is best for product pages, directories, and lists.
  • Screenshots are useful for visual verification and design research.

Workflow

1. Single page extraction

Use context_dev_scrape_markdown for clean Markdown output (best for articles, docs). Use context_dev_scrape_html for raw HTML (best for structure analysis). Use firecrawl_scrape for advanced single-page extraction with options.

2. Site crawling

Use firecrawl_crawl to crawl multiple pages following links. Use firecrawl_map to discover all URLs on a site. Use context_dev_crawl_site for site-wide content extraction. Use context_dev_crawl_sitemap for sitemap-based crawling.

3. Structured data extraction

Use context_dev_extract_structured_data for custom schema extraction from pages. Use context_dev_extract_product for single product data. Use context_dev_extract_products for product listing pages.

4. Visual and metadata

Use context_dev_capture_screenshot for visual page snapshots. Use context_dev_scrape_images for image extraction. Use context_dev_scrape_fonts for typography analysis. Use context_dev_retrieve_brand for brand asset extraction.

Output

Return: extracted content in requested format, structured data as JSON, screenshot URLs, or crawl results with page list.

Example tasks

  • "Extract the content from this article as clean Markdown: [URL]."
  • "Crawl [website] and list all their product pages."
  • "Extract product name, price, and description from this page: [URL]."
  • "Take a screenshot of [URL] at desktop resolution."
  • "Map all pages on [website] and categorize by content type."

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Search academic papers, scholarly articles, and research publications through SandBase. Use when asked for literature review, academic citations, scholarly research, paper discovery, or scientific evidence gathering.

The English summary is being prepared. Showing the original description.

sandbaseai/sandbase-skills202Updated Sep 26, 2026

Research emerging academic trends by combining scholarly databases, web sources, and news to identify rising research areas, breakthrough papers, and shifting scientific consensus. Ideal for R&D teams tracking frontier science.

The English summary is being prepared. Showing the original description.

sandbaseai/sandbase-skills202Updated Sep 26, 2026

Create, implement, or review accessibility review work with format, accessibility, performance, and visual-verification controls. Use when the requested deliverable depends on accessibility review.

The English summary is being prepared. Showing the original description.

sandbaseai/sandbase-skills202Updated Sep 26, 2026

Plan, implement, or review agentic eval work in an existing codebase with compatibility, security, and verification controls. Use when the user explicitly requests agentic eval work.

The English summary is being prepared. Showing the original description.

sandbaseai/sandbase-skills202Updated Sep 26, 2026

ai-seo

Free

Create or review ai seo deliverables with audience, evidence, voice, and conversion-quality controls. Use when the user explicitly asks for ai seo work.

The English summary is being prepared. Showing the original description.

sandbaseai/sandbase-skills202Updated Sep 26, 2026

Apply airflow dag patterns in reproducible local data workflows with version-aware APIs, explicit assumptions, and validation. Use when the user chooses airflow dag patterns or its strengths fit the task.

The English summary is being prepared. Showing the original description.

sandbaseai/sandbase-skills202Updated Sep 26, 2026

See all skills by sandbaseai

Report a problem with this skill