66 个 · 按相关度排序
Build reliable web scrapers and crawlers with Crawlee — Apify's open-source framework for structured web scraping. Use when someone asks to "scrape a website", "build a crawler", "Crawlee", "web scraping at scale", "scrape JavaScript-rendered pages", "crawl with Playwright/Puppeteer", or "extract data from websites reliably". Covers HTTP crawling, browser crawling, request queues, proxy rotation, and data export.
中文简介正在准备中。当前显示原文说明。
Complete knowledge domain for Firecrawl v2 API - web scraping and crawling that converts websites into LLM-ready markdown or structured data. Use when: scraping websites, crawling entire sites, extracting web content, converting HTML to markdown, building web scrapers, handling dynamic JavaScript content, bypassing anti-bot protection, extracting structured data from web pages, or when encountering "content not loading", "JavaScript rendering issues", or "blocked by bot detection". Keywords: firecrawl, firecrawl api, web scraping, web crawler, scrape website, crawl website, extract content, html to markdown, site crawler, content extraction, web automation, firecrawl-py, firecrawl-js, llm ready data, structured data extraction, bot bypass, javascript rendering, scraping api, crawling api, map urls, batch scraping
中文简介正在准备中。当前显示原文说明。
Find and fix internal links that lead to missing pages or server errors. Check rendered pages and HTTP responses, then identify URLs to update or remove and explain how broken links affect search engine crawling.
中文简介正在准备中。当前显示英语简介。
You are an expert in Crawl4AI, the open-source web crawler built for AI applications. You help developers extract clean, structured data from websites for LLM training, RAG pipelines, and content analysis — with automatic markdown conversion, JavaScript rendering, CSS-based extraction, LLM-powered structured extraction, and session management for multi-page crawling.
中文简介正在准备中。当前显示原文说明。
Web search, content extraction, crawling, and deep research via the Tavily CLI. Use this skill whenever the user wants to search the web, find articles, research a topic, look something up online, extract content from a URL, grab text from a webpage, crawl documentation, download a site's pages, discover URLs on a domain, or conduct in-depth research with citations. Also use when they say "fetch this page", "pull the content from", "get the page at https://", "find me articles about", or reference extracting data from external websites. This provides LLM-optimized web search, content extraction, site crawling, URL discovery, and AI-powered deep research — capabilities beyond what agents can do natively. Do NOT trigger for local file operations, git commands, deployments, or code editing tasks.
中文简介正在准备中。当前显示原文说明。
Build or review production-ready Tavily SDK and API integrations for web search, extraction, crawling, mapping, and research. Use when implementing Tavily in an agent, RAG pipeline, or application rather than only running one CLI command.
中文简介正在准备中。当前显示原文说明。
Route Tavily web-search, extraction, mapping, crawling, and cited-research requests to the narrowest `tvly` command. Use for command-line Tavily work, installation checks, authentication setup, or choosing among the specialized Tavily skills.
中文简介正在准备中。当前显示原文说明。
CLI tool for AI-powered web scraping, data extraction, search, and crawling via ScrapeGraph AI. Use when the user needs to scrape websites, extract structured data from URLs, convert pages to markdown, crawl multi-page sites, search the web for information, automate browser interactions (login, click, fill forms), get raw HTML, discover sitemaps, or generate JSON schemas. Triggers on tasks involving: (1) extracting data from websites, (2) web scraping or crawling, (3) converting webpages to markdown, (4) AI-powered web search with extraction, (5) browser automation, (6) generating output schemas for scraping. The CLI is just-scrape (npm package just-scrape).
中文简介正在准备中。当前显示原文说明。
Web scraping with Scrapling - HTTP fetching, stealth browser automation, Cloudflare bypass, and spider crawling via CLI and Python.
中文简介正在准备中。当前显示原文说明。
Web search, content extraction, crawling, and deep research via the Tavily CLI. Use this skill whenever the user wants to search the web, find articles, research a topic, look something up online, extract content from a URL, grab text from a webpage, crawl documentation, download a site's pages, discover URLs on a domain, or conduct in-depth research with citations. Also use when they say "fetch this page", "pull the content from", "get the page at https://", "find me articles about", or reference extracting data from external websites. This provides LLM-optimized web search, content extraction, site crawling, URL discovery, and AI-powered deep research — capabilities beyond what agents can do natively. Do NOT trigger for local file operations, git commands, deployments, or code editing tasks.
中文简介正在准备中。当前显示原文说明。
Review URL parameters used for filters, sorting, pagination, and tracking. Check canonical tags, which identify a preferred page URL, and recommend consistent handling of URL variants to address duplicate content and unnecessary search engine crawling.
中文简介正在准备中。当前显示英语简介。
Scan links within the same domain for the nofollow attribute, a setting used to qualify links for search engines. List affected pages, link text, and destinations, then guide removal or changes based on the intended purpose.
中文简介正在准备中。当前显示英语简介。
Test REST, GraphQL, and gRPC APIs with Strix using schemas, credentials, and example data. Investigate authorization flaws, excessive data exposure, and other API risks, then verify findings with requests that reproduce the issue.
中文简介正在准备中。当前显示英语简介。
Operate Chrome or Brave using signed-in browser sessions to verify deployed pages, reproduce bugs, inspect page structure, and examine errors or network traffic. An optional macOS bridge extends control to native applications and operating system input.
中文简介正在准备中。当前显示英语简介。
Crawl websites through Firecrawl to collect URLs, page content, metadata, and links. Extract content that appears through JavaScript, map site structure, and gather material for broken link checks and broader SEO analysis.
中文简介正在准备中。当前显示英语简介。
Create or review llms.txt, a Markdown guide to a website for AI systems. Discover important pages, write factual summaries, check links and formatting, and produce a ready-to-save file or an improvement report.
中文简介正在准备中。当前显示英语简介。
Walmart category page scraper: input a walmart.com browse or category URL with optional page number, extract paginated product listings with itemId, url, title, brand, image, price, wasPrice, rating, reviewCount, availability, seller info, fulfillmentBadge, and classType. Use when user mentions walmart category, walmart browse page, walmart category listing, scrape walmart category, walmart department scrape, walmart category URL, browse walmart categories, walmart category products, walmart category page scraper, walmart browse scraper, extract walmart category items, walmart filtered category, walmart subcategory products, walmart browse items, walmart department listing, walmart catalog browse, walmart aisle scraper. Also applies to collecting products from a specific walmart category URL with filters applied (e.g. filtered category URLs copied from the browser), scraping all items from a walmart browse section, category-scoped price monitoring on walmart, and parent category crawling where subcategory URLs are enumerated first.
中文简介正在准备中。当前显示原文说明。
Web2 recon pipeline — subdomain enumeration (subfinder, Chaos API, assetfinder), live host discovery (dnsx, httpx), URL crawling (katana, waybackurls, gau), directory fuzzing (ffuf), JS analysis (LinkFinder, SecretFinder), continuous monitoring (new subdomain alerts, JS change detection, GitHub commit watch). Use when starting recon on any web2 target or when asked about asset discovery, subdomain enum, or attack surface mapping.
中文简介正在准备中。当前显示原文说明。
Web2 recon pipeline — subdomain enumeration (subfinder, Chaos API, assetfinder), live host discovery (dnsx, httpx), URL crawling (katana, waybackurls, gau), directory fuzzing (ffuf), JS analysis (LinkFinder, SecretFinder), continuous monitoring (new subdomain alerts, JS change detection, GitHub commit watch). Use when starting recon on any web2 target or when asked about asset discovery, subdomain enum, or attack surface mapping.
中文简介正在准备中。当前显示原文说明。
Web scraping, crawling, and AI-powered answer extraction at scale
中文简介正在准备中。当前显示原文说明。
AI-powered web search, crawling, extraction, and deep research
中文简介正在准备中。当前显示原文说明。
When the user wants to configure, audit, or optimize robots.txt. Also use when the user mentions "robots.txt," "crawler rules," "block crawlers," "AI crawlers," "GPTBot," "allow/disallow," "disallow path," "crawl directives," "user-agent," "block Googlebot," "fix robots.txt," "robots.txt blocking," or "search engine crawling." For indexing, use indexing.
中文简介正在准备中。当前显示原文说明。
Tests a deployed, activated Power Pages site at runtime using browser-based navigation, page crawling, and API request verification via Playwright. Use when the user wants to test, verify, or smoke-test their deployed site.
中文简介正在准备中。当前显示原文说明。
Use and read this skill immediately if the user request is in any way related to SEO or a site's organic search or AI search presence. That includes site audits, rankings, keyword research, competitors, backlinks, click or traffic changes, indexing problems, crawling, redirects, sitemaps, metadata, structured data, Core Web Vitals, internal links, content opportunities, programmatic SEO, local search, Search Console or Google Analytics questions, Google update impact, llms.txt, AI search visibility in ChatGPT, Claude, Perplexity, or Google AI Overviews, and client SEO reporting. Routes to evidence-backed local reports through the SEO CLI and MCP server.
中文简介正在准备中。当前显示原文说明。