E-commerce Data Extraction

Extract product data, prices, reviews, and seller information from any e-commerce platform using Apify's E-commerce Scraping Tool.

Prerequisites

• .env file with APIFY_TOKEN (at ~/.claude/.env)
• Node.js 20.6+ (for native --env-file support)

Workflow Selection

User Need	Workflow	Best For
Track prices, compare products	Workflow 1: Products & Pricing	Price monitoring, MAP compliance, competitor analysis. Add AI summary for insights.
Analyze reviews (sentiment or quality)	Workflow 2: Reviews	Brand perception, customer sentiment, quality issues, defect patterns
Find sellers across stores	Workflow 3: Sellers	Unauthorized resellers, vendor discovery via Google Shopping

Progress Tracking

Task Progress:
- [ ] Step 1: Select workflow and determine data source
- [ ] Step 2: Configure Actor input
- [ ] Step 3: Ask user preferences (format, filename)
- [ ] Step 4: Run the extraction script
- [ ] Step 5: Summarize results

---

Workflow 1: Products & Pricing

Use case: Extract product data, prices, and stock status. Track competitor prices, detect MAP violations, benchmark products, or research markets.

Best for: Pricing analysts, product managers, market researchers.

Input Options

Input Type	Field	Description
Product URLs	detailsUrls	Direct URLs to product pages (use object format)
Category URLs	listingUrls	URLs to category/search result pages
Keyword Search	keyword + marketplaces	Search term across selected marketplaces

Example - Product URLs

{
  "detailsUrls": [
    {"url": "https://www.amazon.com/dp/B09V3KXJPB"},
    {"url": "https://www.walmart.com/ip/123456789"}
  ],
  "additionalProperties": true
}

Example - Keyword Search

{
  "keyword": "Samsung Galaxy S24",
  "marketplaces": ["www.amazon.com", "www.walmart.com"],
  "additionalProperties": true,
  "maxProductResults": 50
}

Optional: AI Summary

Add these fields to get AI-generated insights:

Field	Description
fieldsToAnalyze	Data points to analyze: ["name", "offers", "brand", "description"]
customPrompt	Custom analysis instructions

Example with AI summary:

{
  "keyword": "robot vacuum",
  "marketplaces": ["www.amazon.com"],
  "maxProductResults": 50,
  "additionalProperties": true,
  "fieldsToAnalyze": ["name", "offers", "brand"],
  "customPrompt": "Summarize price range and identify top brands"
}

Output Fields

• name - Product name
• url - Product URL
• offers.price - Current price
• offers.priceCurrency - Currency code (may vary by seller region)
• brand.slogan - Brand name (nested in object)
• image - Product image URL
• Additional seller/stock info when additionalProperties: true

Note: Currency may vary in results even for US searches, as prices reflect different seller regions.

---

Workflow 2: Customer Reviews

Use case: Extract reviews for sentiment analysis, brand perception monitoring, or quality issue detection.

Best for: Brand managers, customer experience teams, QA teams, product managers.

Input Options

Input Type	Field	Description
Product URLs	reviewListingUrls	Product pages to extract reviews from
Keyword Search	keywordReviews + marketplacesReviews	Search for product reviews by keyword

Example - Extract Reviews from Product

{
  "reviewListingUrls": [
    {"url": "https://www.amazon.com/dp/B09V3KXJPB"}
  ],
  "sortReview": "Most recent",
  "additionalReviewProperties": true,
  "maxReviewResults": 500
}

Example - Keyword Search

{
  "keywordReviews": "wireless earbuds",
  "marketplacesReviews": ["www.amazon.com"],
  "sortReview": "Most recent",
  "additionalReviewProperties": true,
  "maxReviewResults": 200
}

Sort Options

• Most recent - Latest reviews first (recommended)
• Most relevant - Platform default relevance
• Most helpful - Highest voted reviews
• Highest rated - 5-star reviews first
• Lowest rated - 1-star reviews first

Note: The sortReview: "Lowest rated" option may not work consistently across all marketplaces. For quality analysis, collect a large sample and filter by rating in post-processing.

Quality Analysis Tips

• Set high maxReviewResults for statistical significance
• Look for recurring keywords: "broke", "defect", "quality", "returned"
• Filter results by rating if sorting doesn't work as expected
• Cross-reference with competitor products for benchmarking

---

Workflow 3: Seller Intelligence

Use case: Find sellers across stores, discover unauthorized resellers, evaluate vendor options.

Best for: Brand protection teams, procurement, supply chain managers.

Note: This workflow uses Google Shopping to find sellers across stores. Direct seller profile URLs are not reliably supported.

Input Configuration

{
  "googleShoppingSearchKeyword": "Nike Air Max 90",
  "scrapeSellersFromGoogleShopping": true,
  "countryCode": "us",
  "maxGoogleShoppingSellersPerProduct": 20,
  "maxGoogleShoppingResults": 100
}

Options

Field	Description
googleShoppingSearchKeyword	Product name to search
scrapeSellersFromGoogleShopping	Set to true to extract sellers
scrapeProductsFromGoogleShopping	Set to true to also extract product details
countryCode	Target country (e.g., us, uk, de)
maxGoogleShoppingSellersPerProduct	Max sellers per product
maxGoogleShoppingResults	Total result limit

---

Supported Marketplaces

Amazon (20+ regions)

www.amazon.com, www.amazon.co.uk, www.amazon.de, www.amazon.fr, www.amazon.it, www.amazon.es, www.amazon.ca, www.amazon.com.au, www.amazon.co.jp, www.amazon.in, www.amazon.com.br, www.amazon.com.mx, www.amazon.nl, www.amazon.pl, www.amazon.se, www.amazon.ae, www.amazon.sa, www.amazon.sg, www.amazon.com.tr, www.amazon.eg

Major US Retailers

www.walmart.com, www.costco.com, www.costco.ca, www.homedepot.com

European Retailers

allegro.pl, allegro.cz, allegro.sk, www.alza.cz, www.alza.sk, www.alza.de, www.alza.at, www.alza.hu, www.kaufland.de, www.kaufland.pl, www.kaufland.cz, www.kaufland.sk, www.kaufland.at, www.kaufland.fr, www.kaufland.it, www.cdiscount.com

IKEA (40+ country/language combinations)

Supports all major IKEA regional sites with multiple language options.

Google Shopping

Use for seller discovery across multiple stores.

---

Running the Extraction

Step 1: Set Skill Path

SKILL_PATH=~/.claude/skills/apify-ecommerce

Step 2: Run Script

Quick answer (display in chat):

node --env-file=~/.claude/.env $SKILL_PATH/reference/scripts/run_actor.js \
  --actor "apify/e-commerce-scraping-tool" \
  --input 'JSON_INPUT'

CSV export:

node --env-file=~/.claude/.env $SKILL_PATH/reference/scripts/run_actor.js \
  --actor "apify/e-commerce-scraping-tool" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_filename.csv \
  --format csv

JSON export:

node --env-file=~/.claude/.env $SKILL_PATH/reference/scripts/run_actor.js \
  --actor "apify/e-commerce-scraping-tool" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_filename.json \
  --format json

Step 3: Summarize Results

Report:

• Number of items extracted
• File location (if exported)
• Key insights based on workflow:
  • Products: Price range, outliers, MAP violations
  • Reviews: Average rating, sentiment trends, quality issues
  • Sellers: Seller count, unauthorized sellers found

---

Error Handling

Error	Solution
APIFY_TOKEN not found	Ensure ~/.claude/.env contains APIFY_TOKEN=your_token
Actor not found	Verify Actor ID: apify/e-commerce-scraping-tool
Run FAILED	Check Apify console link in error output
Timeout	Reduce maxProductResults or increase --timeout
No results	Verify URLs are valid and accessible
Invalid marketplace	Check marketplace value matches supported list exactly