본문으로 건너뛰기
cccskills
무료GitHub에 공개

web-archive-scraper

Search the Wayback Machine for archived versions of websites. Extract cached pages, customer lists, testimonials, and partner directories from sites that have changed or gone offline. Uses the free CDX API — no API key needed.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

연동 도구X (Twitter)
설치 방법 보기

포함된 파일(3)

  • SKILL.md3.5 KB
  • scripts/search_archive.py9.2 KB
  • skill.meta.json263 B

SKILL.md(원문)

설치하기 전에 에이전트에게 전달되는 지시 내용을 확인할 수 있습니다.

Web Archive Scraper

Search the Wayback Machine (Internet Archive) for archived snapshots of websites. Fetch cached page content to find customer lists, testimonials, partner directories, and other information from sites that have changed or shut down.

Quick Start

Only dependency is requests. No API key needed.

# Find all snapshots of a URL
python3 skills/web-archive-scraper/scripts/search_archive.py \
  --url "https://botkeeper.com/customers"

# Search with date range
python3 skills/web-archive-scraper/scripts/search_archive.py \
  --url "https://botkeeper.com" --from 2025-01-01 --to 2026-02-01

# Search all pages under a domain (prefix match)
python3 skills/web-archive-scraper/scripts/search_archive.py \
  --url "https://botkeeper.com" --match prefix --limit 50

# Fetch the actual archived page content
python3 skills/web-archive-scraper/scripts/search_archive.py \
  --url "https://botkeeper.com/customers" --fetch

# Output formats
python3 skills/web-archive-scraper/scripts/search_archive.py --url URL --output json
python3 skills/web-archive-scraper/scripts/search_archive.py --url URL --output csv
python3 skills/web-archive-scraper/scripts/search_archive.py --url URL --output summary

How It Works

  1. CDX API search — Queries web.archive.org/cdx/search/cdx for snapshots matching the URL
  2. Filtering — Filters by date range, HTTP status code, and MIME type
  3. Dedup — Collapses to one snapshot per day by default to avoid redundant results
  4. Content fetch — Optionally fetches the raw archived HTML (using id_ modifier to skip Wayback toolbar)
  5. Text extraction — Strips HTML tags for readable text output when fetching content

CLI Reference

FlagDefaultDescription
--urlrequiredTarget URL to search in the archive
--matchexactMatch type: exact, prefix, host, domain
--fromnoneStart date (YYYY-MM-DD)
--tononeEnd date (YYYY-MM-DD)
--limit25Max number of snapshots to return
--fetchfalseFetch and display the content of the most recent snapshot
--fetch-allfalseFetch content of ALL matched snapshots (use with small --limit)
--status200HTTP status filter (set to "any" to include all)
--outputjsonOutput format: json, csv, summary
--collapsedayDedup level: none, day, month, year

Output Schema

{
  "url": "https://botkeeper.com/customers",
  "timestamp": "20250915143022",
  "datetime": "2025-09-15T14:30:22",
  "status_code": "200",
  "mime_type": "text/html",
  "archive_url": "https://web.archive.org/web/20250915143022/https://botkeeper.com/customers",
  "raw_url": "https://web.archive.org/web/20250915143022id_/https://botkeeper.com/customers",
  "content": "..."
}

The content field is only populated when --fetch or --fetch-all is used.

Cost

Free. The Wayback Machine CDX API requires no authentication or API key. Rate limit is ~15 requests/minute.

Common Use Cases

  • Find customer lists from shut-down companies (e.g., botkeeper.com)
  • Recover testimonials/case studies before a site redesign
  • Track how a competitor's messaging changed over time
  • Find partner directories that have been removed

리뷰

아직 리뷰가 없습니다. 사용해 본 소감을 남겨 주세요.

댓글

로그인 없이 쓸 수 있습니다. 사용 후기, 질문, 작동하지 않은 환경 등을 편하게 남겨 주세요.

아직 댓글이 없습니다. 첫 댓글을 남겨 보세요.

보내면 바로 공개됩니다. 이메일 주소나 전화번호 같은 개인 정보는 쓰지 마세요. URL은 2개까지입니다.

0 / 1,000자

이 양식은 reCAPTCHA로 보호되며 Google 개인정보처리방침 및 서비스 약관이 적용됩니다.

같은 저장소의 스킬

Research supported advertising promises from a specific buyer's context, current product facts, customer language and observed ads. Turn them into static, video or copy ideas; match video ideas to actual recipes and save a product-scoped angle bank for write-video-ad-script. Product-led discovery can use verified facts without reviews. Reference persistence and reach are observations, not proof of performance.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

gooseworks-ai/goose-skills1.3천2026년 10월 23일 업데이트

Analyze ad campaign performance data (Google, Meta, LinkedIn) to identify what's working, what's wasting budget, and specific cut/scale/test recommendations. Runs statistical analysis, funnel diagnostics, and multi-channel budget reallocation with specific dollar-amount shift recommendations and scenario modeling.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

gooseworks-ai/goose-skills1.3천2026년 10월 23일 업데이트

For paid lead-gen and participant-recruitment ads, replaces vanity CPA with true CAC per qualified lead by joining ad-platform data with downstream funnel events, surfaces tracking gaps, and classifies every creative into Scale / Keep / Investigate / Cut.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

gooseworks-ai/goose-skills1.3천2026년 10월 23일 업데이트

The intake stage of a Meta ad harness. Researches the brand first, states what it assumes, and asks only what it cannot infer and what would change money, claims or legal exposure if guessed wrong — at most four questions a turn, in plain business language. Writes the brand's stable facts (ads/brand.md), the campaign brief, and a log of what was asked vs assumed. Use it when someone wants to run Meta ads and the brand's ad facts are not yet recorded.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

gooseworks-ai/goose-skills1.3천2026년 10월 23일 업데이트

Checks paid ads against the ad platform's rules (Meta, TikTok, Google, LinkedIn and others) and the brand's category rules: the copy, on-image or on-screen text, pictures and people. Rewrites what can be fixed honestly, adds the disclosures that are needed, and flags anything that needs a human or legal review, with one pass, fixed or flag line per ad and platform. Use alone when you already have the ad text or images and know where they will run; otherwise run the playbook for the job (e.g. make-static-ads, make-content, email-lifecycle).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

gooseworks-ai/goose-skills1.3천2026년 10월 23일 업데이트

Analyze the message match between your ads and landing pages. Checks if the promise in the ad copy carries through to the landing page headline, body, and CTA. Flags disconnects that kill conversion rates. Works with Google, Meta, and LinkedIn ads.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

gooseworks-ai/goose-skills1.3천2026년 10월 23일 업데이트

gooseworks-ai의 스킬 모두 보기

이 스킬의 문제 신고