Skip to content
cccskills
FreePublished on GitHub

brightdata-core-workflow-b

Run an approved Bright Data dataset job, track its asynchronous snapshot until it reaches a terminal state, and download results within defined size and record limits. The skill also checks the result schema before it is promoted or quarantined.

OriginalShow the original description

Analyze and operate the Bright Data Web Scraper API async snapshot lifecycle with terminal-state and delivery controls. Use when collecting an approved batch, polling a snapshot, or downloading large structured results. Trigger with: "run a Bright Data dataset job", "poll a Bright Data snapshot", "download Web Scraper API results".

jeremylongshore/tons-of-skills-marketplace2,884 stars(GitHub 2,884 + cccskills 0)

Run score Not tested

See how to install

Good for

  • Submitting an approved URL batch for collection
  • Checking progress and status of a dataset snapshot
  • Downloading and validating large structured results

Files included (2)

  • SKILL.md3.9 KB
  • references/official-docs.md802 B

SKILL.md (original)

Review the instructions the agent will receive before you install.

Bright Data Async Snapshot Pipeline

Overview

Implement the documented trigger, progress, and download lifecycle as a state machine. Keep dataset identifiers and approved inputs separate from the API key; cap polling, validate terminal states, and stream results rather than loading arbitrary payloads into memory.

Prerequisites

  • A reviewed dataset ID and approved public input manifest
  • A named-user Bright Data API key in the runtime secret manager
  • A retention, schema, maximum-record, and maximum-byte policy

Instructions

Step 1: Validate the manifest

Read the requested dataset/inputs and Grep for disallowed target classes or fields. Hash the approved input manifest before submission.

Step 2: Trigger once

Use Bash(curl:*) only against the fixed Bright Data API origin with the API key as a Bearer header.

curl --fail-with-body --request POST \
  'https://api.brightdata.com/datasets/v3/trigger?dataset_id=DATASET_ID' \
  --header "Authorization: Bearer $BRIGHTDATA_API_KEY" \
  --header 'Content-Type: application/json' \
  --data-binary @approved-inputs.json

Step 3: Poll deliberately

Persist the returned snapshot_id; poll GET /datasets/v3/progress/SNAPSHOT_ID with bounded attempts and provider-directed delay. Handle ready, failed, empty, expired, and still-building states explicitly.

Step 4: Download and verify

Stream GET /datasets/v3/snapshot/SNAPSHOT_ID in the approved format. Use parts for large results, hold format/compression parameters constant, enforce byte/record ceilings, and validate the schema before promotion.

Tool Discipline

Use Read and Grep for policy and schema checks. Use Write and Edit only for the manifest, state machine, tests, and redacted receipt. Use Bash(curl:*) for fixed-origin Bright Data API calls after authorization; never print the Bearer value or raw result data.

Output

  • Input-manifest hash and snapshot identifier
  • Bounded state-transition log without target data or credentials
  • Schema/size validation and a promoted-or-quarantined result

Examples

Submit a small approved URL batch, record the returned snapshot ID, poll until ready, and stream JSON to quarantine. Reject a response whose schema, record count, or byte count exceeds the manifest even if the provider marks it ready.

Error Handling

FailureMeaningResponse
400 validation responseDataset ID or input shape is invalidCorrect the manifest; do not retry unchanged input
429 or too many jobsTenant or dataset concurrency is exhaustedPause new triggers and wait for owned jobs
Snapshot expired or emptyResult cannot be promotedTrigger a newly approved run or investigate inputs

Resources

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Add a synthetic-data CI gate for customer-owned Abridge integration code. Test consent handling, note-review handoffs, redaction, and rollback behavior with mocked vendor boundaries, without connecting CI to a live Abridge or EHR tenant.

  • Adding regression checks to an Abridge integration repository
  • Testing consent and note-release handling
  • Checking logs for sensitive data exposure
jeremylongshore/tons-of-skills-marketplace2.9KUpdated Oct 29, 2026

Triage Abridge recording, note review, Epic handoff, and access problems using observed symptoms and approved guidance. Identify the affected workflow stage, organize a minimal timeline, and prepare a redacted escalation record for the responsible team.

  • Separate recording problems from upload failures
  • Investigate note creation delays and editor issues
  • Identify the responsible team for Epic handoff failures
jeremylongshore/tons-of-skills-marketplace2.9KUpdated Oct 29, 2026

Map and review an Abridge clinical documentation workflow from consent and recording through draft review, evidence checks, edits, and final disposition. Keep clinician approval explicit, identify failure exits, and separate verified facts from tenant-specific details and pending approvals.

  • Map a clinical note workflow for a care setting
  • Check where consent and patient identity are verified
  • Review draft notes against linked evidence before sign-off
jeremylongshore/tons-of-skills-marketplace2.9KUpdated Oct 29, 2026

Audit how clinician-reviewed Abridge note sections map into Epic templates and downstream workflows. Use approved test records to document destinations, exceptions, evidence, rollback steps, and a go or no-go decision for a specific environment.

  • Checking Abridge note section mappings to Epic templates
  • Auditing a department’s approved documentation handoff
  • Preparing test evidence and a go or no-go decision
jeremylongshore/tons-of-skills-marketplace2.9KUpdated Oct 29, 2026

Review Abridge adoption and workflow value using local usage records, clinician feedback, and verified contract costs. Build an evidence based decision table covering adoption, quality, support burden, and options to expand, hold, or improve the rollout.

  • Reviewing adoption after an Abridge rollout
  • Preparing evidence for a renewal decision
  • Comparing workflow outcomes across care settings
jeremylongshore/tons-of-skills-marketplace2.9KUpdated Oct 29, 2026

Prepare a bounded support bundle for an Abridge issue using allowlisted operational details such as versions, coarse timestamps, and correlation references. Exclude recordings, transcripts, patient identifiers, credentials, and unrestricted logs, then document redactions and review details.

  • Preparing reviewed evidence for an Abridge support case
  • Documenting redactions and excluded data
  • Collecting approved operational details for an incident
jeremylongshore/tons-of-skills-marketplace2.9KUpdated Oct 29, 2026

See all skills by jeremylongshore

Report a problem with this skill