Bright Data Adaptive Capacity Control

Overview

Bright Data does not publish one global proxy request limit. Build separate controllers for proxy traffic and dataset jobs, honor current provider signals, and keep target, cost, and policy ceilings stricter than capacity.

Prerequisites

• A workload profile with target, product, and idempotency classes
• Observed latency, error, and provider-code samples
• Owner-approved concurrency, byte, cost, and wall-time ceilings

Instructions

Step 1: Measure the lane

Read recent receipts and Grep for 429, Retry-After, provider error codes, active dataset jobs, per-IP concentration, and queue depth. Do not infer capacity from successful bursts alone.

Step 2: Separate controllers

Write independent limits for proxy requests, Browser API sessions, async triggers, progress polling, downloads, and delivery workers. One product's success must not raise another product's ceiling.

Step 3: Apply evidence-based backoff

Retry only idempotent transient work. Honor provider guidance, use bounded jitter, cap attempts and elapsed time, and open the circuit on policy, authentication, or data errors.

Step 4: Tune gradually

Edit one limit at a time, run an approved canary, compare throughput, error, and cost, and retain the lower limit unless the evidence and owner approve promotion.

Tool Discipline

Use Read and Grep to analyze redacted metrics and documented codes. Use Write and Edit for controller configuration, tests, and runbooks. This skill does not run live traffic or change Bright Data account limits.

Output

• Per-lane concurrency and queue budgets
• Retry and circuit matrix by documented failure class
• Canary comparison with rollback threshold

Examples

If dataset triggers return a tenant-specific parallel-job error, stop launching new jobs and continue bounded progress polling for owned snapshots. If proxy traffic returns a per-IP 429, reduce pressure and review distribution rather than guessing a universal rate.

Error Handling

Failure	Meaning	Response
No provider code is retained	429 cause is ambiguous	Improve redacted instrumentation before tuning
Retries amplify queue depth	Backoff lacks admission control	Open the circuit and drain owned work
Throughput improves but cost or error ceiling fails	Optimization violates the budget	Roll back the changed limit

Resources

• Proxy error catalog
• Scraper async requests guide
• Download snapshot
• Proxy configuration options