Results for “distributed”

Foundational distributed computing algorithms for consensus, coordination, and fault tolerance

The English summary is being prepared. Showing the original description.

Decompose systems into microservices and apply canonical distributed-systems patterns. Use this skill whenever the user wants to split a monolith into services, design service boundaries, choose between microservices and a modular monolith, implement sagas or the outbox pattern, set up CQRS or event sourcing, configure an API gateway or BFF, add resilience (circuit breakers, retries, timeouts), or implement distributed tracing with OpenTelemetry. Also trigger for distributed transactions, service discovery, event-driven architecture, contract testing with Pact, canary releases, and idempotent consumers.

The English summary is being prepared. Showing the original description.

Distributed machine learning, data mining, and iterative HPC with Exasol. Covers end-to-end ML pipelines (DISTRIBUTE BY + SET scripts + BucketFS), per-entity federated training with partial_fit and ctx.reset(), batch inference, map-reduce ensemble training, distributed ensemble and SON algorithm for frequent itemset mining (Apriori, FP-Growth, association rules, market-basket analysis), Lua execute script orchestration for iterative algorithms (k-means, SGD, gradient descent), scikit-learn model training, parallel hyperparameter search, per-entity forecasting, anomaly detection, model lifecycle in BucketFS (pickle/joblib/ONNX versioning), GPU acceleration via CUDA SLCs (PyTorch/TensorFlow/RAPIDS), and ML-specific performance tuning (skew, OOM, multi-pass chunking).

The English summary is being prepared. Showing the original description.

Implement distributed tracing with Jaeger and Tempo to track requests across microservices and identify performance bottlenecks. Use when debugging microservices, analyzing request flows, or implementing observability for distributed systems.

The English summary is being prepared. Showing the original description.

ABP Framework cross-cutting patterns including authorization, background jobs, distributed events, multi-tenancy, and module configuration. Use when: (1) defining permissions, (2) creating background jobs, (3) publishing/handling distributed events, (4) configuring modules.

The English summary is being prepared. Showing the original description.

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

The English summary is being prepared. Showing the original description.

Distributed computing for larger-than-RAM pandas/NumPy workflows. Use when you need to scale existing pandas/NumPy code beyond memory or across clusters. Best for parallel file processing, distributed ML, integration with existing pandas code. For out-of-core analytics on single machine use vaex; for in-memory speed use polars.

The English summary is being prepared. Showing the original description.

Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.

The English summary is being prepared. Showing the original description.

Foundational distributed computing algorithms for consensus, coordination, and fault tolerance

The English summary is being prepared. Showing the original description.

ABP Framework cross-cutting patterns including authorization, background jobs, distributed events, multi-tenancy, and module configuration. Use when: (1) defining permissions, (2) creating background jobs, (3) publishing/handling distributed events, (4) configuring modules.

The English summary is being prepared. Showing the original description.

Simplest distributed training API. 4 lines to add distributed support to any PyTorch script. Unified API for DeepSpeed/FSDP/Megatron/DDP. Automatic device placement, mixed precision (FP16/BF16/FP8). Interactive config, single launch command. HuggingFace ecosystem standard.

The English summary is being prepared. Showing the original description.

Build Redis caching, request limits, locks that coordinate multiple workers, and messaging. Use concrete implementation patterns to choose data structures, set key expiration, manage connections, and review common production pitfalls.

• Add response caching to an API
• Limit API requests per user
• Prevent workers from running the same job simultaneously

Helps organize PyTorch model training with LightningModule and Trainer, including validation, logging, model saving, and early stopping. It also guides distributed training across multiple GPUs and settings for reduced-precision computation.

• Converting existing PyTorch training code to Lightning
• Adding validation metrics and model checkpoints
• Setting up training across multiple GPUs

Implement and debug large-model training with PyTorch FSDP, which splits training state across devices. Consult documentation and code patterns for sharding choices, mixed precision, and checkpoints when building or reviewing distributed training code.

• Implement distributed training for large models
• Compare FSDP wrapping and sharding patterns
• Review mixed-precision and checkpoint examples

Coordinate multiple AI agents for research, development, testing, and system analysis. It guides role assignment, communication structures, parallel and sequential tasks, shared memory, progress monitoring, and recovery, bringing separate contributions into reports or coordinated development workflows.

• Gather research in parallel and compile a report
• Coordinate application design, implementation, and review
• Run unit, integration, security, and performance tests in parallel

Implement and manage Raft, a method for keeping distributed servers in agreement. Covers leader election, log replication, node membership changes, and recovery after network disruptions to maintain consistent state across the cluster.

• Implementing leader election across distributed nodes
• Keeping replicated logs consistent
• Adding or removing cluster nodes

Coordinate gossip protocols, where nodes pass information among peers, for distributed systems. The skill covers update propagation, state synchronization, conflicting changes, failure detection, and tuning communication so nodes eventually reach a consistent state.

• Designing update propagation between distributed nodes
• Synchronizing state and resolving concurrent changes
• Managing node membership and detecting failures

Implement distributed data synchronization with CRDTs, data structures designed to merge concurrent updates. Covers counters, sets, shared values, and sequences, with incremental updates and tracking of dependencies between operations.

• Implementing shared counters across multiple nodes
• Merging concurrent additions and removals in shared sets
• Sending only changes during data synchronization

Train, deploy, and manage neural networks with Flow Nexus in isolated E2B environments. Use custom architectures or ready-made templates, run predictions, monitor distributed training across multiple environments, and evaluate models before sharing them.

• Train an image classification model
• Forecast time series with LSTM
• Deploy a sentiment analysis template

Set up monitoring for communication between services in Istio or Linkerd environments. Combine metrics, logs, and distributed traces, which follow requests across services, to investigate delays and errors, visualize dependencies, and define service quality targets.

• Trace requests across multiple services
• Create communication metrics and dashboards
• Investigate latency, errors, and connectivity issues

Design and implement Saga workflows across services, using compensating actions to undo completed work after failures. Define step deadlines, handle repeated messages safely, and monitor stalled order, booking, or other long-running processes.

• Coordinating orders across inventory, payment, and shipping services
• Designing compensation for failed bookings
• Investigating workflows stuck during compensation

Check investor statements against a fund's NAV pack, the records of its net asset value. Recalculate each investor's capital balance, compare statement lines, and report discrepancies with their causes before statements are distributed.

• Checking investor statements before distribution
• Tracing differences in allocated profit or loss
• Checking capital balances across reporting periods

Develop and troubleshoot distributed applications with Aspire. Coordinate services written in different languages through AppHost, configure connections and startup dependencies, inspect logs and metrics, and prepare deployment settings.

• Creating an Aspire application from a template
• Connecting services written in different languages
• Diagnosing services through logs and traces

Investigate SGLang serving failures by collecting server state, preserving failing requests, and replaying them on a clean target. Summarize evidence and likely causes, then choose the next debugging tool for crashes, stalls, or performance regressions.

• Investigate failed health checks and growing queues
• Reproduce request-specific timeouts or crashes
• Investigate output changes after deployment