Results for “pipeline parallel”

Structured 6-step procedure for improving, renovating, or rebuilding existing pipelines, individual project folders, documentation structures, or software stacks. Addressable as "pipeline optimizer" (for whole topic pipelines, e.g. a software, research, or game-dev pipeline) or "project-folder optimizer" (for individual project folders within a pipeline, e.g. a single software tool or paper project). Triggers on tasks like "improve pipeline X", "optimize the stack", "rebuild Y", "renovation", "pipeline refactoring", "clean up project folder", "improve folder structure", "unify conventions", "documentation consolidation", "integrate into existing system", or any substantial intervention in established structures. Delivers building-stock analysis, purpose clarification, ideal sketch, gap plan, empirical pain-point identification, and retests with fresh subagents. Prevents parallel standards, duplication, and pipeline breaks.

The English summary is being prepared. Showing the original description.

Explains how to replace the removed $pipeline command with $plan for planning and $team for coordinated parallel work. This retirement notice provides migration guidance rather than an executable workflow of its own.

• Updating instructions that still reference $pipeline
• Choosing separate commands for planning and parallel execution
• Finding the command for persistent goal runs

Configure large language model training with NVIDIA Megatron-Core across multiple GPUs. This skill helps choose how to split computation, set training parameters, monitor GPU efficiency, and address memory limits or unstable training.

• Planning multi-GPU training for a LLaMA-style model
• Distributing MoE model experts across GPUs
• Tuning training throughput and GPU utilization

Configure large language model training across multiple NVIDIA GPUs with Megatron-Core. The skill helps choose how to split models and workloads, set training parameters, monitor performance, and address memory shortages or low GPU utilization.

• Planning distributed training for a LLaMA-style model
• Distributing MoE expert networks across GPUs
• Tuning GPU utilization and training throughput

Carry out a validated development plan in an isolated Git worktree, a separate working directory. Coordinate parallel implementation, run quality checks, update documentation, and take the changes through pull request creation, merging, execution metrics, and cleanup.

• Implementing a validated development plan
• Running dependent tasks in parallel stages
• Preparing failing tests before implementation

Parallel SDLC batch runner. Analyzes a list of independent task descriptions, groups them by heuristic file-conflict prediction, dispatches one worktree-isolated background agent per task (each running the full pipeline-orchestrator: BA → Dev → QA → Sec → Docs), and aggregates results into a single batch summary. Use when: - `/sdlc:batch` invokes this skill after parsing its arguments (Step 2 of commands/batch.md) - You have 2+ independent feature/fix descriptions to run through the SDLC pipeline in parallel Do NOT use for: - A single task (use pipeline-orchestrator directly via /sdlc:start) - Tasks that must share a single working tree (this skill isolates each task in its own git worktree)

The English summary is being prepared. Showing the original description.

Guide a development brief through specification, dependent tickets, implementation, review, and verification. Coordinate parallel work while keeping human approval at key decisions and recording agreed vocabulary, business rules, and the reasons behind technical choices.

• Turning a development brief into a specification and tickets
• Coordinating parallel implementation across dependent tickets
• Approving acceptance criteria and testing boundaries

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Helps configure and troubleshoot DeepSpeed for training large models across multiple devices. Covers memory optimization, parallel training, and DeepNVMe storage transfers, with examples for tuning data movement and configuring models that divide work among specialized components.

• Configuring distributed training for large models
• Tuning tensor transfers to NVMe storage
• Setting up models with specialized expert components

Manage shell commands and AI agent tasks through a shared gbrain job queue. Track progress, steer agents, pause or resume work, and coordinate parallel tasks, with guidance for long operations, completion reporting, and reusable stage checkpoints.

• Run scripts as monitored background jobs
• Coordinate parallel research by AI agents
• Redirect, pause, or resume an agent task

Write Workflow scripts that divide research, reviews, or migrations among agents. The reference explains parallel and staged processing, independent verification, failure handling, budgets, and resuming runs so the script has a clear execution structure.

• Design a review workflow with independent verification
• Divide broad research among agents and combine findings
• Plan staged processing for a large migration

Implement and troubleshoot large-model training with DeepSpeed across multiple GPUs. The skill covers memory optimization, parallel training, and numerical precision choices, alongside examples for storage transfers and models that divide work among specialized components.

• Review settings for distributed model training
• Tune transfers between NVMe storage and tensors
• Configure training with specialized model components

Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.

• Pretraining Llama 3.1 8B on eight GPUs
• Configuring multi-node training with SLURM
• Enabling Float8 training on H100 GPUs

Improve CircleCI build and test pipelines by identifying slow or unreliable jobs, then adjusting caching, data sharing between jobs, triggers, and parallel execution. Receive incremental configuration changes with expected effects, validation criteria, and rollback guidance.

• Reduce long CI build and test times
• Improve dependency cache reuse
• Remove duplicate jobs and unnecessary runs

Design CI/CD pipelines that run test suites. Covers GitHub Actions and GitLab CI templates, parallelism and sharding, artifact management, flaky-test quarantine, test-result publishing, coverage quality gates, OIDC keyless deploy, and copy-paste workflows for Playwright, Jest, and multi-stage pipelines. Use when: "CI/CD," "GitHub Actions," "pipeline," "test in CI," "GitLab CI," "continuous integration," "test automation pipeline," "shard tests in CI." Not for: per-test flaky healing at runtime — use test-reliability; go/no-go release decisions and smoke-test checklists — use release-readiness; test-result dashboards and trend reporting — use qa-metrics. Related: playwright-automation, qa-metrics, test-reliability, coverage-analysis, release-readiness.

The English summary is being prepared. Showing the original description.

Design and implement multi-stage CI/CD pipelines using GitHub Actions with matrix builds, dependency caching, artifact management, and secret handling. Create workflows that span linting, testing, building, and deployment stages with parallel execution and conditional logic. Use when setting up automated testing and deployment for a new project, migrating from Jenkins or CircleCI to GitHub Actions, implementing matrix builds across platforms, adding build caching, or creating multi-stage pipelines with security scanning and quality gates.

The English summary is being prepared. Showing the original description.

Treina modelos de linguagem grandes (2B-462B de parâmetros) usando NVIDIA Megatron-Core com estratégias avançadas de paralelismo. Use ao treinar modelos >1B de parâmetros, precisar de máxima eficiência de GPU (47% MFU em H100), ou necessitar de paralelismo tensor/pipeline/sequência/contexto/expert. Framework pronto para produção usado em Nemotron, LLaMA, DeepSeek.

The English summary is being prepared. Showing the original description.

Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.

The English summary is being prepared. Showing the original description.

Plan autonomous Claude Code workflows, from sequential development steps to parallel agents working from a specification. The material explains how to preserve progress between runs, check quality, coordinate merges, and set limits that stop repeated execution.

• Designing a sequential development pipeline
• Preserving progress across repeated agent runs
• Coordinating parallel work and merges

Create workflows that pause for approval and resume later, or scripts that distribute work across multiple agents. Choose a saved template for ongoing processes or a JavaScript script to collect structured results from parallel tasks.

• Build a workflow with approval checkpoints
• Split review work across multiple agents
• Collect structured results from research tasks

Design workflows in which AI agents handle multiple steps or specialist tasks. Choose a suitable structure, generate starter configurations, and define information handoffs, retries, validation checks, and limits on cost and context size.

• Choosing between one agent and a multi-agent workflow
• Generating a starter configuration for a multi-step pipeline
• Defining information handoffs and error recovery rules

Classical end-to-end empirical analysis workflow in the traditional Python econometric stack — pandas + numpy + scipy + statsmodels + linearmodels + pyfixest + rdrobust + econml + causalml + matplotlib/seaborn. **Defaults to economics empirical-paper style** (AER / QJE / AEJ) — every run produces a publication-ready output set with a multi-column regression table (M1→M6 progressive controls/FE) as the centerpiece, plus Table 1 (descriptives), mechanism / heterogeneity / robustness tables, and event-study + coefficient + trend figures. Covers the full 8-step pipeline an applied economist or quantitative social scientist runs on every paper — (1) data cleaning, (2) variable construction & transformation, (3) descriptive statistics & Table 1, (4) statistical diagnostic tests, (5) baseline empirical modeling, (6) robustness battery, (7) further analysis (mechanism, heterogeneity, mediation, moderation), (8) publication-ready tables & figures. **Also covers two parallel domain modes that share the same 8-step scaffolding** — **Mode A — Epidemiology / public health** (target-trial emulation via `zepid` / hand-rolled `pandas`, IPTW + g-formula + TMLE doubly-robust triplet via `zepid` / `econml` / `lifelines`, Mendelian randomization via `pymr` / `mrtool` (or `rpy2` → `MendelianRandomization`/`TwoSampleMR`), KM / AFT / Cox survival via `lifelines`, E-value sensitivity, principal stratification — STROBE / TRIPOD reporting), and **Mode B — ML causal inference** (DML via `econml.dml` / `doubleml`, S/T/X/R/DR meta-learners via `econml.metalearners` / `causalml`, causal forest via `econml.grf` / `causalml`, Dragonnet / TARNet / CEVAE neural causal via `causalml`, BCF via `pymc-bart` / `bcf-py`, matrix completion, CATE distribution + policy tree via `econml.policy` / `policytree-py`, off-policy evaluation, conformal causal via `mapie`, fairness audit via `fairlearn`, DAG learning via `causal-learn` / `cdt` / LLM-assisted). Prescribes which library to reach for at each step, shows the canonical code, and links to deeper `references/` files for variant-specific patterns. Use when the user asks for a **complete empirical analysis** in Python, wants to replicate an applied-economics paper from scratch, needs a reproducible workflow that is NOT opinionated on any single vertical package (contrast with StatsPAI), wants explicit control over every estimator and diagnostic, or asks "how do I write a full empirical pipeline in Python?". Also triggers when the user names a specific classical step in isolation — "winsorize at 1/99%", "run Breusch-Pagan", "build a Table 1 balance table", "do a placebo test", "event study plot", "mediation analysis" — and wants it wired into the broader pipeline. Mode A triggers on "target trial emulation", "IPTW", "TMLE", "Mendelian randomization", "STROBE", "公共健康", "流行病学". Mode B triggers on "DML", "double machine learning", "causal forest", "meta-learner", "Dragonnet", "BCF", "policy tree", "conformal causal", "fairness audit", "因果机器学习".

The English summary is being prepared. Showing the original description.

Decision protocol for the map-reduce / dynamic fan-out pattern in LM pipelines — "given list L, run f(item) for each item in parallel, then combine". Activates when the coder agent is about to process N items with N LM calls (per-doc summarize, per-query retrieve, per-candidate rank, parallel tool fan-out). Encodes the *when*, *how many at once*, *what to do when one fails*, and *how to reduce* — not the API of any single framework. Cross-framework: LangGraph `Send`, CrewAI parallel tasks / Flow, `asyncio.gather`, `ThreadPoolExecutor`, LlamaIndex batch retrieval.

The English summary is being prepared. Showing the original description.

The systematic orchestration of AI-powered marketing workflows that combine content generation, approval processes, multi-channel distribution, and quality gates into cohesive automation systems. This skill integrates AI generation tools (Jasper, Claude, GPT) with automation platforms (Zapier, Make, n8n) and marketing systems to build scalable content pipelines. It focuses on maintaining brand consistency, implementing rigorous quality gates, and balancing automation with strategic human oversight. Key capabilities include designing parallel approval flows, monitoring costs, and architecting "invisible" automation that enhances productivity without sacrificing quality.Use when "AI workflow, automate content, content automation, workflow automation, AI pipeline, automated marketing, content distribution automation, approval workflow, scale content production, AI orchestration, automation, workflow, ai-orchestration, content-pipeline, approval-workflow, multi-channel, quality-gates, cost-control" mentioned.

The English summary is being prepared. Showing the original description.