20 skills · sorted by relevance
Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.
Configure large language model pretraining with torchtitan across multiple GPUs. Covers ways to split training work, launch jobs on computing clusters, enable Float8 numerical processing, and save or resume training through distributed checkpoints.
Answer Python questions and write code using examples fetched from pythonsheets.com. Find patterns for syntax, asynchronous tasks, networking, databases, machine learning, and distributed computing, or use topic-based questions to prepare for Python interviews.
Distributed computing for larger-than-RAM pandas/NumPy workflows. Use when you need to scale existing pandas/NumPy code beyond memory or across clusters. Best for parallel file processing, distributed ML, integration with existing pandas code. For out-of-core analytics on single machine use vaex; for in-memory speed use polars.
The English summary is being prepared. Showing the original description.
Foundational distributed computing algorithms for consensus, coordination, and fault tolerance
The English summary is being prepared. Showing the original description.
Foundational distributed computing algorithms for consensus, coordination, and fault tolerance
The English summary is being prepared. Showing the original description.
Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.
Run batch processing, scientific simulations, and machine learning training on Google Kubernetes Engine. Configure parallel jobs, queues that manage resource access, and distributed computation, with guidance on scaling idle capacity down and recovering from interruptions.
Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.
Scalable data processing for ML workloads. Streaming execution across CPU/GPU, supports Parquet/CSV/JSON/images. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use for batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipelines.
The English summary is being prepared. Showing the original description.
Help users write correct R code for async, parallel, and distributed computing using mirai. Use when users need to run R code asynchronously or in parallel, write mirai code with correct dependency passing, set up parallel workers, convert from future or parallel, use mirai_map, integrate with Shiny or promises, or configure cluster/HPC computing.
The English summary is being prepared. Showing the original description.
Parallel/distributed computing for larger-than-RAM data. Components: DataFrames (parallel pandas), Arrays (parallel NumPy), Bags, Futures, Schedulers. Scales laptop to HPC cluster. For single-machine speed use polars; for out-of-core without cluster use vaex.
The English summary is being prepared. Showing the original description.
Distributed computing for larger-than-RAM pandas/NumPy workflows. Use when you need to scale existing pandas/NumPy code beyond memory or across clusters. Best for parallel file processing, distributed ML, integration with existing pandas code. For out-of-core analytics on single machine use vaex; for in-memory speed use polars.
The English summary is being prepared. Showing the original description.
Scalable data processing for ML workloads. Streaming execution across CPU/GPU, supports Parquet/CSV/JSON/images. Integrates with Ray Train, PyTorch, TensorFlow. Scales from single machine to 100s of nodes. Use for batch inference, data preprocessing, multi-modal data loading, or distributed ETL pipelines.
The English summary is being prepared. Showing the original description.
Parallel/distributed computing. Scale pandas/NumPy beyond memory, parallel DataFrames/Arrays, multi-file processing, task graphs, for larger-than-RAM datasets and parallel workflows.
The English summary is being prepared. Showing the original description.
Parallel and distributed computing for large-scale data processing
The English summary is being prepared. Showing the original description.
Expert guide for Backend.AI distributed computing platform. Automatically activates when users ask about: - Backend.AI architecture, components (Manager, Agent, Storage Proxy, Webserver, App Proxy) - Features (session scheduling, Sokovan orchestrator, multi-tenancy, resource allocation) - APIs (REST, GraphQL), authentication, RBAC authorization - Container runtime (kernels, jail sandbox, hook library, virtual folders) - Accelerator support (CUDA, ROCm, TPU, NPU, Graphcore IPU) - Client SDKs (Python, Java, JavaScript, PHP) - Setup, requirements (Python 3.13+, Docker, PostgreSQL, Redis, etcd) - How WebUI connects to/interacts with Backend.AI backend - Plugin interfaces, development setup, infrastructure Use when user mentions "Backend.AI", "backend.ai", "Sokovan", component names, or asks about the backend platform this WebUI connects to. allowed-tools: WebFetch, Read
The English summary is being prepared. Showing the original description.
Expert guide for Backend.AI distributed computing platform. Automatically activates when users ask about: - Backend.AI architecture, components (Manager, Agent, Storage Proxy, Webserver, App Proxy) - Features (session scheduling, Sokovan orchestrator, multi-tenancy, resource allocation) - APIs (REST, GraphQL), authentication, RBAC authorization - Container runtime (kernels, jail sandbox, hook library, virtual folders) - Accelerator support (CUDA, ROCm, TPU, NPU, Graphcore IPU) - Client SDKs (Python, Java, JavaScript, PHP) - Setup, requirements (Python 3.13+, Docker, PostgreSQL, Redis, etcd) - How WebUI connects to/interacts with Backend.AI backend - Plugin interfaces, development setup, infrastructure Use when user mentions "Backend.AI", "backend.ai", "Sokovan", component names, or asks about the backend platform this WebUI connects to. allowed-tools: WebFetch, Read
The English summary is being prepared. Showing the original description.
Processamento escalável de dados para workloads de ML. Execução em streaming em CPU/GPU, suporta Parquet/CSV/JSON/imagens. Integra-se com Ray Train, PyTorch, TensorFlow. Escala de uma única máquina para centenas de nós. Use para inferência em lote, pré-processamento de dados, carregamento de dados multi-modal ou pipelines distribuídos de ETL.
The English summary is being prepared. Showing the original description.
Apache Spark, Hadoop, distributed computing, and large-scale data processing for petabyte-scale workloads
The English summary is being prepared. Showing the original description.