Results for “data parallel”

Reads from, writes to, and manages relational databases using MATLAB Database Toolbox. Use when connecting to databases, reading data with sqlread or fetch, filtering with rowfilter, writing with sqlwrite, updating with sqlupdate, executing SQL statements, managing transactions with commit and rollback, mapping MATLAB classes to tables with ORM (Mappable, ormread, ormwrite, ormupdate), or performing any database operation from MATLAB. Triggers on: database, SQL, sqlread, sqlwrite, sqlupdate, fetch, execute, rowfilter, RowFilter, ORM, Mappable, ormread, ormwrite, ormupdate, orm2sql, transaction, commit, rollback, Database Toolbox, PostgreSQL, MySQL, SQLite, SQL Server, Oracle, database connection, database table, query database, insert data, update rows, delete rows, stored procedure, prepared statement, odbc, databaseConnectionOptions, datasource, data source, DSN, connection string, multithreaded, parallel.

The English summary is being prepared. Showing the original description.

Guide users or agents to the correct MATLAB tool for processing large tabular data in file-based formats (CSV, Parquet, delimited text, spreadsheets, MDF) that may not fit in memory. Use when a user or agent mentions large files, big data, out-of-memory errors, OOM, scaling up, tall arrays, datastores, or needs to process multiple tabular files. Covers the decision between datastore + tall, datastore + transform, and parallel execution. Also use when a user or agent has working in-memory code (readtable, parquetread) that runs out of memory and needs a migration path. Also covers building custom datastore classes for proprietary or non-standard formats — use when the task requires subclassing matlab.io.Datastore, implementing a custom reader, building an extensible datastore, or integrating a new file format with tall arrays or parallel computing. Do NOT use for MAT files (use matfile instead).

The English summary is being prepared. Showing the original description.

Set up worker environment and per-worker state for parallel pools. Use when code needs paths, environment variables, database connections, loaded libraries, or expensive objects available on workers before parfor/parfeval runs. Teaches parallel.pool.Constant, parfevalOnAll, and parpool name-value pairs. Also use when refactoring existing code that uses spmd for side-effect setup (an anti-pattern). Triggers: worker setup, pool constant, per-worker state, non-serializable, loadlibrary on workers, database connection parfor, addpath workers, spmd before parfor, worker environment, reduce parfor overhead, parfor setup, resource creation in parallel loop, cannot serialize error, undefined function or variable on workers error, load data per worker, reduce data transfer, parallelize setup, improve parallel code.

The English summary is being prepared. Showing the original description.

Research current web information with Parallel CLI and organize findings with supporting sources. The skill covers search, page extraction, detailed reports, adding sourced fields to existing lists, finding matching entities, and recurring monitoring of web changes.

• Research current information and locate academic sources
• Extract content from public webpages and PDFs
• Add sourced information to company or product lists

Route code changes and SQL analysis to isolated Kortix work sessions with clear requirements and workspace details. Coordinate parallel investigations or independent pull request reviews, then summarize deliverables, verification results, and differences between findings.

• Delegate feature implementation or bug fixes to a separate session
• Compare two agents' approaches to a refactor
• Combine findings from two independent pull request reviews

Download genome assemblies, gene records, and ortholog data from NCBI using the modern Datasets v2 CLI (replaces assembly_summary.txt scraping and many EFetch workflows). Use when bulk-pulling genome assemblies, gene metadata across species, ortholog sets, or BLAST databases; when E-utilities are too slow for genome-scale work; or when automatic checksum verification, parallel download, and clean accession-driven retrieval are required. Encodes the JSON-lines output format, dataformat conversion, --dehydrated for cloud workflows, and when Datasets is/isn't the right tool.

The English summary is being prepared. Showing the original description.

Use this skill when conditioning, loading, preparing, or labeling signal data for analysis or ML training. Covers: cleaning a single signal (fill gaps, remove drift, deoutlier, denoise, resample/align a time base) BEFORE analysis; building a `signalDatastore` pipeline; creating a `labeledSignalSet` for Signal Labeler; deriving labels (filename, folder, in-file, ROI, time-frequency ROI); stratified train/val/test splits; framing long signals; parallel processing; and shaping datastore output for `trainnet`. Triggers include "clean up this signal", "remove drift / detrend", "fill gaps", "remove spikes / outliers", "denoise", "resample to a uniform rate", "align channels", "labels from filenames", "stratified split", "prepare for Signal Labeler", and function names like `fillgaps`, `fillmissing`, `detrend`, `filloutliers`, `smoothdata`, `resample`, `synchronize`, `signalDatastore`, `labeledSignalSet`, `filenames2labels`, `folders2labels`, `splitlabels`, `framesig`, `framelbl`, `createDatastores`.

The English summary is being prepared. Showing the original description.

GTM prospecting, enrichment, outreach, and Deepline Play work/audits. Providers: adyntel,affinity,ai_ark,akta,allegrow,amplemarket,apify,attention,attio,aviato,bettercontact,bigquery,bluesky,bounceban,brave,browserbase,builtwith,cbinsights,clay,clickhouse,cloudflare,contactout,contextdev,crunchbase,crustdata-v3,customer_db,databricks,dataforseo,datagma,dealroom,deepline_ip_to_company,deepline_native,deeplineagent,discolike,dropleads,edges,emailbison,emailguard,enformion,enigma,exa,findymail,firecrawl,fireflies,firmable,forager,fullenrich,fundable,generic_http,getleads,gong,google_ads_audiences,google_workspace,govfiles,grain,hackernews,harvestapi,heyreach,hubspot,hunter,icypeas,instantly,intercom,ipqs,kaspr,kernel,leadmagic,lemlist,limadata,linkedin_ads_audiences,luma,lusha,meta_audiences,millionverifier,nooks,notion,openmart,opensosdata,openwebninja,outreach,parallel,peopledatalabs,pitchbook,podscan,polymarket,postgres,predictleads,prospeo,quickenrich,rapidapi,redshift,rocketreach,salesforce,salesforge,….

The English summary is being prepared. Showing the original description.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

The English summary is being prepared. Showing the original description.

Download genome assemblies, gene records, and ortholog data from NCBI using the modern Datasets v2 CLI (replaces assembly_summary.txt scraping and many EFetch workflows). Use when bulk-pulling genome assemblies, gene metadata across species, ortholog sets, or BLAST databases; when E-utilities are too slow for genome-scale work; or when automatic checksum verification, parallel download, and clean accession-driven retrieval are required. Encodes the JSON-lines output format, dataformat conversion, --dehydrated for cloud workflows, and when Datasets is/isn't the right tool.

The English summary is being prepared. Showing the original description.

Download genome assemblies, gene records, and ortholog data from NCBI using the modern Datasets v2 CLI (replaces assembly_summary.txt scraping and many EFetch workflows). Use when bulk-pulling genome assemblies, gene metadata across species, ortholog sets, or BLAST databases; when E-utilities are too slow for genome-scale work; or when automatic checksum verification, parallel download, and clean accession-driven retrieval are required. Encodes the JSON-lines output format, dataformat conversion, --dehydrated for cloud workflows, and when Datasets is/isn't the right tool.

The English summary is being prepared. Showing the original description.

Build Python workflows for filtering, grouping, joining, and reshaping table data with Polars. Get guidance on migrating pandas code and using lazy evaluation, which optimizes a processing plan before execution, for large datasets.

• Filtering and summarizing large CSV files
• Migrating pandas processing code to Polars
• Joining and reshaping multiple tables

Transform sequential Python code into parallel/concurrent implementations. Use when asked to parallelize Python code, improve code performance through concurrency, convert loops to parallel execution, or identify parallelization opportunities. Handles CPU-bound (multiprocessing), I/O-bound (asyncio, threading), and data-parallel (vectorization) scenarios.

The English summary is being prepared. Showing the original description.

Audit a dataset or access rule before it joins an identity-graph build (access rules behave like datasets in NQL here). Enumerates failure modes (hub identifiers, high-degree nodes, suspicious values, over-connected identifiers), tests hypotheses in parallel, quantifies damage by rows / edges / entities, and proposes minimal filters ranked by severity. When issues are found, returns a validated `CREATE MATERIALIZED VIEW` NQL the caller can run to produce a graph-ready clean source; if the data passes, says so and recommends it unchanged. Plans and authors the clean-view NQL; does not execute it. Use when: "audit this dataset before the graph build", "find bad edges in <source>", "check identity data quality", "recommend filters for the graph build", "quantify damage from <identifier_type>", "pre-graph DQ". (narrative-identity)

The English summary is being prepared. Showing the original description.

Coordinate parallel AI agent sessions with dmux, a tool that manages terminal panes. Split development, research, testing, or review into separate tasks, isolate overlapping file changes, and review results before merging them.

• Running research and implementation in parallel
• Splitting database, API, and UI work across agents
• Coordinating test monitoring and fixes

Process molecular datasets in Python through a simpler interface to RDKit, a chemistry toolkit. Convert and standardize structures, calculate molecular properties, compare and cluster compounds, generate 3D conformations, and visualize results for drug discovery workflows.

• Load and standardize molecular datasets
• Calculate molecular properties and similarities
• Cluster compounds and select diverse subsets

Manage separate local development environments for multiple branches using the project's pnpm worktree CLI. Each environment has its own ports and dependencies, with options to share Supabase or isolate database changes for testing.

• Running and testing multiple branches at the same time
• Testing database migrations in a separate database
• Reusing an existing worktree for related fixes

RAG Builder with Parallel Document Processing Vector database construction with local embeddings (zero cost) Handles PDF download, text extraction, chunking, and vector database creation Absorbed B5 (Parallel Document Processor) capabilities Use when: building RAG, creating vector database, downloading PDFs, embedding documents, batch processing Triggers: build RAG, create vector database, download PDFs, embed documents, batch PDF processing

The English summary is being prepared. Showing the original description.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

The English summary is being prepared. Showing the original description.

Parallel/distributed computing for larger-than-RAM data. Components: DataFrames (parallel pandas), Arrays (parallel NumPy), Bags, Futures, Schedulers. Scales laptop to HPC cluster. For single-machine speed use polars; for out-of-core without cluster use vaex.

The English summary is being prepared. Showing the original description.

Diagnose and mitigate Salesforce data skew — ownership skew (single user owns >10,000 records) and parent-child skew (>10,000 children under one parent) — that cause sharing recalculation slowness, group membership lock errors, and record-level locking failures. Covers lookup skew and UNABLE_TO_LOCK_ROW, Defer Sharing Calculation (deferGroupMembership / deferSharingRules in Sharing.settings), ownership bucket groups, Controlled by Parent OWD, Bulk API serial vs parallel concurrency for skewed loads, and the documented load ordering for roles, groups and sharing rules. NOT for planning and batching a recalculation job — use data/sharing-recalculation-performance. NOT for designing the sharing model itself — use admin/sharing-and-visibility.

The English summary is being prepared. Showing the original description.

Manages git worktrees for parallel agent isolation. Creates isolated worktrees for parallel agents on parallel/{name}-{timestamp} branches, merges results with conflict detection, prunes stale metadata, and reports health status. NOT for user-initiated feature branches — use git worktree directly for that.

The English summary is being prepared. Showing the original description.

Estimate intracellular reaction rates from steady-state carbon-13 tracing measurements. Validate carbon atom mappings and measurement inputs, simulate labeling, fit constrained fluxes, and diagnose which pathway rates the data can meaningfully constrain.

• Estimate metabolic fluxes from carbon-13 labeling
• Validate atom mappings and measurement inputs
• Simulate labeling for proposed reaction rates

Use Dask to process tables, numerical arrays, and file collections in pieces across CPU cores or machines. Helps choose data structures, size memory-friendly chunks, coordinate dependent tasks, and diagnose slow or memory-heavy workflows.

• Aggregate CSV or Parquet files that exceed available memory
• Process large scientific arrays in chunks
• Clean JSON records and logs before analysis