644 skills · sorted by relevance
Analyze CSV, Parquet, and JSON files to answer questions through grouping, joins, cleaning, time series, and charts. Chooses DuckDB or Polars and a suitable processing location based on the task and available memory.
Build Java applications that store, query, update, and delete data in Azure Cosmos DB. Covers synchronous and asynchronous clients, partition keys for distributing data, consistency settings, request usage, and error handling.
Improve slow Apache Spark jobs by reviewing data partitions, caching, memory settings, and shuffles, which move data between workers. Use the guidance to investigate bottlenecks and tune pipelines for larger datasets.
Forecast sales, demand, or sensor readings with Google's pretrained TimesFM model, without custom training. Check hardware and dataset memory needs, include external factors such as prices or holidays, and obtain forecasts with uncertainty ranges.
Write complex SQL, improve slow queries, and design database schemas for transaction processing and analytics. Uses schema details and query execution plans to guide indexing, validate results, and assess performance with realistic data volumes.
Triage proposed personal data uses to determine whether they can proceed, need a privacy impact assessment, require a legally mandated assessment, or must stop. Results explain policy conflicts, applicable triggers, and conditions to address before proceeding.
Develop and troubleshoot Apache Beam's YAML SDK. Understand its source files, run data processing pipelines locally or on Dataflow, test changes, and investigate provider issues or mismatched record fields.
Extract details from TikTok video URLs, including captions, author profiles, view and engagement counts, music, and hashtags. Process individual posts or a list of URLs, with image links available for slideshow posts.
RAG Builder with Parallel Document Processing Vector database construction with local embeddings (zero cost) Handles PDF download, text extraction, chunking, and vector database creation Absorbed B5 (Parallel Document Processor) capabilities Use when: building RAG, creating vector database, downloading PDFs, embedding documents, batch processing Triggers: build RAG, create vector database, download PDFs, embed documents, batch PDF processing
The English summary is being prepared. Showing the original description.
Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient visualization of big data, and machine learning on large datasets. Apply when users need to work with large CSV/HDF5/Arrow/Parquet files, perform fast statistics on massive datasets, create visualizations of big data, or build ML pipelines that do not fit in memory.
The English summary is being prepared. Showing the original description.
Gene Expression Omnibus(GEO) and The Cancer Genome Atlas (TCGA) provide us with a wealth of data, such as RNA-seq, DNA Methylation, SNP and Copy number variation data. It's easy to download data from TCGA using the gdc tool, but processing these data into a format suitable for bioinformatics analysis requires more work. This R package was developed to handle these data.
The English summary is being prepared. Showing the original description.
Best practices for polars data processing with dataframely. Covers definitions of Schema and Collection, usage of .validate() and .filter(), type hints, and testing. Use when writing or modifying code involving dataframely or polars data frames.
The English summary is being prepared. Showing the original description.
Guides the creation and review of data processing agreements under GDPR Article 28(3), covering all eight mandatory clauses. References the 2021 Standard Contractual Clauses and provides a compliance checklist for processor contracts. Activate when onboarding processors, reviewing DPAs, or auditing processor compliance. Keywords: DPA, data processing agreement, Article 28, processor, mandatory clauses, standard contractual clauses.
The English summary is being prepared. Showing the original description.
GDPR-compliant Data Processing Agreement drafting per Article 28(3). Covers all 8 mandatory provisions including subject matter, duration, nature and purpose, data types, categories of data subjects, controller and processor obligations, and sub-processor cascade requirements.
The English summary is being prepared. Showing the original description.
Governs employee health data processing for fitness-for-work assessments, occupational health surveillance, COVID testing legacy programmes, and absence management. Applies Art. 9(2)(b) employment obligations and Art. 9(2)(h) health professional exceptions. Covers data minimisation, occupational health provider relationships, and return-to-work procedures. Keywords: health data, Art. 9, occupational health, fitness-for-work, special category, employment, sickness absence.
The English summary is being prepared. Showing the original description.
Builds end-to-end predictive modeling pipelines on tabular datasets—data exploration, preprocessing, baseline modeling, and mandatory optimization—producing valid saved predictions in the required output format. Use for any supervised machine learning or data modeling task, including classification and regression on tabular data, Kaggle-style competitions or submission-file challenges, training a model to predict a target from features, or requests to fit/tune/evaluate models with scikit-learn, XGBoost, Random Forest, Logistic Regression, and similar non-deep-learning methods. Runs fully automated end-to-end without asking the user for intermediate decisions.
The English summary is being prepared. Showing the original description.
Process vehicle insurance Excel data using Pandas - file handling, data cleaning, merging, validation. Use when processing Excel/CSV files, handling data imports, implementing business rules (negative premiums, zero commissions), debugging data pipelines, or optimizing Pandas performance. Keywords: data_processor.py, Excel, CSV, Pandas, merge, deduplication, date normalization.
The English summary is being prepared. Showing the original description.
Spark, pandas, polars, DuckDB optimization for batch data processing. Activate on: batch processing, Spark optimization, polars, DuckDB, pandas performance, data frame, shuffle, partition, memory optimization. NOT for: streaming pipelines (use streaming-pipeline-architect), warehouse queries (use data-warehouse-optimizer).
The English summary is being prepared. Showing the original description.
GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.
The English summary is being prepared. Showing the original description.
Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient visualization of big data, and machine learning on large datasets. Apply when users need to work with large CSV/HDF5/Arrow/Parquet files, perform fast statistics on massive datasets, create visualizations of big data, or build ML pipelines that do not fit in memory.
The English summary is being prepared. Showing the original description.
Spark, pandas, polars, DuckDB optimization for batch data processing. Activate on: batch processing, Spark optimization, polars, DuckDB, pandas performance, data frame, shuffle, partition, memory optimization. NOT for: streaming pipelines (use streaming-pipeline-architect), warehouse queries (use data-warehouse-optimizer).
The English summary is being prepared. Showing the original description.
GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.
The English summary is being prepared. Showing the original description.
Converts documents into clean, chunked datasets suitable for embeddings and vector search. Produces chunked JSONL files with metadata, deduplication logic, and quality checks. Use when preparing "training data", "vector datasets", "document processing", or "embedding data".
The English summary is being prepared. Showing the original description.
Analyze large scientific tables with Vaex, including datasets that exceed available memory. Build calculated columns, filter rows, compute statistics, visualize aggregated data, and convert files while identifying operations that may still require substantial RAM.