Results for “datasets”

HuggingFace hf CLI: search/download/upload models, datasets.

The English summary is being prepared. Showing the original description.

Fast in-memory DataFrame library for datasets that fit in RAM. Use when pandas is too slow but data still fits in memory. Lazy evaluation, parallel execution, Apache Arrow backend. Best for 1-100GB datasets, ETL pipelines, faster pandas replacement. For larger-than-RAM data use dask or vaex.

The English summary is being prepared. Showing the original description.

Molecular ML with diverse featurizers and pre-built datasets. Use for property prediction (ADMET, toxicity) with traditional ML or GNNs when you want extensive featurization options and MoleculeNet benchmarks. Best for quick experiments with pre-trained models, diverse molecular representations. For graph-first PyTorch workflows use torchdrug; for benchmark datasets use pytdc.

The English summary is being prepared. Showing the original description.

Download large datasets from NCBI efficiently using EPost, history server, batching, rate limiting, and retry logic. Use when bulk-fetching tens of thousands of sequences, pulling all results of a large ESearch, designing reproducible pipelines, comparing E-utilities to NCBI Datasets v2 CLI, or implementing checksum-validated downloads. Encodes WebEnv TTL (~8h), EPost 200-ID limit, retmax caps, parallelization design, and integrity verification.

The English summary is being prepared. Showing the original description.

Download large datasets from NCBI efficiently using EPost, history server, batching, rate limiting, and retry logic. Use when bulk-fetching tens of thousands of sequences, pulling all results of a large ESearch, designing reproducible pipelines, comparing E-utilities to NCBI Datasets v2 CLI, or implementing checksum-validated downloads. Encodes WebEnv TTL (~8h), EPost 200-ID limit, retmax caps, parallelization design, and integrity verification.

The English summary is being prepared. Showing the original description.

HuggingFace hf CLI: search/download/upload models, datasets.

The English summary is being prepared. Showing the original description.

Converts documents into clean, chunked datasets suitable for embeddings and vector search. Produces chunked JSONL files with metadata, deduplication logic, and quality checks. Use when preparing "training data", "vector datasets", "document processing", or "embedding data".

The English summary is being prepared. Showing the original description.

Plan database structure changes and existing data updates for production systems. Provides migration patterns for adding columns and indexes, switching applications in stages, and documenting recovery steps while accounting for downtime and database locks.

• Adding columns or indexes to production tables
• Updating large datasets in batches
• Planning a staged column replacement without downtime

Guide language model training with TRL using example answers, preferred response pairs, or reward scores. The skill covers data preparation, method selection, reward model training, evaluation, and adjustments for memory limits or unstable training.

• Train a language model to follow instructions using example answers
• Train on preferred and rejected responses with DPO
• Build a reward model and run an RLHF training pipeline

Helps you use Unsloth for language model fine-tuning with LoRA and QLoRA, methods that adapt models with fewer trainable parameters. Supports API questions, implementation, and debugging using reference material drawn from official documentation.

• Learning how to fine-tune models with LoRA or QLoRA
• Looking up Unsloth features and APIs
• Investigating errors in training code

Get guidance for fine-tuning large language models with Axolotl, adapting them through additional training. Use YAML configuration examples, dataset references, and training methods such as LoRA to set up training and investigate problems.

• Preparing YAML configuration for model fine-tuning
• Exploring LoRA and QLoRA training methods
• Checking dataset formats and API usage

Track machine learning experiments in W&B, record training settings and metrics, and compare results in dashboards. Automate searches for training settings and keep version histories for models and datasets.

• Comparing accuracy across training runs
• Monitoring training progress in dashboards
• Searching for better training settings

Prepare text for language models by splitting it into tokens, small units represented by numbers. The skill covers batch processing, training custom vocabularies, tracking positions in the original text, and adjusting input lengths.

• Preparing large text collections for language models
• Training a custom vocabulary from text files
• Mapping model predictions back to original text

Set up GPU servers on Lambda Labs for machine learning training and inference. Follow workflows for SSH access, persistent storage, Python API operations, and distributed training that spreads work across multiple GPUs or servers.

• Set up a GPU server for long training jobs
• Fine-tune a large language model across multiple GPUs
• Keep datasets and checkpoints between sessions

Tune AgentDB memory use and search speed through quantization, which stores vectors more compactly, search indexes, caching, and batch operations. Includes configuration examples and benchmarks to help balance accuracy, speed, and storage as datasets grow.

• Reduce memory use in an AgentDB application
• Tune search speed while preserving accuracy
• Plan settings for millions of vectors

Tune AgentDB memory use and search speed through quantization, which stores vectors more compactly, search indexes, caching, and batch operations. Includes configuration examples and benchmarks to help balance accuracy, speed, and storage as datasets grow.

• Reduce memory use in an AgentDB application
• Tune search speed while preserving accuracy
• Plan settings for millions of vectors

Run Apify Actors, programs that collect web data, through the Composio integration. Retrieve structured results, create reusable tasks with preset inputs, and inspect execution logs for both quick collection jobs and longer runs.

• Collect restaurant data with a Google Places scraper
• Retrieve selected dataset fields in JSON format
• Create reusable tasks for recurring data collection

Analyze CSV, Parquet, and JSON files to answer questions through grouping, joins, cleaning, time series, and charts. Chooses DuckDB or Polars and a suitable processing location based on the task and available memory.

• Summarizing and joining data files
• Exploring trends and distributions
• Cleaning data before analysis

Create and update dashboards with charts, metrics, and tables drawn from connected data sources or attached files. Link displayed values to their sources so readers can inspect queries, calculations, and freshness while reviewing business indicators or recurring reports.

• Track KPIs and progress toward targets
• Visualize CSV, TSV, or JSON data
• Build recurring reports from connected data

Use Hugging Face Transformers to load pretrained models, generate text, and classify text, images, or audio. Prepare model inputs, control generation settings, and fine-tune models on custom datasets with the Trainer API.

• Prototype text, image, or audio classification
• Configure text generation and conversational responses
• Fine-tune a model on a custom dataset

Stress-test research plans by comparing possible scenarios, such as sample losses, unavailable materials, or methods failing to transfer. Record assumptions and evidence, compare response options, and define contingency plans and observable triggers for changing course.

• Plan responses to higher-than-expected sample losses
• Prepare backups for unavailable reagents or datasets
• Compare research options under resource constraints

Inspect and transform local DICOM medical imaging files, including metadata, image frames, and compression formats. Generate technical inventories and preview images, check installed codecs, and review pseudonymized derivatives without treating those checks as proof of de-identification.

• Create a technical inventory of local DICOM files
• Check compression support and frame memory needs
• Export a frame as a non-diagnostic preview

Analyze drug concentration and response data to calculate exposure, fit models of drug movement and effects, and simulate dosing schedules. Produces exploratory research results with explicit assumptions and diagnostic checks.

• Calculate exposure and half-life from concentration data
• Compare models of drug concentration over time
• Simulate dosing schedules for research

Build Python workflows for filtering, grouping, joining, and reshaping table data with Polars. Get guidance on migrating pandas code and using lazy evaluation, which optimizes a processing plan before execution, for large datasets.

• Filtering and summarizing large CSV files
• Migrating pandas processing code to Polars
• Joining and reshaping multiple tables