“parquet”的搜索结果

Analyze CSV, Parquet, and JSON files to answer questions through grouping, joins, cleaning, time series, and charts. Chooses DuckDB or Polars and a suitable processing location based on the task and available memory.

中文简介正在准备中。当前显示英语简介。

• Summarizing and joining data files
• Exploring trends and distributions
• Cleaning data before analysis

Explore Hugging Face datasets through the Dataset Viewer API. Preview and retrieve rows, search text, filter records, inspect sizes and column statistics, and find Parquet download links for further analysis or data extraction.

中文简介正在准备中。当前显示英语简介。

• Preview dataset contents and data splits
• Search text and filter dataset rows
• Inspect dataset sizes, statistics, and Parquet links

Parquet格式化器。用于加载和格式化parquet类型的文件。 本SKILL使用依赖data_juicer，请在调用前安装好python环境并安装data_juicer，你可用以下指令进行安装： pip install py-data-juicer 当用户提到Parquet格式化、Parquet数据加载、读取Parquet文件、Parquet转JSONL等需求时使用此skill。

中文简介正在准备中。当前显示原文说明。

Use DuckDB from MATLAB via Database Toolbox (R2026a+) as a non-math operations engine on large tabular files (CSV/Parquet/JSON) and as a zero-config embedded database. Use when connecting to DuckDB, querying CSV, Parquet, and JSON files directly with SQL, reducing or profiling large data before MATLAB analysis, creating portable development databases, or installing DuckDB extensions. Triggers on: DuckDB, duckdb(), large CSV/Parquet/JSON, file too large for readtable, filter/aggregate at source, deduplicate, reduce before analysis, profile large file, persistent file import, analytical engine, SQL on CSV, SQL on Parquet, SQL on JSON, query CSV with SQL, query Parquet with SQL, run SQL on files, SQL queries on files, query files directly, SQL without database, in-process SQL.

中文简介正在准备中。当前显示原文说明。

Use the GeoParquet specification and bundled inspection helpers to author, validate, distribute, or safely review geospatial Parquet data, including GeoParquet 2.0 vector metadata, external writer choices, and the alpha Parquet Raster proposal.

中文简介正在准备中。当前显示原文说明。

Use Dask to process tables, numerical arrays, and file collections in pieces across CPU cores or machines. Helps choose data structures, size memory-friendly chunks, coordinate dependent tasks, and diagnose slow or memory-heavy workflows.

中文简介正在准备中。当前显示英语简介。

• Aggregate CSV or Parquet files that exceed available memory
• Process large scientific arrays in chunks
• Clean JSON records and logs before analysis

Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.

中文简介正在准备中。当前显示英语简介。

• Preprocessing datasets larger than memory
• Running batch predictions across multiple GPUs
• Preparing image and tabular data for training

Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.

中文简介正在准备中。当前显示英语简介。

• Preprocessing datasets larger than memory
• Running batch predictions across multiple GPUs
• Preparing image and tabular data for training

DuckDBを使用した大規模データ分析の専門スキル。DuckDBのアーキテクチャ、SQL構文、パフォーマンス最適化、各種ファイル形式（CSV、Parquet、JSON等）の効率的な読み込み・書き出しを熟知。メモリ効率の良い分析、複雑なクエリの最適化、データパイプライン構築を支援。Use when analyzing large datasets, querying CSV/Parquet/JSON files directly, building data pipelines, or optimizing analytical queries with DuckDB.

中文简介正在准备中。当前显示原文说明。

Analyze large scientific tables with Vaex, including datasets that exceed available memory. Build calculated columns, filter rows, compute statistics, visualize aggregated data, and convert files while identifying operations that may still require substantial RAM.

中文简介正在准备中。当前显示英语简介。

• Computing statistics on tables larger than available memory
• Creating calculated features without storing full columns
• Visualizing large datasets with aggregated grids

Build Python workflows for filtering, grouping, joining, and reshaping table data with Polars. Get guidance on migrating pandas code and using lazy evaluation, which optimizes a processing plan before execution, for large datasets.

中文简介正在准备中。当前显示英语简介。

• Filtering and summarizing large CSV files
• Migrating pandas processing code to Polars
• Joining and reshaping multiple tables

Build Apache Spark pipelines for large datasets, including transformations, aggregations, and streaming analysis. Get PySpark or Scala code, partitioning strategies for dividing data across workers, and configuration advice to investigate slow jobs and resource usage.

中文简介正在准备中。当前显示英语简介。

• Build a pipeline to transform and aggregate large datasets
• Investigate slow Spark jobs and uneven data distribution
• Tune memory settings and data partitions

Use when running analytical SQL over Parquet/CSV/JSON without a warehouse, replacing pandas for data wrangling, joining S3 data in-place, building local data marts, or embedding OLAP into an app. Triggers: read_parquet/read_csv setup, partitioned dataset queries, hive partitioning, glob patterns for S3, COPY TO export, attach Postgres/MySQL, UDFs in Python/R, MotherDuck cloud sync, columnar performance vs row stores. NOT for OLTP workloads (concurrent writes), distributed analytics at petabyte scale (use Spark/Trino), or vector search (use pgvector/Lance).

中文简介正在准备中。当前显示原文说明。

Use when running analytical SQL over Parquet/CSV/JSON without a warehouse, replacing pandas for data wrangling, joining S3 data in-place, building local data marts, or embedding OLAP into an app. Triggers: read_parquet/read_csv setup, partitioned dataset queries, hive partitioning, glob patterns for S3, COPY TO export, attach Postgres/MySQL, UDFs in Python/R, MotherDuck cloud sync, columnar performance vs row stores. NOT for OLTP workloads (concurrent writes), distributed analytics at petabyte scale (use Spark/Trino), or vector search (use pgvector/Lance).

中文简介正在准备中。当前显示原文说明。

Answers plain-word questions from a repo's CSV, JSON, Parquet and YAML files, or a .duckdb file, with DuckDB, and shows the SQL it ran. Read-only. Trigger to query, count, join, filter or summarise data files, including analytics CSVs exported from a video platform; to build a Parquet or DuckDB cache; to check no .duckdb, .wal or .parquet cache is committed or stale; when a query hits a locked database file; or say duckrunner (ask, cache, check, refresh). Needs the user-installed duckdb Python package; never a database server. DuckDB docs, S3 and MotherDuck are the duckdb-skills plugin's; outside research researchscribe's.

中文简介正在准备中。当前显示原文说明。

Work with geographic points, lines, and areas in Python using GeoPandas. Guidance and local audit tools help check coordinate systems, geometry validity, spatial join matches, and export settings before trusting or sharing analysis results.

中文简介正在准备中。当前显示英语简介。

• Checking geographic data before analysis
• Reviewing coordinate systems and transformations
• Auditing matches between points and areas

Create and manage Arize datasets for evaluation and experiments using the ax CLI. Import records from files, append examples, inspect dataset versions, and export a chosen version for offline analysis, with checks for field consistency and export completeness.

中文简介正在准备中。当前显示英语简介。

• Creating evaluation datasets from files
• Adding examples to an existing test set
• Inspecting dataset metadata and versions

动态统计Excel总行数，当数据量过大（≥10000行）时自动转换为Parquet格式加速读取，并对指定目标列进行条件筛选、分类汇总与结果导出，适用于超大体积Excel文件的快速读取与统计分析。

中文简介正在准备中。当前显示原文说明。

读取多 Sheet Excel 文件并统计规模，支持大文件向 Parquet 格式转换、分类数据统计及可视化报告生成。

中文简介正在准备中。当前显示原文说明。

万行以上 Excel 数据集的高性能分析引擎。提供 openpyxl read_only 流式读取（iter_rows 支持 10 万行以上）、Parquet 转换加速、内存优化、分块处理和大文件写入模式。**遇到以下任一情况就主动使用本 skill**：①数据行数 ≥ 10k（由 sn-da-excel-workflow 的行数评估步骤触发）；②用户出现触发词：大文件 / 大数据量 / 性能优化 / 内存不足 / OOM / 百万行 / 十万行 / 流式读取 / Parquet / 分块处理 / large file / big data / streaming read / chunked processing；③直接使用 pd.read_excel() 导致超时或内存溢出；④用户明确要求对大规模数据集进行高性能处理。仅不用于：小于 10k 行的常规 Excel 分析（使用 sn-da-excel-workflow 即可）。

中文简介正在准备中。当前显示原文说明。

当Excel文件总行数超过1万行时，通过转换为Parquet格式提升读取性能，提取目标指标并计算最大值，最后将结果输出为Excel并对特定行进行高亮标注。

中文简介正在准备中。当前显示原文说明。

Migrate a whole ArcGIS Hub site into a PortalJS Arc portal end-to-end. Harvests the Hub /data.json (DCAT-US) inventory, exports every FeatureService layer through the ArcGIS REST query API with resultOffset paging, converts each to the serverless dual tier (PMTiles render + GeoParquet query) with tabular items to Parquet, pushes everything to Cloudflare R2 via Git LFS, appends dual-tier datasets.json entries, and writes a source-vs-derived parity report. Use to move a City or sector ArcGIS Hub open-data portal onto PortalJS with no server-side compute.

中文简介正在准备中。当前显示原文说明。

Auto-ingest a geospatial file (GeoJSON, Shapefile, GeoPackage, KML/KMZ, FlatGeobuf, CSV-with-geometry) into a PortalJS portal on the user's own machine, with no server. Normalizes CRS to EPSG:4326, derives a PMTiles render tier and a GeoParquet query tier, pushes all three artifacts to Cloudflare R2 via Git LFS, and appends one dual-tier datasets.json entry the showcase auto-renders. Use when the source is a vector geo format that needs a map or spatial-query view.

中文简介正在准备中。当前显示原文说明。

Guide users or agents to the correct MATLAB tool for processing large tabular data in file-based formats (CSV, Parquet, delimited text, spreadsheets, MDF) that may not fit in memory. Use when a user or agent mentions large files, big data, out-of-memory errors, OOM, scaling up, tall arrays, datastores, or needs to process multiple tabular files. Covers the decision between datastore + tall, datastore + transform, and parallel execution. Also use when a user or agent has working in-memory code (readtable, parquetread) that runs out of memory and needs a migration path. Also covers building custom datastore classes for proprietary or non-standard formats — use when the task requires subclassing matlab.io.Datastore, implementing a custom reader, building an extensible datastore, or integrating a new file format with tall arrays or parallel computing. Do NOT use for MAT files (use matfile instead).

中文简介正在准备中。当前显示原文说明。