Results for “big data”

Discovers requirements and designs a borderless open data lakehouse using Lakehouse for Apache Iceberg and BigQuery data agents. Use when architecting multi-cloud storage infrastructure (Cloud Storage, AWS S3, Azure Blob), establishing ingestion and AI serving subsystems, configuring Cross-Cloud Interconnect, or deploying Gemini Enterprise Agent Platform and BigQuery data agents. Don't use for single-cloud data warehouses, or when the focus is on Knowledge Catalog metadata governance and Spark-driven IDE analytics workflows (use google-cloud-solution-agentic-analytics-spark-knowledge-catalog instead).

The English summary is being prepared. Showing the original description.

Use when building data pipelines, ETL jobs, event processors, message-broker producers/consumers, application caching layers, database migrations, BigQuery queries, or event-sourcing handlers in TypeScript. Triggers on edits to indexer/ETL/migration code, Prisma or Drizzle schemas, BigQuery integration code, RabbitMQ/Kafka/SQS/BullMQ producer or consumer code, Redis cache wrappers, or mentions of "data engineering", "ETL", "pipeline", "indexer", "event sourcing", "data warehouse", "data migration", "message queue", "RabbitMQ", "Kafka", "SQS", "BullMQ", "Redis cache", or "caching".

The English summary is being prepared. Showing the original description.

Use when the user asks about revenue, payments, subscriptions, billing, CRM deals, support tickets, production database tables, or other data that PostHog does not collect natively. Also use when a query fails because a table does not exist or returns no results for expected external data. The data warehouse can import from SaaS tools (Stripe, Hubspot, etc.), production databases (Postgres, MySQL, BigQuery, Snowflake), and other arbitrary data sources. Covers checking existing sources, identifying the right source type, and guiding the setup.

The English summary is being prepared. Showing the original description.

Use this skill for processing and analyzing large tabular datasets (billions of rows) that exceed available RAM. Vaex excels at out-of-core DataFrame operations, lazy evaluation, fast aggregations, efficient visualization of big data, and machine learning on large datasets. Apply when users need to work with large CSV/HDF5/Arrow/Parquet files, perform fast statistics on massive datasets, create visualizations of big data, or build ML pipelines that do not fit in memory.

The English summary is being prepared. Showing the original description.

Plan or run quality checks and peak calling, which identifies enriched genomic regions, for ChIP-seq, CUT&RUN, and CUT&Tag. The workflow accounts for controls and replicates and prepares signal tracks, quality reports, and differential binding designs.

• Check data quality and enrichment signals
• Choose narrow or broad peaks for the biological target
• Create bigWig tracks and quality dashboards

BigQuery/Snowflake接続、EDA、可視化、Marimoノートブック作成を行うサブエージェント。 データ分析関連の4つのルール（data_analysis, visualization, notebook, marimo_variable_naming）を統合。 「データ分析して」「BigQueryに接続」「EDAを実行」「Marimoで分析」等のリクエストで発動。

The English summary is being prepared. Showing the original description.

Algorithms and data structures from first principles through advanced analysis. Covers sorting (bubble, insertion, selection, merge, quick, heap, radix), searching (linear, binary, BFS, DFS, Dijkstra, A*), fundamental data structures (arrays, linked lists, stacks, queues, hash tables, trees, heaps, graphs, tries), complexity analysis (Big-O, Big-Omega, Big-Theta, amortized), recurrence relations, and algorithm design paradigms (divide-and-conquer, greedy, dynamic programming, backtracking). Use when analyzing, selecting, implementing, or comparing algorithms and data structures.

The English summary is being prepared. Showing the original description.

Stores, retrieves, and manages data as objects in Cloud Storage (Google Cloud Storage, or GCS) buckets. Use when you need to interact with Cloud Storage — create or configure buckets, upload, download, stream, or transfer data, organize objects with folders, generate signed URLs, control access (IAM, ACLs, public access prevention), set storage classes and tiering (Standard, Nearline, Coldline, Archive), manage cost and lifecycle, protect data (versioning, encryption/CMEK, retention and Bucket Lock, object holds, soft delete), host static websites, trigger Pub/Sub notifications on object changes, mount buckets as a file system (gcsfuse), or optimize storage performance at any scale. Covers the gcloud storage / gsutil CLI, JSON and XML APIs, client libraries, Terraform, and Cloud Storage MCP servers. Don't use for block storage (Persistent Disk), data warehousing/analytics (BigQuery), or databases (Cloud SQL, Spanner, Bigtable, Firestore).

The English summary is being prepared. Showing the original description.

Manages datasets, tables, and jobs in BigQuery. Use when you need to interact with BigQuery, run SQL queries, manage BigQuery resources (datasets, tables, views), or perform basic data ingestion and analysis.

The English summary is being prepared. Showing the original description.

Azure Data Lake Storage Gen2 SDK for Python. Use for hierarchical file systems, big data analytics, and file/directory operations. Triggers: "data lake", "DataLakeServiceClient", "FileSystemClient", "ADLS Gen2", "hierarchical namespace".

The English summary is being prepared. Showing the original description.

dlt (data load tool) is an open-source Python library that loads data from APIs, files and databases into warehouses and lakes, inferring the schema and tracking incremental state for you. Use when a user asks to build a data pipeline in Python, load a REST API into DuckDB, BigQuery, Snowflake or Postgres, set up incremental or merge loading, enforce a schema contract, or replace a hand-written ETL script or a hosted ELT connector.

The English summary is being prepared. Showing the original description.

dbt (data build tool) transforms data in your warehouse using SQL SELECT statements. Learn project setup, models, tests, documentation, incremental materializations, and integration with data warehouses like PostgreSQL, BigQuery, and Snowflake.

The English summary is being prepared. Showing the original description.

Apache Spark is a distributed engine for batch, SQL and streaming data processing, used from Python through PySpark. Use when a user asks to process big data, run distributed computations, build ETL pipelines, analyse data at scale, read Parquet from S3, or stream from Kafka with PySpark.

The English summary is being prepared. Showing the original description.

Route data-related requests to the right Monte Carlo skill or workflow. USE WHEN alerts, incidents, data broken, stale, coverage gaps, data quality, or any ambiguous data observability request.

The English summary is being prepared. Showing the original description.

Leverages BigQuery's built-in machine learning and GenAI capabilities for advanced data analytics. Use when you need to write SQL queries that perform time-series forecasting, predict values, detect outliers or anomalies, find key drivers, perform semantic search or vector search, classify text, calculate similarity, summarize content, translate language, evaluate models, filter by semantic conditions, measure the causal effect of an intervention, compute correlations between columns, detect change points or structural breaks, extract trend or seasonality components, or leverage generative AI capabilities in BigQuery. Do not use for general BigQuery dataset, table, or job management requests.

The English summary is being prepared. Showing the original description.

Assists in provisioning instances/tables, designing performant schemas, and querying data in Bigtable. Use when designing Bigtable row keys, configuring column families, writing SQL queries or client library code (Java, Go, Python) for Bigtable, or diagnosing performance/hotspotting issues. Also use when provisioning Bigtable clusters using gcloud or cbt CLIs. Don't use for generic Cloud SQL administration.

The English summary is being prepared. Showing the original description.

Design relational database schemas (and choose when to go NoSQL) that stay maintainable and fast. Use this skill whenever the user mentions tables, DDL, entities and relationships, normalization (1NF/2NF/3NF), primary and foreign keys, UUID vs bigint IDs, indexes (B-tree, composite, covering, partial), EXPLAIN, constraints (CHECK, UNIQUE, exclusion), transactions and isolation levels, migrations (Alembic, Prisma, Flyway, expand-contract, backfilling), or SQL vs NoSQL (MongoDB, DynamoDB, Cassandra, graph databases). Also trigger for "design the database", "model this domain", "which database should I use", or writing ORM models and migration files for PostgreSQL, MySQL, SQLite, or SQL Server.

The English summary is being prepared. Showing the original description.

Use when designing, reviewing, modifying, migrating, optimizing, securing, operating, troubleshooting, or decommissioning data architectures across data model exploration, SQL, ISO SQL conformance, NoSQL, vector, graph, document, time-series, NewSQL, search, in-memory, SQLite, CDC/Debezium, DBMS storage/query/recovery internals, database incident prevention, hardware acceleration, cache, big-data, lakehouse, and warehouse systems.

The English summary is being prepared. Showing the original description.

Evaluate code generation models with HumanEval, MBPP, and other benchmarks. Test generated programs, compare models across programming languages, and record pass@k scores, which estimate the chance that at least one of k solutions passes the tests.

• Comparing code generation models on shared benchmarks
• Measuring correctness across programming languages
• Evaluating models trained to follow instructions

Azure Storage Services including Blob Storage, File Shares, Queue Storage, Table Storage, and Data Lake. Answers questions about storage access tiers (hot, cool, cold, archive), when to use each tier, and tier comparison. Provides object storage, SMB file shares, async messaging, NoSQL key-value, and big data analytics. Includes lifecycle management. USE FOR: blob storage, file shares, queue storage, table storage, data lake, upload files, download blobs, storage accounts, access tiers, storage tiers, hot cool cold archive, storage tier comparison, when to use storage tiers, lifecycle management, Azure Storage concepts. DO NOT USE FOR: SQL databases, Cosmos DB (use azure-prepare), messaging with Event Hubs or Service Bus (use azure-messaging).

The English summary is being prepared. Showing the original description.

Azure Storage Services including Blob Storage, File Shares, Queue Storage, Table Storage, and Data Lake. Answers questions about storage access tiers (hot, cool, cold, archive), when to use each tier, and tier comparison. Provides object storage, SMB file shares, async messaging, NoSQL key-value, and big data analytics. Includes lifecycle management. USE FOR: blob storage, file shares, queue storage, table storage, data lake, upload files, download blobs, storage accounts, access tiers, storage tiers, hot cool cold archive, storage tier comparison, when to use storage tiers, lifecycle management, Azure Storage concepts. DO NOT USE FOR: SQL databases, Cosmos DB (use azure-prepare), messaging with Event Hubs or Service Bus (use azure-messaging).

The English summary is being prepared. Showing the original description.

Token数量过滤器。筛选器将总令牌数的样本保留在特定范围内。 本SKILL使用依赖data_juicer，请在调用前安装好python环境并安装data_juicer，你可用以下指令进行安装： pip install py-data-juicer 当用户提到token数量过滤、文本token数筛选、按token数过滤、文本分词过滤等需求时使用此skill。

The English summary is being prepared. Showing the original description.

特殊字符过滤器。过滤器将具有特殊字符比率的样品保持在特定范围内。 本SKILL使用依赖data_juicer，请在调用前安装好python环境并安装data_juicer，你可用以下指令进行安装： pip install py-data-juicer 当用户提到特殊字符过滤、特殊字符比例过滤、文本特殊字符检测、清理乱码文本等需求时使用此skill。

The English summary is being prepared. Showing the original description.

指定数值字段过滤器。根据指定的数值字段信息进行筛选。 本SKILL使用依赖data_juicer，请在调用前安装好python环境并安装data_juicer，你可用以下指令进行安装： pip install py-data-juicer 当用户提到数值字段过滤、字段范围筛选、数值范围过滤、元数据数值过滤、按数值过滤等需求时使用此skill。

The English summary is being prepared. Showing the original description.