「data pipeline」の検索結果

Build and troubleshoot Cloudflare Basin analytics workflows with Basin Pipelines, Basin Catalog, and Basin SQL. Use for streaming data into R2 Iceberg tables, managing catalogs, or querying those tables; also use for requests using the former Data Platform, Pipelines, R2 Data Catalog, or R2 SQL names.

日本語の概要は準備中です。原文の説明を表示しています。

Data engineering patterns for ETL pipelines, data warehousing, Apache Spark, and data quality validation

日本語の概要は準備中です。原文の説明を表示しています。

Manages Fabric Dataflow Gen2 items, including creation, M editing, connections, output destinations, getDefinition and updateDefinition inspection, executeQuery, refresh history, and Gen1-to-Gen2 save-as upgrades via saveAsNativeArtifact. Use for Dataflow items or Power Query M definitions. For pipeline JSON use pipeline-migration; for Spark or Warehouse transformations use their own skills.

日本語の概要は準備中です。原文の説明を表示しています。

Build and troubleshoot Cloudflare Basin analytics workflows with Basin Pipelines, Basin Catalog, and Basin SQL. Use for streaming data into R2 Iceberg tables, managing catalogs, or querying those tables; also use for requests using the former Data Platform, Pipelines, R2 Data Catalog, or R2 SQL names.

日本語の概要は準備中です。原文の説明を表示しています。

Universal LaTeX document skill: create, compile, and convert any document to professional PDF with PNG previews. Supports resumes, reports, cover letters, invoices, academic papers, theses/dissertations, academic CVs, presentations (Beamer), scientific posters, formal letters, exams/quizzes, books, cheat sheets, reference cards, exam formula sheets, fillable PDF forms (hyperref form fields), conditional content (etoolbox toggles), mail merge from CSV/JSON (Jinja2 templates), version diffing (latexdiff), charts (pgfplots + matplotlib), tables (booktabs + CSV import), images (TikZ), Mermaid diagrams, AI-generated images, watermarks, landscape pages, bibliography/citations (BibTeX/biblatex), multi-language/CJK (auto XeLaTeX), algorithms/pseudocode, colored boxes (tcolorbox), SI units (siunitx), Pandoc format conversion (Markdown/DOCX/HTML ↔ LaTeX), and PDF-to-LaTeX conversion of handwritten or printed documents (math, business, legal, general). Compile script supports pdflatex, xelatex, lualatex with auto-detection, latexmk backend, texfot log filtering, PDF/A output, and verbosity control (--verbose/--quiet). Empirically optimized scaling: single agent 1-10 pages, split 11-20, batch-7 pipeline 21+. Use when user asks to: (1) create a resume/CV/cover letter, (2) write a LaTeX document, (3) create PDF with tables/charts/images, (4) compile a .tex file, (5) make a report/invoice/presentation, (6) anything involving LaTeX or pdflatex, (7) convert/OCR a PDF to LaTeX, (8) convert handwritten notes, (9) create charts/graphs/diagrams, (10) create slides, (11) write a thesis or dissertation, (12) create an academic CV, (13) create a poster, (14) create an exam/quiz, (15) create a book, (16) convert between document formats (Markdown, DOCX, HTML to/from LaTeX), (17) generate Mermaid diagrams for LaTeX, (18) create a formal business letter, (19) create a cheat sheet or reference card, (20) create an exam formula sheet or crib sheet, (21) condense lecture notes/PDFs into a cheat sheet, (22) create a fillable PDF form with text fields/checkboxes/dropdowns, (23) create a document with conditional content/toggles (show/hide sections), (24) generate batch/mail-merge documents from CSV/JSON data, (25) create a version diff PDF (latexdiff) highlighting changes between documents, (26) create a homework or assignment submission with problems and solutions, (27) create a lab report with data tables, graphs, and error analysis, (28) encrypt or password-protect a PDF, (29) merge multiple PDFs into one, (30) optimize/compress a PDF for web or email, (31) lint or check a LaTeX document for common issues, (32) count words in a LaTeX document, (33) analyze document statistics (figures, tables, citations), (34) fetch BibTeX from a DOI, (35) convert a Graphviz .dot file to PDF/PNG, (36) convert a PlantUML .puml file to PDF/PNG, (37) create a one-pager/fact sheet/executive summary, (38) create a datasheet or product specification sheet, (39) extract pages from a PDF (page ranges, odd/even), (40) check LaTeX package availability before compiling, (41) analyze citations and cross-reference with .bib files, (42) debug LaTeX compilation errors, (43) make a document accessible (PDF/A, tagged PDF), (44) create lecture notes or course handouts, (45) fill an existing PDF form (fillable fields or non-fillable with annotations), (46) extract text or tables from a PDF (pdfplumber, pypdf), (47) OCR a scanned PDF to text (pytesseract), (48) create a PDF programmatically with reportlab (Canvas, Platypus), (49) rotate or crop PDF pages (pypdf), (50) add a watermark to an existing PDF, (51) extract metadata from a PDF (title, author, subject).

日本語の概要は準備中です。原文の説明を表示しています。

Build event streaming and real-time data pipelines with Kafka, Pulsar, Redpanda, Flink, and Spark. Covers producer/consumer patterns, stream processing, event sourcing, and CDC across TypeScript, Python, Go, and Java. When building real-time systems, microservices communication, or data integration pipelines.

日本語の概要は準備中です。原文の説明を表示しています。

Data ingestion patterns for loading data from cloud storage, APIs, files, and streaming sources into databases. Use when importing CSV/JSON/Parquet files, pulling from S3/GCS buckets, consuming API feeds, or building ETL pipelines.

日本語の概要は準備中です。原文の説明を表示しています。

Comprehensive toolkit for developing with the CocoIndex library. Use when users need to create data transformation pipelines (flows), write custom functions, or operate flows via CLI or API. Covers building ETL workflows for AI data processing, including embedding documents into vector databases, building knowledge graphs, creating search indexes, or processing data streams with incremental updates.

日本語の概要は準備中です。原文の説明を表示しています。

Methodology for exploratory data analysis on scientific files. Decision frameworks by data type (tabular, sequence, image, spectral, structural, omics), quality assessment, report generation, format detection across 200+ formats. Use when given a data file for initial exploration or to pick an analysis before a pipeline.

日本語の概要は準備中です。原文の説明を表示しています。

Automate construction data processing using LLM (ChatGPT, Claude, LLaMA). Generate Python/Pandas scripts, extract data from documents, and create automated pipelines without deep programming knowledge.

日本語の概要は準備中です。原文の説明を表示しています。

Migrate Databricks workloads from classic compute to serverless compute. Use when migrating notebooks, jobs, pipelines, or Scala JARs (`spark_jar_task`) from classic clusters to serverless, checking if existing code is serverless-compatible, or writing new serverless-compatible code. Provides concrete fixes for the serverless Spark Connect architecture and guides the full migration. Not for classic DBR version upgrades or cluster configuration changes within classic compute.

日本語の概要は準備中です。原文の説明を表示しています。

Build automated PII detection and redaction pipelines using spaCy NER, Microsoft Presidio, and AWS Macie integration. Includes confidence scoring, custom entity type definitions, batch processing workflows, and multi-format document scanning for structured and unstructured data sources.

日本語の概要は準備中です。原文の説明を表示しています。

Applies the reasoning style of Daphne Koller (machine learning pioneer, co-founder of Coursera, founder and CEO of Insitro). Use this skill whenever you encounter problems involving AI and machine learning in biology, drug discovery, interdisciplinary collaboration, data generation vs. data mining, or transitioning from academia to industry. Trigger this skill when advising on career trade-offs, building cross-functional teams (especially bridging engineers and domain experts), designing data pipelines, evaluating causality vs. correlation, or applying AI to physical systems ('where bits meet atoms'). Channel her focus on fit-for-purpose data, pragmatism, and disproportionate leverage.

日本語の概要は準備中です。原文の説明を表示しています。

Design, implement, and validate reproducible machine-learning pipelines spanning data preparation, training, evaluation, registry, and deployment gates. Use when the user requests an ML pipeline, needs to turn model scripts into an orchestrated workflow, or provides pipeline components that must be connected safely.

日本語の概要は準備中です。原文の説明を表示しています。

Think and work like an expert Data Scientist. Use when a task calls for Data Scientist judgment. Reasons from CRISP-DM business estimands, leakage-safe sklearn Pipelines and nested CV, SQL/warehouse semantic metrics, A/B power and SRM/AA guardrails, causal DAG covariate discipline, and Model Cards/Datasheets while treating train-test leakage, Simpson's paradox, peeking, and PSI>0.25 drift as first-class failure modes.

日本語の概要は準備中です。原文の説明を表示しています。

Design environment strategy for testing across dev, CI, preview, staging, and production — Docker Compose test infrastructure, multi-stage Dockerfiles, seed-data lifecycle, per-PR preview environments, production parity, and external-dependency stubbing at the HTTP boundary. Use when: "set up test environment," "docker-compose for tests," "per-PR preview environment," "staging parity," "spin up test infra," "environment tiers." Not for: choosing mock-vs-stub-vs-fake per dependency — use service-virtualization; factory and fixture data patterns — use test-data-management; pipeline/Actions config — use ci-cd-integration. Related: test-data-management, ci-cd-integration, contract-testing, service-virtualization.

日本語の概要は準備中です。原文の説明を表示しています。

Data Version Control for ML projects. Track large datasets and models alongside Git, build reproducible ML pipelines, and run experiments with metric comparison. Works with any storage backend including S3, GCS, Azure, and local filesystems.

日本語の概要は準備中です。原文の説明を表示しています。

Dagster is a data pipeline orchestrator built around the concept of software-defined assets. Learn to define assets, ops, jobs, schedules, sensors, and resources for building maintainable data platforms.

日本語の概要は準備中です。原文の説明を表示しています。

Apache Spark is a distributed engine for batch, SQL and streaming data processing, used from Python through PySpark. Use when a user asks to process big data, run distributed computations, build ETL pipelines, analyse data at scale, read Parquet from S3, or stream from Kafka with PySpark.

日本語の概要は準備中です。原文の説明を表示しています。

Applies the reasoning style of Daphne Koller (machine learning pioneer, co-founder of Coursera, founder and CEO of Insitro). Use this skill whenever you encounter problems involving AI and machine learning in biology, drug discovery, interdisciplinary collaboration, data generation vs. data mining, or transitioning from academia to industry. Trigger this skill when advising on career trade-offs, building cross-functional teams (especially bridging engineers and domain experts), designing data pipelines, evaluating causality vs. correlation, or applying AI to physical systems ('where bits meet atoms'). Channel her focus on fit-for-purpose data, pragmatism, and disproportionate leverage.

日本語の概要は準備中です。原文の説明を表示しています。

Import structured data into Neo4j — LOAD CSV, CALL IN TRANSACTIONS, neo4j-admin database import full (offline bulk), apoc.load.csv/json, apoc.periodic.iterate, driver batch writes. Covers method selection, header file format, type coercion, null handling, ON ERROR modes, CONCURRENT TRANSACTIONS, pre-import constraint setup, and post-import validation. Use when importing CSV/JSON/Parquet files, migrating relational data to graph, or bulk-loading large datasets. Does NOT handle unstructured document/PDF/vector chunking pipelines — use neo4j-document-import-skill. Does NOT handle live app write patterns (MERGE/CREATE) — use neo4j-cypher-skill. Does NOT handle neo4j-admin backup/restore/config — use neo4j-cli-tools-skill.

日本語の概要は準備中です。原文の説明を表示しています。

Neo4j Graph Data Science (GDS) embedded plugin via Python client or Cypher — covers graphdatascience client 2.x, GraphDataScience, gds.graph.project.native, gds.graph.project.cypher, snake_case endpoints, graph catalog operations, stream/stats/mutate/write modes, memory estimation, PageRank, Louvain, WCC, FastRP, KNN, Node Similarity, ML pipelines, and cleanup. Use for Aura Pro, self-managed, local, or offline Neo4j DBMS with the GDS plugin installed. Does NOT cover Aura Graph Analytics GDS Sessions, AuraGraphDataScience, GdsSessions, gds.graph.project.remote, or AuraDB Cypher API projection/session management — use neo4j-aura-graph-analytics-skill. Does NOT handle Cypher authoring — use neo4j-cypher-skill. Does NOT cover driver setup — use neo4j-driver-python-skill or other driver skill.

日本語の概要は準備中です。原文の説明を表示しています。

Neo4j Python Driver v6 — driver lifecycle, execute_query, managed and explicit transactions, async (AsyncGraphDatabase), result handling, data type mapping, error handling, UNWIND batching, connection pool tuning, and causal consistency. Use when writing Python code that connects to Neo4j via GraphDatabase.driver, execute_query, execute_read, execute_write, AsyncGraphDatabase, neo4j.Result, or RoutingControl. Package name is `neo4j` (not neo4j-driver) since v6. Python >=3.10 required. Does NOT handle Cypher query authoring — use neo4j-cypher-skill. Does NOT cover driver upgrades or breaking changes — use neo4j-migration-skill. Does NOT cover GraphRAG pipelines (neo4j-graphrag package) — use neo4j-graphrag-skill.

日本語の概要は準備中です。原文の説明を表示しています。

Use when adding or changing a scheduled data or ML pipeline — Dagster assets, Prefect flows, Airflow DAGs or a cron'd entrypoint — idempotent partitioned stages, retries, backfills, quality gates, and testing the pipeline without the scheduler

日本語の概要は準備中です。原文の説明を表示しています。