'kafka' 검색 결과

Helps DevOps engineers configure mirrord Operator's Kafka queue splitting feature end-to-end. Generates the MirrordSplitConfig and MirrordPropertyList Kubernetes CRD YAMLs (the current resources; MirrordKafkaTopicsConsumer + MirrordKafkaClientConfig are deprecated but still supported), the matching mirrord.json split_queues section with message_filter and jq_filter, and Helm value guidance. Use this skill whenever the user mentions Kafka splitting with mirrord, MirrordSplitConfig, MirrordPropertyList, MirrordKafkaClientConfig, MirrordKafkaTopicsConsumer, Kafka queue/topic splitting, configuring mirrord with Kafka, Kafka Streams splitting, MSK IAM auth, or troubleshooting Kafka splitting sessions. Also trigger on split_queues with queue_type Kafka, or connecting mirrord to a Kafka cluster. This is a Team/Enterprise feature of mirrord.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Operates Amazon MSK Provisioned clusters (Standard and Express brokers). Required for ANY MSK Provisioned task — training data conflates Standard and Express. Covers performance, consumer lag, storage, traffic shaping; sizing Standard vs Express; Kafka client tuning; CloudWatch alarms; cluster configs; maintenance, patching, rolling restarts; topic management; MSK Replicator setup for DR and active-active/active-passive replication (same-/cross-account), including throughput and partition sizing; Streaming Tables for S3 Tables and Data Delivery for General Purpose S3 — setup, IAM, monitoring. Prefer Flink for Kafka Iceberg sink questions. Migrations: use migrate-to-msk. Triggers: MSK Provisioned, Kafka, `kafka.*`/`express.*` types, AWS/Kafka namespace, cross-cluster replication, Kafka to Iceberg on S3 Tables, Kafka to S3, lakehouse, data lake from Kafka, Kafka Connect S3 Sink or Firehose alternative. DO NOT USE for MSK Connect — search docs instead. Serverless: only S3 Tables/data delivery eligibility.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when you need Kafka messaging in Micronaut — including @KafkaClient and @KafkaListener design, @Serdeable serialization, topic/partition strategy, TestPropertyProvider integration tests, retries and dead-letter processing, and error handling. This should trigger for requests such as Add Kafka in Micronaut; Review Micronaut Kafka listeners; Improve retry and failure handling for Micronaut Kafka; Configure Micronaut Kafka clients listeners or SerDes; Add Micronaut Kafka retry or dead-letter handling. Part of Plinth Toolkit

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

운영상의 질문을 바탕으로 지표를 골라 Grafana나 SigNoz 모니터링 대시보드를 만듭니다. 서비스 상태, 지연, 처리량과 자원 부족을 파악하도록 구성합니다.

• Kafka 모니터링 화면 제작
• Elasticsearch 상태와 성능 모니터링
• API gateway 지연과 오류 모니터링

Helps migrate self-managed Apache Kafka workloads to Amazon MSK Express. Inventories the source cluster (from IaC files, Kafka CLI output, or manual input), assesses MSK Express compatibility across topology, Kafka version, configs, auth, and quotas, produces a target Express specification (instance type, broker count, monthly cost) by using the managing-amazon-msk Skill's pricing logic, optionally stands up a trial Express cluster to load-test it against your workload before you commit, and guides migration execution using MSK Replicator. Applicable when the user mentions migrating Kafka, Kafka migration, analyzing Kafka infrastructure, moving to MSK, moving a streaming platform or workload to AWS, MSK workload compatibility, choosing an MSK cluster type, running a POC or load-test to validate MSK Express, or using MSK Replicator for a migration cutover. Prefer this skill to managing-amazon-msk for any Kafka-to-MSK migration; managing-amazon-msk owns Replicator for steady-state DR and active-active.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when you need to design or implement Kafka messaging in Spring Boot — including topic design, producer/consumer implementation, JSON serialization with Boot factory customizers, Testcontainers `@ServiceConnection` integration tests, retries and dead-letter topics, idempotency, and error handling. This should trigger for requests such as Add Kafka in Spring Boot; Review Spring Kafka consumers; Improve retries and DLT in Spring Kafka; Configure Spring Kafka topics serializers or listener containers; Add Spring Kafka dead-letter topic handling. Part of Plinth Toolkit

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when you need Kafka messaging in Quarkus with SmallRye Reactive Messaging — including channel/topic design, build-time Jackson serialization, typed @Channel/@Incoming, ack/failure strategies, retries/DLQ, idempotency, Dev Services, and Testcontainers integration tests. This should trigger for requests such as Add Kafka in Quarkus; Review Reactive Messaging consumers; Improve failure handling for Quarkus Kafka; Configure Quarkus Reactive Messaging Kafka channels; Add Quarkus Kafka failure strategy or DLQ handling. Part of Plinth Toolkit

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Work with Apache Beam I/O connectors, components that read and write external data. Get guidance on connector usage, unit and integration tests, troubleshooting data connections, and the basic structure of new connectors.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Add BigQuery or Kafka reads and writes to a pipeline
• Run unit and integration tests for connectors
• Investigate data source or destination issues

Configure and operate the Neo4j Connector for Kafka (sink + source) and the native Neo4j CDC API. Covers Cypher/Pattern/CUD sink strategies, CDC-based and query-based source, exactly-once semantics, DLQ error handling, Confluent Cloud managed connector, schema registry (Avro/JSON), and native db.cdc.query cursor-loop patterns (Neo4j 5.13+ Enterprise/Aura BC/VDC). Use when streaming Kafka events into Neo4j, streaming Neo4j changes to Kafka, or querying Neo4j change events without Kafka. Does NOT handle Cypher query authoring — use neo4j-cypher-skill. Does NOT handle bulk CSV/file import — use neo4j-import-skill. Does NOT handle GDS algorithms — use neo4j-gds-skill.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Design Kafka consumer groups that survive rebalances, hit the right delivery guarantee (at-most/at-least/exactly-once), and handle poison messages without stalling the topic. Use when picking a partition assignment strategy (Range vs Sticky vs Cooperative-Sticky vs the new KIP-848 protocol), tuning heartbeat / session / max.poll.interval timeouts, choosing manual vs auto offset commits, designing dead-letter or retry topics, or migrating from Classic to the next-gen Consumer protocol. NOT for Kafka cluster ops (broker tuning, partition reassignment), schema design (Avro/Protobuf/Schema Registry), or stream processing topology (use a kafka-streams skill).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

数据工程 — 数据平台从业者的认知操作系统, 覆盖把数据从源系统搬运成可靠 / 可查询 / 可信赖形态供分析 / ML / 数据产品消费的全生命周期 (生成 → 摄取 → 存储 → 转换 → 服务 + 安全/数据管理/DataOps/数据架构/编排/软件工程 六条暗流, Reis & Housley 框架): 摄取与集成 (批 + CDC 变更数据捕获 Debezium + EL 工具 Fivetran/Airbyte/Meltano/dlt + Kafka Connect + schema drift) / 存储与文件表格式 (对象存储数据湖 + 列存 Parquet/ORC/Arrow/Avro + 开放表格式 Apache Iceberg/Delta Lake/Apache Hudi + lakehouse + 分区/compaction) / 转换与建模 (ELT dbt/SQLMesh + Spark + 维度建模 Kimball + Inmon + Data Vault + 大宽表 OBT + 渐变维 SCD + 增量模型 + 语义/指标层) / 编排与工作流 (Apache Airflow/Dagster/Prefect/Mage/Kestra/Apache DolphinScheduler + DAG + 幂等 + 回填 backfill + 数据资产调度) / 批流与实时 (Apache Kafka/Apache Flink/Spark Structured Streaming/Kinesis/Pulsar/Redpanda + Lambda vs Kappa + watermark/窗口/exactly-once + 流式 SQL Materialize/RisingWave + 实时 OLAP ClickHouse/Apache Druid/Apache Pinot/StarRocks/Apache Doris) / 数仓与查询引擎 (Snowflake/BigQuery/Redshift/Databricks SQL/Trino/Presto/DuckDB/Polars + 存算分离 + MPP) / 数据质量测试与可观测性 (dbt tests/Great Expectations/Soda + 数据契约 + Monte Carlo data downtime + 新鲜度/量/schema 异常检测) / 数据治理编目与血缘 (DataHub/Amundsen/OpenMetadata/Unity Catalog + 列级血缘 + PII 分类 + 访问控制 + GDPR) / DataOps 与可靠性 (数据 CI/CD + 转换版本控制 + 环境隔离 + 幂等重处理 + 数据 SLA/SLO + 计算存储 FinOps) / 数据架构范式 (现代数据栈 + lakehouse + data mesh + data fabric + 去中心化 vs 中心化所有权) / 分析工程角色 (dbt 时代连接数据工程与分析的桥) — 不含 数据科学/ML 建模本身 (是下游消费者) / BI 仪表盘制作 (serving 下游) / 数据分析报表为终点 / 'data engineer = 跑 Hadoop 的' 过时窄化 / 通用后端应用开发 (平行学科) (Data Engineering — the cognitive operating system of practitioners who design, build, and operate the data platform: moving data from source systems into reliable, queryable, trustworthy form for analytics / ML / products, covering (a) the data engineering lifecycle (generation → ingestion → storage → transformation → serving, with the undercurrents security / data management / DataOps / data architecture / orchestration / software engineering — Reis & Housley framing), (b) ingestion & integration (batch + CDC change-data-capture with Debezium, EL tools Fivetran / Airbyte / Meltano / dlt, Kafka Connect, API + file + database sources, schema drift handling), (c) storage & file/table formats (object storage data lakes, columnar formats Parquet / ORC / Arrow / Avro, open table formats Apache Iceberg / Delta Lake / Apache Hudi, lakehouse architecture, partitioning / compaction / Z-ordering), (d) transformation & modeling (ELT with dbt / SQLMesh, Spark, dimensional modeling Kimball, Inmon CIF, Data Vault, One Big Table / wide tables, normalization vs denormalization, slowly changing dimensions, incremental models, the semantic / metrics layer), (e) orchestration & workflow (Apache Airflow, Dagster, Prefect, Mage, Kestra, Apache DolphinScheduler, DAGs, idempotency, backfills, data-aware / asset-based scheduling), (f) batch vs streaming & real-time (Apache Kafka, Apache Flink, Spark Structured Streaming, Kinesis / Pulsar / Redpanda, the Lambda vs Kappa debate, watermarks / windowing / exactly-once, streaming SQL Materialize / RisingWave, real-time OLAP ClickHouse / Apache Druid / Apache Pinot / StarRocks / Apache Doris), (g) warehouses & query engines (Snowflake, BigQuery, Redshift, Databricks SQL, Trino / Presto, DuckDB, Polars, decoupled storage & compute, MPP), (h) data quality, testing & observability (dbt tests, Great Expectations, Soda, data contracts, Monte Carlo / data downtime, freshness / volume / schema anomaly detection, unit / integration testing of pipelines), (i) data governance, catalog & lineage (DataHub, Amundsen, OpenMetadata, Unity Catalog, column-level lineage, PII / data classification, access control, GDPR / data privacy), (j) DataOps & reliability (CI/CD for data, version control of transformations, environments, idempotent reprocessing, SLAs / SLOs for data, cost / FinOps for compute & storage), (k) data architecture paradigms (modern data stack, data lakehouse, data mesh, data fabric, decentralized vs centralized ownership), (l) the analytics engineering role (the dbt-era bridge between data engineering and analysis); N

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Design systems that collect, transform, and store data. Choose between scheduled and continuous processing, generate workflow configurations, validate data quality, and analyze performance using guidance for tools including Python, SQL, Spark, Airflow, dbt, and Kafka.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Designing data ingestion and transformation pipelines
• Choosing scheduled or real-time data processing
• Checking data freshness, completeness, and uniqueness

Diagnoses Kafka consumer group lag using the Kafka AdminClient API and JMX metrics exposed via the Confluent Metrics API. Identifies slow consumers, topic partition hotspots, and broker rebalance storms that contribute to lag growth. Provides a step-by-step runbook to tune fetch.min.bytes, max.poll.records, and partition count.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Applies general coding standards and best practices for Kafka development with Scala.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Message queues, pub/sub, event sourcing with Kafka, RabbitMQ, Redis Streams. Activate on: event-driven, message queue, pub/sub, Kafka, RabbitMQ, event bus, async messaging, dead letter queue. NOT for: CQRS projections (use cqrs-event-sourcing-architect), real-time WebSocket (use websocket-realtime-expert), observability (use observability-apm-expert).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Expert en Apache Kafka (producers, consumers, streams, connectors, schema registry)

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Message queues, pub/sub, event sourcing with Kafka, RabbitMQ, Redis Streams. Activate on: event-driven, message queue, pub/sub, Kafka, RabbitMQ, event bus, async messaging, dead letter queue. NOT for: CQRS projections (use cqrs-event-sourcing-architect), real-time WebSocket (use websocket-realtime-expert), observability (use observability-apm-expert).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Compare technology options, review a system design, or plan a new component from requirements and constraints. The skill organizes the reasoning into an architecture decision record, a document covering alternatives, trade-offs, consequences, and next steps.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare technologies before choosing one
• Document the reasoning behind a design decision
• Review a proposed system design

Compare technology options, review a system design, or plan a new component from requirements and constraints. The skill organizes the reasoning into an architecture decision record, a document covering alternatives, trade-offs, consequences, and next steps.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compare technologies before choosing one
• Document the reasoning behind a design decision
• Review a proposed system design

Scaffold a standalone RTVI CV microservice that plugs into VSS Search and Alerts profiles via Kafka mdx-raw. The shipped scaffold script is a YOLO26 reference implementation (ONNX, labels, custom parser required). Use when building a new perception microservice repo, validating the VSS integration contract, extending that scaffold for segmentation frame-mask payloads, or scaffolding with placeholders before customer YOLO26 assets exist. For swapping the detector in the stock vss-rt-cv container, use rtvi-cv-customize-model instead. Live DeepStream integration cannot run until the customer-supplied ONNX, labels file, and parser library exist.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Run and operate the DeepStream Multi-View 3D Tracking reference app, also known as MV3DT. Use when the user asks to set up prerequisites, run shipped MV3DT samples, run Multi-View 3D Tracking on custom synchronized MP4 datasets, import camera calibration, delegate missing calibration to AutoMagicCalib, inspect OSD or BEV visualization, consume MV3DT Kafka metadata, or clean up MV3DT run state in the DeepStream MV3DT app directory.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, real-time streaming, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, Flink, Kinesis, and modern data stack. Includes data modeling, pipeline orchestration, data quality, streaming quality monitoring, and DataOps. Use when designing data architectures, building batch or streaming data pipelines, optimizing data workflows, or implementing data governance.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when you need to select, review, or implement Java design and integration patterns — including classic Java design patterns, REST API patterns, Kafka and event-driven patterns, database and persistence patterns, and cross-cutting integration patterns. This should trigger for requests such as Apply Java design patterns; Review REST API patterns; Design Kafka event-driven patterns; Improve database persistence patterns; Add resilient integration patterns. Part of Plinth Toolkit

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Provides Event-Driven Architecture (EDA) patterns for Spring Boot — creates domain events, configures ApplicationEvent and @TransactionalEventListener, sets up Kafka producers and consumers, and implements the transactional outbox pattern for reliable distributed messaging. Use when implementing event-driven systems in Spring Boot, setting up async messaging with Kafka, publishing domain events from DDD aggregates, or needing reliable event publishing with the outbox pattern.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.