跳到正文
cccskills
免费在 GitHub 公开

kafka-best-practices

Apache Kafka 4.2 best practices for event streaming, topic design, and producer-consumer patterns

中文简介正在准备中。当前显示原文说明。

baekenough/second-brain
查看安装方法

包含的文件（1）

  • SKILL.md1.7 KB

SKILL.md（原文）

安装前可以查看交给 AI 代理的指令内容。

Apache Kafka Best Practices

Version: Kafka 4.2.0. ZooKeeper is fully removed in Kafka 4.x — use KRaft mode for all cluster deployments.

Producer Patterns

Idempotent Producer (CRITICAL)

  • Enable enable.idempotence=true
  • Prevents duplicate messages
  • Requires acks=all, retries > 0, max.in.flight.requests.per.connection <= 5

Exactly-Once Semantics

  • Use transactional API: initTransactions(), beginTransaction(), commitTransaction()
  • For exactly-once end-to-end processing

Performance

  • Batching: linger.ms (wait for batch to fill)
  • Compression: compression.type=snappy or lz4
  • batch.size: 16KB default, tune based on message size

Consumer Patterns

Offset Management

  • Auto-commit: enable.auto.commit=true (at-least-once)
  • Manual commit: commitSync() or commitAsync() (better control)

Rebalancing

  • Cooperative sticky assignor: minimal rebalancing disruption
  • session.timeout.ms and heartbeat.interval.ms tuning

At-Least-Once vs Exactly-Once

  • At-least-once: default, idempotent processing required
  • Exactly-once: transactional consumer + producer

Topic Design

Partitioning

  • Partition count: based on throughput (MB/s ÷ partition throughput)
  • Key-based partitioning for ordering guarantees
  • More partitions = higher throughput (but more overhead)

Retention

  • Time-based: retention.ms
  • Size-based: retention.bytes
  • Log compaction: for changelog topics (cleanup.policy=compact)

References

评价

还没有评价。欢迎分享你的使用感受。

评论

无需登录即可发表。欢迎分享使用感受、提问，或说明在哪些环境下无法运行。

还没有评论。来写第一条吧。

提交后会立即公开。请勿填写邮箱、电话等个人信息。URL 最多 2 个。

0 / 1000 字

此表单受 reCAPTCHA 保护，适用 Google 的隐私权政策和服务条款。

同一仓库的技能

action-validator

免费

Pre-action boundary checking — validates agent tool calls against declared capabilities and task contracts

中文简介正在准备中。当前显示原文说明。

baekenough/second-brain152026年10月8日 更新

adaptive-harness

免费

Auto-detect project context and optimize harness — deactivate unused agents/skills, suggest missing experts, generate project profile

中文简介正在准备中。当前显示原文说明。

baekenough/second-brain152026年10月8日 更新

adversarial-review

免费

Adversarial code review using attacker mindset — trust boundary, attack surface, business logic, and defense evaluation

中文简介正在准备中。当前显示原文说明。

baekenough/second-brain152026年10月8日 更新

airflow-best-practices

免费

Apache Airflow best practices for DAG authoring, testing, and production deployment

中文简介正在准备中。当前显示原文说明。

baekenough/second-brain152026年10月8日 更新

alembic-best-practices

免费

Alembic migration patterns for naming conventions, safety checks, expand-contract, env.py configuration, and CI integration

中文简介正在准备中。当前显示原文说明。

baekenough/second-brain152026年10月8日 更新

ambiguity-gate

免费

Pre-routing ambiguity analysis — scores request clarity and asks clarifying questions when needed (inspired by ouroboros)

中文简介正在准备中。当前显示原文说明。

baekenough/second-brain152026年10月8日 更新

查看 baekenough 的全部技能

报告此技能的问题