Skip to content
cccskills
FreePublished on GitHub

document-chunker

Split documents into overlapping token chunks for RAG pipelines and LLM context windows. Zero dependencies.

The English summary is being prepared. Showing the original description.

See how to install

Files included (14)

  • SKILL.md2.1 KB
  • banner.png149.7 KB
  • document_chunker.py5.3 KB
  • en/SKILL.md2.0 KB
  • es/SKILL.md2.1 KB
  • ja/SKILL.md2.3 KB
  • ru/SKILL.md2.6 KB
  • SKILL.en.md2.0 KB
  • SKILL.es.md2.2 KB
  • SKILL.fr.md2.2 KB
  • SKILL.ja.md2.4 KB
  • SKILL.ru.md2.7 KB
  • SKILL.zh.md2.1 KB
  • zh/SKILL.md2.0 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

<img src="banner.png" width="100%" alt="document-chunker banner">

Deutsch — Offizielle Deutsch-Version / Documento Oficial en Deutsch.

Document Chunker (Deutsch)

Splits documents into overlapping token chunks. Optimized for RAG pipelines and LLM context windows. Zero dependencies — Python stdlib + re only.

Usage

As Library

from document_chunker import DocumentChunker

chunker = DocumentChunker(chunk_size=400, overlap=80)
chunks = chunker.chunk_text("Long text...")

for chunk in chunks:
    print(f"Chunk {chunk['chunk_id']}: {chunk['tokens']} tokens")

Chunking a File

chunks = chunker.chunk_document("document.md", source="My Project")

Chunking an Entire Directory

from document_chunker import chunk_corpus

chunks = chunk_corpus(["doc1.md", "doc2.txt"], source="Corpus")

CLI

python document_chunker.py document.md    # Single file
python document_chunker.py ./docs/        # Entire directory

Parameters

ParameterDefaultDescription
chunk_size400Max tokens per chunk
overlap80Overlapping tokens between chunks

Supported File Types

.txt, .md, .py, .sh

Änderungsprotokoll

1.0.0 (2026-03-12)

  • Ported from BACH system/tools/document_chunker.py

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

Akademisches Studien- und Fristenmanagement mit Quellenprüfung, Datenschutz und realistischer Handlungsplanung.

The English summary is being prepared. Showing the original description.

ellmos-ai/skills8Updated Oct 19, 2026

Aktives Lernen, Erarbeitung von Fachinhalten und strukturierte Erstellung von Synthesen.

The English summary is being prepared. Showing the original description.

ellmos-ai/skills8Updated Oct 19, 2026

Prüfungsvorbereitung, Selbsttests, Probeklausuren und systematische Fehleranalyse.

The English summary is being prepared. Showing the original description.

ellmos-ai/skills8Updated Oct 19, 2026

スティーブン・ヘイズによるアクセプタンス&コミットメント・セラピー(ACT):心理的柔軟性の6つのコアプロセスを備えたヘキサフレックス・モデル。

The English summary is being prepared. Showing the original description.

ellmos-ai/skills8Updated Oct 19, 2026

Acceptance & Commitment Therapy (ACT) nach Steven Hayes: Hexaflex-Modell mit den sechs Kernprozessen psychischer Flexibilität.

The English summary is being prepared. Showing the original description.

ellmos-ai/skills8Updated Oct 19, 2026

Terapia de Aceptación y Compromiso (ACT) según Steven Hayes: modelo Hexaflex con los seis procesos nucleares de la flexibilidad psicológica.

The English summary is being prepared. Showing the original description.

ellmos-ai/skills8Updated Oct 19, 2026

See all skills by ellmos-ai

Report a problem with this skill