Skip to content
cccskills
Official · OpenClawFreePublished on GitHubEnglish overview

knowledge-source-ingest

Track bounded collections such as chat channels, issue streams, feeds, and mailboxes across repeated imports. Retained manifests record stable item IDs and hashes, while overlap-aware comparisons identify changes and deletions for downstream Knowledge workflows.

OriginalShow the original description

Incrementally ingest enumerable external sources into Knowledge with canonical source identity, retained snapshot manifests, overlap-aware diffing, and reconciliation of changed or deleted items. Includes a generic manifest tool and a read-only Discrawl adapter.

openclaw/openclaw393,184 stars(GitHub 393,184 + cccskills 0)
Works withDiscord

Run score Not tested

See how to install

Good for

  • Comparing successive exports of a chat channel
  • Finding added, changed, or missing items in an issue stream
  • Retaining source coverage and snapshot manifests
  • Preparing source material for Knowledge ingestion

Overview

What it does

This skill prepares bounded external data for Knowledge workflows. Sources can include chat channels, issue lists, feeds, and mailboxes. It assigns a stable identity to each source and keeps snapshot manifests that record coverage, observation time, item counts, and other details. It compares repeated imports, including imports with overlapping date ranges, and identifies added, changed, deleted, unchanged, or unresolved items. A generic manifest tool and a read-only Discrawl adapter are included. The skill handles source identity and coverage; separate Knowledge ingestion workflows turn retained content into connected knowledge.

When it helps

Use it when you import the same source repeatedly and need to see what changed since the previous snapshot. It can help when collection ranges overlap or when you need to record whether a snapshot is complete or partial. Recurring ingestion jobs need to be created through knowledge-automation first. Documents and conversations should be passed to knowledge-ingest, and meetings to knowledge-meeting-ingest.

Example requests

  • “Create a manifest for this channel export and compare it with the previous snapshot.”
  • “Prepare a plan showing added, changed, and deleted items in this issue list.”
  • “Pass the meeting data I collected to the Knowledge meeting ingestion workflow.”

Notes

The Discrawl adapter is read-only and requires an approved way to acquire the data. This skill does not create recurring ingestion jobs. Do not put credentials or private raw content in Knowledge page text.

This overview was written by AI (OpenAI Codex) from the public SKILL.md. Check the original below for the exact behavior.

Document check

AI checked the SKILL.md for operations that need care and for setup requirements. The score comes only from actually running the skill (hands-on test).

SafetyAre sending data, deleting files, and running commands disclosed?
BPartly lackingThe adapter is read-only and credential handling is specified, but acquisition requires an approved reader.
Easy setupDoes it need few tools or API keys, and are they documented?
BPartly lackingNode scripts, an approved reader, and Knowledge tools are required; reader setup is not detailed.

Not counted in the score. Rated by AI (OpenAI Codex) as of Oct 26, 2026. Treat it as a guide and read the original before installing.

Disagree? Request a re-evaluation

Tell us which point is wrong and why. No login needed. We will check it against the SKILL.md, redo the evaluation, and show the result here.

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

Files included (17)

  • SKILL.md5.4 KB
  • references/assertions.md4.6 KB
  • references/conventions.md14.3 KB
  • references/identity.md4.8 KB
  • references/meeting-entity-example.md6.8 KB
  • references/profiles.md2.8 KB
  • references/publication.md22.1 KB
  • references/publish.md3.0 KB
  • references/recovery.md2.0 KB
  • references/setup.md1.7 KB
  • references/source-manifest.md1.4 KB
  • references/sources.md3.0 KB
  • references/verify.md2.7 KB
  • references/writing.md1.9 KB
  • scripts/chat-transcript.mjs8.8 KB
  • scripts/discrawl-normalize.mjs7.4 KB
  • scripts/source-manifest.mjs14.1 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Source Ingest Skill

Use this for bounded collections such as chat channels, issue streams, feeds, or mailboxes where later runs can overlap prior coverage. Read shared conventions, publication, and the manifest protocol. After acquisition, route documents and conversations through knowledge-ingest, which uses knowledge-entities for entity enrichment. Route meetings through knowledge-meeting-ingest. This skill owns source identity and coverage; the ingestion workflows turn the retained content into useful connected knowledge. For creating recurring ingestion jobs, use knowledge-automation first.

Contract

  • Resolve a stable provider/source key before reading content. Display names are labels, never identity.
  • Maintain one canonical source page with source_role: canonical, provider, and source_key. Reuse it on every run.
  • For each bounded observation, preserve the generated manifest as an attachment on a source page with source_role: snapshot, exact coverage, observation time, manifest hash, count, completeness, watermark, and dependency on the canonical source page.
  • Manifests retain stable item IDs and content hashes, not source text. Preserve original/exported content separately as evidence where authorized.
  • Query prior snapshot pages for the same provider/source key before planning. Download their manifests, verify attachment hashes, and pass all overlapping snapshots to the planner. Do not rely on conversation memory.
  • Reuse unchanged source bytes and existing records. Continue previously omitted enrichment from those retained sources when requested; an unchanged manifest does not mean entity enrichment already happened. Ingest added items. Reconcile changed items against existing pages/claims. Treat deletion as observed source state, not permission to erase Knowledge; correct or withdraw only when the evidence warrants it.
  • A complete snapshot can establish absence inside its exact coverage. Partial or unknown snapshots cannot; unresolved missing IDs remain unresolved.

Procedure

  1. Inspect knowledge_status and knowledge_ontology for the intended vault scope and the source fields, then search Knowledge for the canonical source and prior snapshots by provider/source key. If an owner-private ingestion unexpectedly resolves to a conversation scope or an empty vault after a restart, stop before writing and restore the intended authorized scope; never infer that prior coverage is absent from the wrong scope.
  2. Acquire a bounded export with an approved reader. For Discrawl, run scripts/discrawl-normalize.mjs with its configured read-only archive, exact channel ID when known, and [since,before) RFC3339 bounds. The normalized payload includes stable author IDs plus archive-observed display fields. Preserve the raw JSONL, and create a separate line-stable readable transcript when human review or line citations are needed.
  3. Generate a deterministic manifest with scripts/source-manifest.mjs snapshot. Attach it before publishing the snapshot page; record the returned SHA-256.
  4. Download all overlapping prior manifests and run scripts/source-manifest.mjs plan. Inspect covered/uncovered intervals and the added/changed/deleted/unchanged/unresolved lists.
  5. Test 3–5 substantive items or discussions, then ingest through knowledge-ingest (or knowledge-meeting-ingest for meetings). For chat, group related messages with their reply context; use knowledge-entities to enrich relevant people, companies, projects and concepts from their contributions. Carry supported developments into State, dated timelines and contextual back-links, not only a channel digest. Use stable external IDs in source citations and the retained snapshot as a dependency. Reuse its originals rather than uploading one copy per discussion.
  6. Publish or update derived records, then publish the snapshot page. A snapshot is complete only when acquisition and manifest creation covered the declared interval; derived extraction may still be partial and must be reported as such.
  7. Read back the canonical page, snapshot page, attachment, derived records, and source-dependent connections. Verify hashes and exact coverage.

Discrawl example

node scripts/discrawl-normalize.mjs --config /approved/discrawl.yaml \
  --channel 1234567890 --since 2026-09-21T00:00:00-07:00 \
  --before 2026-09-28T00:00:00-07:00 --output messages.jsonl
node scripts/source-manifest.mjs snapshot --provider discord \
  --source-key discord:GUILD_ID:1234567890 \
  --coverage-start 2026-09-21T00:00:00-07:00 \
  --coverage-end 2026-09-28T00:00:00-07:00 \
  --input messages.jsonl --output manifest.jsonl.gz --summary snapshot.json
node scripts/source-manifest.mjs plan --current manifest.jsonl.gz \
  --previous prior-1.jsonl.gz --previous prior-2.jsonl.gz --output plan.json

Do not include credentials, archive paths, or raw private content in Knowledge page prose. No scheduler, connector daemon, or plugin-owned ingestion job is created; the skill and scripts support an authorized agent-run workflow.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

1password

Official · OpenClawFreeSummary

Set up and authenticate the 1Password CLI, then supply stored passwords and API keys to commands or configuration. The skill covers desktop integration, service accounts, and interactive sign-in while keeping secrets out of chat and logs.

  • Setting up the 1Password CLI
  • Supplying API keys to commands
  • Using service account authentication in CI
openclaw/openclaw393.2KUpdated Oct 26, 2026

acp-router

Official · OpenClawFreeSummary

Route requests from OpenClaw to external coding agents such as Claude Code or Cursor. Start or continue sessions using ACP, a protocol for connecting agents, or relay prompts directly through acpx while preserving conversation context.

  • Starting a Claude Code session in a thread
  • Continuing work with an external coding agent
  • Relaying prompts directly through acpx
openclaw/openclaw393.2KUpdated Oct 26, 2026

add-model-provider

Official · OpenClawFreeSummary

Add a model provider to OpenClaw and verify it with a real response. The skill validates configuration, keeps credentials in environment or file references, and reports test results and any default model changes.

  • Adding a model provider to OpenClaw
  • Configuring credentials through environment or file references
  • Checking whether a model returns a real response
openclaw/openclaw393.2KUpdated Oct 26, 2026

agent-transcript

Official · OpenClawFreeSummary

Prepare an agent activity transcript for a GitHub pull request or issue. Extract relevant prompts, decisions, and results from local logs, remove sensitive material, and preview the edited record before separately authorized publication.

  • Preparing an agent activity record for a pull request
  • Extracting task-relevant prompts and results from local logs
  • Previewing a transcript with sensitive information removed
openclaw/openclaw393.2KUpdated Oct 26, 2026

apple-notes

Official · OpenClawFreeSummary

Manage Apple Notes on macOS through memo, a command-line tool. Create, browse, search, edit, delete, and move notes between folders, or export selected notes to HTML or Markdown for use outside the app.

  • Creating notes with titles
  • Finding notes by folder or approximate search
  • Editing notes and organizing them into folders
openclaw/openclaw393.2KUpdated Oct 26, 2026

apple-reminders

Official · OpenClawFreeSummary

Manage Apple Reminders from your Mac with remindctl, a terminal tool. View tasks by date, add or edit reminders, organize lists, and mark tasks complete or delete them, including personal to-dos that sync to iOS.

  • Review today’s tasks and overdue reminders
  • Add personal to-dos with due dates
  • Organize tasks visible on an iPhone or iPad
openclaw/openclaw393.2KUpdated Oct 26, 2026

See all skills by openclaw

Report a problem with this skill