Skip to content
cccskills
FreePublished on GitHub

source-ingest

Fetch and normalize supported source-tutorial inputs into local, traceable text artifacts.

The English summary is being prepared. Showing the original description.

WILLOSCAR/research-units-pipeline-skills
See how to install

Files included (2)

  • SKILL.md2.0 KB
  • scripts/run.py24.8 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Source Ingest

Triggers & routing

  • Trigger: source ingest, ingest sources, normalize tutorial sources, 网页抽取, 资料归一化.
  • Use when: source-tutorial 的 C1，需要把 sources/manifest.yml 中的网页/PDF/repo/docs 变成可追溯文本。
  • Skip if: source manifest 还没定，或来源尚未确认。
  • Network: required for remote URLs.
  • Guardrail: 只把成功抽取的内容当作有效 source；失败来源必须落盘记录，不能默默忽略。

Goal: normalize mixed source inputs into local tutorial-ready text while preserving provenance.

Inputs

  • sources/manifest.yml

Outputs

  • sources/index.jsonl
  • sources/provenance.jsonl

Supported kinds

  • webpage
  • pdf
  • markdown
  • repo
  • docs_site
  • video

Behavior

  • Accept local file paths or remote URLs from sources/manifest.yml.
  • For video, use transcript-first ingestion:
    • provided transcript_locator
    • Bilibili subtitles when available
    • otherwise fail clearly instead of pretending the watch page is usable text
  • Plain YouTube/Bilibili watch pages should not be modeled as kind: webpage.
  • Continue past optional-source failures and record warnings.
  • Fail closed when any source marked required: true cannot be ingested.

Script

Quick Start

  • uv run python .codex/skills/source-ingest/scripts/run.py --workspace <workspace>

All Options

  • --workspace <dir> (required)
  • --unit-id <U###>
  • --inputs <semicolon-separated>
  • --outputs <semicolon-separated>
  • --checkpoint <C#>

Examples

  • Ingest all listed sources:
    • uv run python .codex/skills/source-ingest/scripts/run.py --workspace <workspace>

Troubleshooting

Issue: repo or docs site ingests too much noise

Fix:

  • Narrow the manifest to the most relevant docs root or replace the source with a specific docs page.

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

_template_reference_first

Free

Internal template for creating or refactoring a skill into the repository's reference-first shape. **Trigger**: reference-first template, blueprint skill, create a reusable skill, refactor a script-heavy skill. **Use when**: you need a lean `SKILL.md`, explicit `references/`, machine-readable `assets/`, and a minimal deterministic `run.py`.

The English summary is being prepared. Showing the original description.

WILLOSCAR/research-units-pipeline-skills515Updated Oct 5, 2026

agent-survey-corpus

Free

Download a small corpus of open-access arXiv survey/review PDFs about agentic systems and extract text for style learning.

The English summary is being prepared. Showing the original description.

WILLOSCAR/research-units-pipeline-skills515Updated Oct 5, 2026

anchor-sheet

Free

Extract per-subsection “anchor facts” (NO PROSE) from evidence packs so the writer is forced to include concrete numbers/benchmarks/limitations instead of generic summaries.

The English summary is being prepared. Showing the original description.

WILLOSCAR/research-units-pipeline-skills515Updated Oct 5, 2026

appendix-table-writer

Free

Curate reader-facing survey tables for the Appendix (clean layout + high information density), using only in-scope evidence and existing citation keys.

The English summary is being prepared. Showing the original description.

WILLOSCAR/research-units-pipeline-skills515Updated Oct 5, 2026

argument-selfloop

Free

Build a final, machine-readable argument snapshot from current H3 sections and refresh section fingerprints before merge.

The English summary is being prepared. Showing the original description.

WILLOSCAR/research-units-pipeline-skills515Updated Oct 5, 2026

artifact-contract-auditor

Free

Audit one research Workspace for declared Unit outputs and Pipeline target Artifacts, writing `output/CONTRACT_REPORT.md`; use for mid-Run coverage snapshots or final delivery completeness, not deep provenance integrity.

The English summary is being prepared. Showing the original description.

WILLOSCAR/research-units-pipeline-skills515Updated Oct 5, 2026

See all skills by WILLOSCAR

Report a problem with this skill