Skip to content
cccskills

Results for “dedup”

284 skills · sorted by relevance

nemo-curator

FreeSummary

Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.

  • Clean web data for model training
  • Remove exact and near duplicates from large text collections
  • Redact personal information from training text
davila7/claude-code-templates32.8KUpdated Oct 18, 2026

nemo-curator

FreeSummary

Prepare training datasets for large language models with GPU-assisted cleaning. Covers duplicate removal, quality filtering, and personal information redaction, with workflows for text, images, video, and audio using NeMo Curator.

  • Clean web data for model training
  • Remove exact and near duplicates from large text collections
  • Redact personal information from training text
Orchestra-Research/AI-Research-SKILLs13.7KUpdated Oct 11, 2026

Curação de dados acelerada por GPU para treinamento de LLM. Suporta texto/imagem/vídeo/áudio. Recursos incluem deduplicação fuzzy (16× mais rápida), filtragem de qualidade (30+ heurísticas), deduplicação semântica, redação de PII, detecção NSFW. Escala em GPUs com RAPIDS. Use para preparar datasets de treinamento de alta qualidade, limpar dados web ou deduplicar grandes corpora.

The English summary is being prepared. Showing the original description.

artubss/SKILLS-CLAUDE-CODE11Updated May 17, 2026

GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.

The English summary is being prepared. Showing the original description.

huang-sh/DeepScience4Updated Jul 15, 2026

GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.

The English summary is being prepared. Showing the original description.

Lord1Egypt/awesome-skill-forge2Updated Jun 10, 2026

extractor

FreeSummary

Extract characters, settings, and plot-critical props from an episode script into structured records. Describe appearance, styling, spatial details, and lighting, then reuse existing entries and link them to the current episode to avoid duplicate production assets.

  • Listing characters and settings needed for an episode
  • Selecting only props that matter to the plot
  • Reusing existing assets without creating duplicates
chatfire-AI/huobao-drama16.4KUpdated Oct 6, 2026

Fix duplicate Nostr events in ClickHouse when using ReplacingMergeTree for NIP-33 addressable events (Kind 30000+). Use when: (1) Edited videos/events appear as duplicates, (2) Same d_tag shows multiple events with different IDs, (3) FINAL keyword doesn't deduplicate properly for parameterized replaceable events. The issue is that FINAL deduplicates by ORDER BY key (typically `id`), not by (pubkey, kind, d_tag).

The English summary is being prepared. Showing the original description.

divinevideo/divine-mobile266Updated Oct 19, 2026

Create, merge, and deduplicate BibTeX bibliography files using R packages (RefManageR, bibtex). Parse .bib files into structured R objects, merge multiple bibliographies with deduplication by DOI or title similarity, generate entries from DOI/ISBN/arXiv ID, and export clean sorted .bib files. Use when creating a new .bib file for an R Markdown or Quarto project, merging bibliographies from multiple collaborators, deduplicating a .bib that has grown through copy-paste accumulation, or generating BibTeX entries programmatically from DOIs or other identifiers.

The English summary is being prepared. Showing the original description.

pjt222/agent-almanac42Updated Oct 15, 2026

Build AI data deduplication systems. TRIGGERS - Use when user needs help with ai-deduplication related tasks.

The English summary is being prepared. Showing the original description.

lionelsimai/claude-skills-collection30Updated Feb 8, 2026

Router for the 101 SfSkills `data` skill packages. Data model, data movement and data quality: migrations, bulk loads, query optimisation, deduplicating at volume, archival. Ordinary-volume duplicate cleanup and prevention use salesforce-admin; come here for hundreds-of-thousands+ dedup or third-party tools. LDV architecture uses salesforce-architect. Use when the request mentions data model, data migration, data load, Data Loader, Bulk API, external id, deduplication at volume, archival, SOSL, cross-object search, sandbox seed data, SandboxPostCopy, native Data Seeding. Finds and opens the exact skill package to read; it does not contain the guidance itself.

The English summary is being prepared. Showing the original description.

PranavNagrecha/AwesomeSalesforceSkills19Updated Oct 4, 2026

Before creating a GitHub issue or epic, search for an existing related one and force a reuse / replace / new decision — so the issue-first flow can't silently mint a duplicate. The create-time half of the dedup gate (the pickup-time half is /issue-sanity-check). Use whenever you're about to open an issue/epic, or run /issue-dedup. Backed by a hard PreToolUse hook on issue_write create.

The English summary is being prepared. Showing the original description.

FriendlyInternet/nuxt-crouton10Updated Oct 18, 2026

dedup

Free

Find and merge duplicate or near-duplicate notes anywhere in a vault (rules, concepts, people, leads) with a deterministic scanner and a supersede-not-delete merge policy. Human review before any merge is written. Triggers: "/dedup", "clean up duplicates", "dedupe <folder>".

The English summary is being prepared. Showing the original description.

tonydzi/second-brain-starter-kit9Updated Oct 16, 2026

Build AI data deduplication systems. TRIGGERS - Use when user needs help with ai-deduplication related tasks.

The English summary is being prepared. Showing the original description.

Winbda/claude-skills-collection4Updated Apr 6, 2026

GPU-accelerated data curation for LLM training. Supports text/image/video/audio. Features fuzzy deduplication (16× faster), quality filtering (30+ heuristics), semantic deduplication, PII redaction, NSFW detection. Scales across GPUs with RAPIDS. Use for preparing high-quality training datasets, cleaning web data, or deduplicating large corpora.

The English summary is being prepared. Showing the original description.

ibragimov-oasis/vibe-coder3Updated Jun 24, 2026

Build a workflow that extracts and deduplicates threat indicators such as URLs and IP addresses, creates readable reports with links disabled, and prepares standardized STIX 2.1 data for sharing through MISP or TAXII.

  • Preparing threat reports with links disabled
  • Consolidating indicators from multiple sources
  • Distributing threat indicators through MISP or TAXII
mukul975/Anthropic-Cybersecurity-Skills34.7KUpdated Aug 31, 2026

Build a central vulnerability dashboard with DefectDojo to combine scanner findings, remove duplicates, and track remediation. Includes Docker deployment, automated scan imports from CI/CD pipelines, Jira ticket integration, and API queries for reporting metrics.

  • Combining findings from multiple security scanners
  • Importing scan results automatically from CI/CD pipelines
  • Connecting vulnerability findings to Jira tickets
mukul975/Anthropic-Cybersecurity-Skills34.7KUpdated Aug 31, 2026

Check names, aliases, and existing pages before saving content to a gbrain knowledge store. The skill reads likely matches to identify duplicates, links to existing information, and routes new material to the appropriate ingestion skills.

  • Check notes for duplicates before saving
  • Resolve different names for the same person or company
  • Review files before a bulk import
garrytan/gbrain31KUpdated Oct 19, 2026

sequence-load

FreeSummary

Find prospects by role, industry, company size, and location, then add approved contacts to an Apollo sequence, a series of outreach emails. The workflow enriches details, creates contacts with deduplication, and reports enrollment results.

  • Finding prospects by role and company criteria
  • Adding a batch of contacts to an Apollo sequence
  • Loading another batch into an existing sequence
anthropics/knowledge-work-plugins30.5KUpdated Oct 19, 2026

papercuts

FreeSummary

Record recurring setup and workflow problems that a repository can fix, so future contributors can avoid the same friction. Also review existing entries, remove duplicates, prioritize useful fixes, and preserve a history of resolved problems.

  • Recording confusing setup steps
  • Tracking recurring problems in repository scripts
  • Deduplicating and prioritizing friction reports
every-app/open-seo24.1KUpdated Oct 14, 2026

Convert Snowflake dbt SQL models to BigQuery through a bulk migration workflow. Preserve Jinja template logic while adapting configuration, explicit type conversions, JSON access, and deduplication patterns, with translated files saved in their directory structure.

  • Migrating Snowflake dbt models to BigQuery in bulk
  • Preserving dbt references and incremental processing logic
  • Adapting JSON, array, and STRUCT access during migration
google/skills21.4KUpdated Oct 17, 2026

Audit server-side advertising measurement by comparing browser and server events, checking duplicate handling, consent, and data quality. Reconcile platform diagnostics with source logs and business conversions, then return prioritized findings, owners, and verification steps.

  • Investigate missing or duplicated conversion events
  • Compare browser and server event data
  • Review consent and data handling before transmission
AgriciDaniel/claude-ads10.1KUpdated Oct 17, 2026

sarif-parsing

FreeSummary

Process SARIF files, a standard format for static analysis results, to filter alerts, remove duplicates, combine findings from multiple tools, and produce reports. It also supports comparing runs and integrating result checks into CI/CD pipelines.

  • Combine findings from multiple security scanners
  • Filter alerts by severity and remove duplicates
  • Compare earlier and current results for new issues
trailofbits/skills7.6KUpdated Oct 16, 2026

Detect and merge wiki pages representing the same concept under different names. Use for page-identity deduplication and consolidation. Destructive merges require confirmation; not for link repair or general linting.

The English summary is being prepared. Showing the original description.

Ar9av/obsidian-wiki3.6KUpdated Oct 18, 2026

Use when the user asks to "QA my conversion tracking before launch", "check my UTMs / pixel / event firing", "set up a tracking pre-flight", or "set the dedup rule so Meta and Google stop double-counting"; builds and fixes the measurement plumbing — conversion-event firing, UTM hygiene, cross-platform dedup rules, attribution-window alignment, and offline/iOS-ATT modeled-gap flags — as a pre-flight checklist plus a UTM/event-spec builder. Not for scoring R1/R2 — that is a scored veto in ad-account-auditor; not for account structure — use campaign-architect. 付费广告转化追踪QA/UTM规范/跨平台去重

The English summary is being prepared. Showing the original description.

aaron-he-zhu/aaron-marketing-skills3KUpdated Oct 19, 2026