31 skills · sorted by relevance
Build text tokenizers that split multilingual text into small units for AI models. Learn how to train SentencePiece on raw text, configure BPE or Unigram splitting methods, and convert text to token IDs and back.
Build text tokenizers, which split text into units for AI models, directly from raw multilingual text. Learn custom vocabularies with BPE or Unigram, convert text to pieces or numeric IDs, and decode the results.
Prepare text for language models using HuggingFace Tokenizers, which splits text into model-readable units. Train custom vocabularies, process batches, track original text positions, and configure input lengths for use with transformers.
Prepare large text collections for AI models with fast tokenizers, which split text into model inputs. Train custom vocabularies, trace tokens to original text positions, adjust input lengths, and integrate the results with transformers.
Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.
The English summary is being prepared. Showing the original description.
Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.
The English summary is being prepared. Showing the original description.
Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.
The English summary is being prepared. Showing the original description.
Prepare text for language models by splitting it into tokens, small units represented by numbers. The skill covers batch processing, training custom vocabularies, tracking positions in the original text, and adjusting input lengths.
Prepare and analyze genomic intervals, regions defined by chromosome coordinates, with Gtars. Validate BED files, compare overlaps and coverage, combine region sets, and plan tokenization or reference storage while tracking data provenance and resource limits.
PCI DSS v4.0 payment card data security compliance, assessment, and implementation. Use for PCI DSS scoping, cardholder data environment (CDE) security, SAQ and ROC preparation, QSA engagement, tokenization, and merchant compliance.
The English summary is being prepared. Showing the original description.
Assessment of pseudonymization techniques and re-identification risk. Covers tokenization, hashing, encryption-based pseudonymization, and hybrid approaches. Includes re-identification risk scoring using the motivated intruder test, quantitative metrics (marketer, journalist, prosecutor models), and linkage attack resilience evaluation. References ENISA 2019 pseudonymization report.
The English summary is being prepared. Showing the original description.
Build Bengali search and AI features with guidance on consistent character encoding, word splitting, digit conversion, and safe text display. Includes patterns for cleaning training data, storing text, and handling Bengali alongside English.
Help secure payment systems using PCI DSS, the payment card data security standard. Covers encryption, log masking, access controls, and tokenization, which replaces card details with substitute values to reduce direct handling of sensitive data.
Add or verify model support in SGLang's Rust processor so chat requests produce the same prompt text and token IDs as Python. Use reference-generated test cases to find differences in tools, reasoning settings, conversation history, and errors.
High-performance toolkit for genomic interval analysis in Rust with Python bindings. Use when working with genomic regions, BED files, coverage tracks, overlap detection, tokenization for ML models, or fragment analysis in computational genomics and machine learning applications.
The English summary is being prepared. Showing the original description.
A high-performance Rust toolkit (with Python bindings and a CLI) for genomic interval analysis; use it when you need fast overlap queries, coverage track generation, genomic tokenization for ML, reference sequence verification, or fragment processing.
The English summary is being prepared. Showing the original description.
Audit applications and infrastructure handling payment card data against PCI DSS v4.0. Heavy emphasis on scope determination (the single most-leveraged variable) plus the engineering-relevant requirements — Req 3 (storage of CHD), Req 4 (transmission), Req 6 (secure SDLC), Req 7-8 (access), Req 10 (logging), Req 11 (testing), Req 12 (program). Use when the user mentions 'PCI,' 'PCI DSS,' 'PCI DSS 4.0,' 'payment card,' 'cardholder data,' 'CHD,' 'PAN,' 'PCI scope,' 'PCI compliance,' 'SAQ,' 'AoC,' 'attestation of compliance,' 'tokenization,' 'P2PE,' 'network segmentation for PCI,' or audits any system that stores, processes, or transmits payment card data.
The English summary is being prepared. Showing the original description.
High-performance toolkit for genomic interval analysis in Rust with Python bindings. Use when working with genomic regions, BED files, coverage tracks, overlap detection, tokenization for ML models, or fragment analysis in computational genomics and machine learning applications.
The English summary is being prepared. Showing the original description.
Rust-backed Python library for fast genomic token arithmetic and BED processing. High-performance BED I/O, interval set ops (intersect, merge, complement, subtract), region tokenization against a universe, universe construction. Use for preprocessing large BED collections and ML token vocabularies.
The English summary is being prepared. Showing the original description.
HuggingFace Transformers with biomedical LMs (BioBERT, PubMedBERT, BioGPT, BioMedLM) for scientific NLP: NER (genes, diseases, chemicals), relation extraction, QA, text classification, abstract summarization. Covers loading, biomedical tokenization, inference pipelines, fine-tuning. Alternatives: spaCy en_core_sci_lg (rule-based NER), Stanza (biomedical models), NLTK.
The English summary is being prepared. Showing the original description.
Applies the mental models and frameworks of Andrej Karpathy (deep learning, former Director of AI at Tesla, founding member of OpenAI, Eureka Labs). Use this skill whenever you are helping the user build neural networks from scratch, debug deep learning pipelines, evaluate AI agent workflows, design LLM apps, or navigate the transition to Software 3.0 (vibe coding). It is highly relevant for pedagogy (untangling complex knowledge), assessing AI capabilities vs. limitations (jagged intelligence, tokenization limits), and architectural decisions (end-to-end optimization vs. complex pipelines). Reach for this whenever discussing LLM training, autonomous systems, or AI-assisted coding.
The English summary is being prepared. Showing the original description.
Applies the mental models and frameworks of Andrej Karpathy (deep learning, former Director of AI at Tesla, founding member of OpenAI, Eureka Labs). Use this skill whenever you are helping the user build neural networks from scratch, debug deep learning pipelines, evaluate AI agent workflows, design LLM apps, or navigate the transition to Software 3.0 (vibe coding). It is highly relevant for pedagogy (untangling complex knowledge), assessing AI capabilities vs. limitations (jagged intelligence, tokenization limits), and architectural decisions (end-to-end optimization vs. complex pipelines). Reach for this whenever discussing LLM training, autonomous systems, or AI-assisted coding.
The English summary is being prepared. Showing the original description.
Use Neo4j GenAI Plugin ai.text.* functions and procedures for in-Cypher embedding generation, text completion, structured output, chat, tokenization, and batch ingestion. Covers ai.text.embed(), ai.text.embedBatch(), ai.text.completion(), ai.text.structuredCompletion(), ai.text.aggregateCompletion(), ai.text.chat(), ai.text.tokenCount(), ai.text.chunkByTokenLimit(), and provider configuration for OpenAI, Azure OpenAI, VertexAI, and Amazon Bedrock. Requires CYPHER 25. Replaces deprecated genai.vector.encode(). Use when writing pure-Cypher GraphRAG, embedding nodes in-graph, generating structured maps from prompts, or calling LLMs inside Cypher queries. Does NOT handle neo4j-graphrag Python library pipelines — use neo4j-graphrag-skill. Does NOT handle vector index creation/search — use neo4j-vector-index-skill.
The English summary is being prepared. Showing the original description.
Fintech engineering expertise — payment processing (Stripe, Plaid), PCI DSS compliance, financial data modeling (double-entry bookkeeping), fraud detection patterns, bank-grade security (encryption, tokenization), open banking APIs, cryptocurrency/blockchain integration, regulatory compliance (KYC/AML), and idempotent financial transaction design. Use for payment systems, banking apps, trading platforms, and fintech infrastructure.
The English summary is being prepared. Showing the original description.