64 skills · sorted by relevance
Build and evaluate Mamba models for long sequences using state-space models, which summarize past inputs in an internal state. Covers setup, pretrained text generation, differences between Mamba-1 and Mamba-2, and speed comparisons with Transformers.
Run machine learning workloads on Modal's cloud GPUs without managing servers. Guides Python configuration for training, inference, batch processing, and API deployment, including automatic scaling, persistent storage, scheduled jobs, and troubleshooting.
Reduce GPU memory use by loading language models in 8-bit or 4-bit formats with bitsandbytes and HuggingFace Transformers. Includes configuration examples, QLoRA fine-tuning of compressed models, and optimizers that reduce memory needed during training.
Run machine learning workloads on Modal's cloud GPUs without managing servers. Guides Python configuration for training, inference, batch processing, and API deployment, including automatic scaling, persistent storage, scheduled jobs, and troubleshooting.
Optimize transformer attention, the calculation that relates tokens to one another, using Flash Attention. The skill guides PyTorch integration, GPU memory reduction, performance measurement, and output comparisons for training or inference with long sequences.
Implement Mamba, a model architecture that processes sequences through an internal state with computation scaling linearly with input length. Covers pretrained text generation, Mamba-1 and Mamba-2 configuration, speed comparisons with Transformers, and GPU memory troubleshooting.
Helps reduce GPU memory use by loading large language models in 4-bit or 8-bit formats, a technique called quantization. Covers bitsandbytes configuration, memory checks, QLoRA fine-tuning, and training with memory-saving optimizers.
Apply HQQ quantization, which stores language model weights using fewer bits without calibration examples. Covers precision settings, inference backends, saving compressed models, deployment with HuggingFace Transformers or vLLM, and further training with LoRA.
Operate FastReID setup, configuration, datasets, model/inference APIs, training/evaluation workflows, deployment/export paths, and extension projects for person and vehicle re-identification.
The English summary is being prepared. Showing the original description.
Guides FastVideo video, image, and audio generation, config-first inference, serving, training, distillation, evaluation, and performance workflows.
The English summary is being prepared. Showing the original description.
Geração estruturada rápida e serving para LLMs com RadixAttention para cache de prefixos. Use para saídas JSON/regex, decodificação restrita, workflows com agentes que fazem chamadas de ferramentas, ou quando você precisa de 5× mais rápido que vLLM com compartilhamento de prefixos. Alimenta 300.000+ GPUs na xAI, AMD, NVIDIA e LinkedIn.
The English summary is being prepared. Showing the original description.
Acelere a inferência de LLMs usando especulative decoding, múltiplas cabeças Medusa e técnicas de lookahead decoding. Use ao otimizar velocidade de inferência (aceleração de 1,5-3,6×), reduzir latência em aplicações em tempo real ou fazer deploy de modelos com recursos computacionais limitados. Cobre modelos draft, atenção em árvore, iteração de Jacobi, geração paralela de tokens e estratégias de deploy em produção.
The English summary is being prepared. Showing the original description.
Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.
The English summary is being prepared. Showing the original description.
End-to-end 16S/ITS amplicon workflow from demultiplexed FASTQ to a consensus differential-abundance result, orchestrating cutadapt primer removal, per-run DADA2 ASV inference (learnErrors/mergeSequenceTables/removeBimeraDenovo), region-matched taxonomy assignment, a SEPP/Greengenes2 tree, alpha/beta diversity at a declared sampling depth (phyloseq/vegan, adonis2 paired with betadisper), compositional DA as a consensus of >=2 tools (ALDEx2/ANCOM-BC2) on unrarefied counts, and optional PICRUSt2 functional prediction gated on NSTI. Covers the stage-ordering decisions (primers before truncation, per-run error model, rarefy for diversity not DA, predicted potential not activity) and defers each per-step choice to the six microbiome skills. Use when staging an amplicon study end to end or chaining ASV inference, taxonomy, diversity, and differential abundance. For shotgun reads see workflows/metagenomics-pipeline.
The English summary is being prepared. Showing the original description.
Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.
The English summary is being prepared. Showing the original description.
Configure supervised fine-tuning for models that process images and text. Define which components remain unchanged, set LoRA adapters for updating selected model weights, choose image resolution limits, and investigate training runs whose evaluation results fail to improve.
Plan and configure large language model training with OpenRLHF to better match human preferences. Covers training method selection, distributed GPU setup, inference acceleration with vLLM, and troubleshooting memory or training stability issues.
Integrate Flash Attention into transformer models to reduce processing time and GPU memory use for long sequences. Covers PyTorch integration, advanced attention options, and benchmarks that compare speed, memory use, and output accuracy.
Use this skill when building, deploying, evaluating, debugging, or measuring latency for the DeepStream SOP Inference Microservice — a GPU-accelerated FastAPI service that detects whether operators perform assembly-line steps in order via event boundary detection (GEBD) plus VLM classification. Trigger even if the user does not name it: verify operator step sequence, detect missing or out-of-order SOP steps, score factory/work-cell video for procedure compliance, run VLM-based SOP checking on industrial cameras, or call /v1/chat/completions with a file, RTSP, or Basler camera. Also trigger for its internals: SOPVideoProcessor, DeepStream GEBD model (e.g. DDM) via Triton CAPI, nvds_custom_postprocess, Cosmos Reason 1/2 vLLM, SSE streaming, Kafka NvProto/JSON output, Basler/Pylon camera + emulation, Docker compose, chunk-level latency. Do NOT trigger for generic DeepStream pipelines, object detection/tracking, NIM imports, or video summarization.
The English summary is being prepared. Showing the original description.
Amplicon sequence variant (ASV) inference from 16S rRNA or ITS amplicon sequencing using DADA2. Covers quality filtering, error learning, denoising, and chimera removal. Use when processing demultiplexed amplicon FASTQ files to generate an ASV table for downstream analysis.
The English summary is being prepared. Showing the original description.
Infer orthologous genes and gene families across species using OrthoFinder3 (HOG-based phylogenetic orthology), SonicParanoid2, Broccoli, ProteinOrtho, OMA / FastOMA hierarchical orthologous groups, eggNOG-mapper, JustOrthologs, and TOGA whole-genome-alignment orthology. Use when building single-copy ortholog sets for phylogenomics, classifying co-orthologs and in/out-paralogs after gene duplication, propagating functional annotation via orthology with awareness of the ortholog conjecture, distinguishing speciation from duplication via gene-tree species-tree reconciliation, computing Quest-for-Orthologs benchmark performance, or running synteny-aware ortholog detection in WGD-affected lineages.
The English summary is being prepared. Showing the original description.
Universal Runtime best practices for PyTorch inference, Transformers models, and FastAPI serving. Covers device management, model loading, memory optimization, and performance tuning.
The English summary is being prepared. Showing the original description.
Create AI avatar and talking head videos via inference.sh CLI. Recommended: P-Video-Avatar (fastest, cheapest, built-in TTS). Also: OmniHuman, Fabric, PixVerse. Audio: Inworld TTS-2 (100+ languages, emotion steering for characters), ElevenLabs, Kokoro. Capabilities: audio-driven avatars, text-to-avatar, lipsync videos, talking head generation, virtual presenters, UGC content. Use for: AI presenters, explainer videos, virtual influencers, dubbing, marketing videos, UGC ads, gaming avatars, NPC dialogue. Triggers: ai avatar, talking head, lipsync, avatar video, virtual presenter, ai spokesperson, audio driven video, heygen alternative, synthesia alternative, talking avatar, lip sync, video avatar, ai presenter, digital human, ugc, ugc video, ugc ad, avatar ugc
The English summary is being prepared. Showing the original description.
Guides ChatGLM2-6B Transformers inference, local demos, FastAPI/OpenAI-compatible serving, C-Eval evaluation, and P-Tuning v2 fine-tuning.
The English summary is being prepared. Showing the original description.