43 skills · sorted by relevance
Find why MNN CPU inference is slow by examining threads, scheduling, memory layout, and the processor instruction path actually used. Operator-level measurements and comparisons with other inference frameworks help identify which layer needs improvement across ARM, x86_64, and RISC-V.
Run language models locally with llama.cpp on CPUs, Apple Silicon, or AMD and Intel GPUs. The skill guides model setup, memory-saving GGUF format choices, interactive chat, and serving models through an OpenAI-compatible API.
Guides you through running large language models on CPUs, Apple Silicon, and AMD or Intel GPUs. Covers choosing GGUF model files, reducing memory through quantization, and starting an OpenAI-compatible chat server.
Convert LLMs to GGUF and reduce model size through quantization, which stores weights at lower numerical precision. Prepare models for CPU or Apple Silicon inference, compare compression levels, and integrate them with local AI tools.
Convert language models to GGUF and reduce their size through quantization, which stores model weights at lower precision. Guides cover compression choices, local inference on CPUs and Apple Silicon, and integration with tools such as Ollama.
Diagnose errors and incorrect inference results in MNN's QNN backend for Qualcomm HTP processors. Compare intermediate calculation data against CPU results to locate the first faulty operator, distinguish bugs from precision limits, and guide operator support and performance fixes.
Improve MNN's Vulkan backend by separating CPU scheduling bottlenecks from GPU computation costs. It guides operator and shader changes, new feature integration with compatibility fallbacks, and device testing to verify both correctness and end-to-end performance.
Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.
Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.
Connect applications to NEAR AI Cloud for AI responses generated in protected computing environments. Use its OpenAI-compatible API and follow the verification flow for hardware proofs and response signatures to check which protected environment signed a response.
Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.
The English summary is being prepared. Showing the original description.
Create or update Kubernetes YAML configuration files for GKE Autopilot and Standard. Translates application requirements into manifests covering container permissions, CPU and memory allocation, health checks, storage, external routing, and GPU resources for AI model serving.
Guide MNN Metal backend development and optimization, from finding performance bottlenecks to changing GPU computation, combining operators, and adjusting execution scheduling. A task-based document index directs work toward relevant implementation rules, benchmarks, and correctness checks.
Executa inferência de LLM em CPU, Apple Silicon e GPUs consumer sem hardware NVIDIA. Use para edge deployment, Macs M1/M2/M3, GPUs AMD/Intel ou quando CUDA não está disponível. Suporta quantização GGUF (1,5-8 bits) para redução de memória e aceleração de 4-10× vs PyTorch em CPU.
The English summary is being prepared. Showing the original description.
Integrate Flash Attention into Transformer models to speed up attention, the calculation that relates input elements, while reducing GPU memory use. Covers PyTorch integration, performance measurement, output comparison, and a separate H100 optimization path.
Set up machine learning training and inference on NVIDIA DGX Spark. Get guidance on choosing containers and packages, resolving CUDA compatibility errors, and checking that PyTorch can access the GPU before starting a workload.
Guides you through setting up RWKV, generating text, processing long documents in chunks, and fine-tuning models. Explains how to carry context between steps and compare memory use with Transformer models.
GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.
The English summary is being prepared. Showing the original description.
Set up and optimize large language model inference, or response generation, on NVIDIA GPUs. Use quantization, batch processing, and multiple GPUs to improve serving performance, and expose models through a chat completion API.
Plan memory use and monitor long training jobs on NVIDIA DGX Spark. It guides troubleshooting for out-of-memory failures, tracks temperature and power alongside training logs, and helps assess competition between training and inference workloads.
Investigate SGLang runs that freeze across multiple GPUs. It uses stack traces, communication logs, GPU memory dumps, and comparisons between processes to locate state differences, trace their causes, and guide fixes.
Guides the use of RWKV, a language model architecture that carries information forward in a compact state. Covers streaming text generation, long document processing, fine-tuning, and memory comparisons with Transformers.
Optimize transformer attention, the calculation that relates tokens to one another, using Flash Attention. The skill guides PyTorch integration, GPU memory reduction, performance measurement, and output comparisons for training or inference with long sequences.
Helps reduce GPU memory use by loading large language models in 4-bit or 8-bit formats, a technique called quantization. Covers bitsandbytes configuration, memory checks, QLoRA fine-tuning, and training with memory-saving optimizers.