'cpu inference' 검색 결과

Find why MNN CPU inference is slow by examining threads, scheduling, memory layout, and the processor instruction path actually used. Operator-level measurements and comparisons with other inference frameworks help identify which layer needs improvement across ARM, x86_64, and RISC-V.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate slow CPU operators or LLM inference
• Diagnose thread count and memory usage anomalies
• Locate the cause of a performance regression

Run language models locally with llama.cpp on CPUs, Apple Silicon, or AMD and Intel GPUs. The skill guides model setup, memory-saving GGUF format choices, interactive chat, and serving models through an OpenAI-compatible API.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Running a language model on a CPU-only computer
• Setting up local chat on an Apple Silicon Mac
• Choosing model quantization to fit available memory

Guides you through running large language models on CPUs, Apple Silicon, and AMD or Intel GPUs. Covers choosing GGUF model files, reducing memory through quantization, and starting an OpenAI-compatible chat server.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Running a local LLM on an Apple Silicon Mac
• Trying LLM inference on a CPU-only machine
• Choosing a model format that fits available memory

Convert LLMs to GGUF and reduce model size through quantization, which stores weights at lower numerical precision. Prepare models for CPU or Apple Silicon inference, compare compression levels, and integrate them with local AI tools.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Preparing an LLM for CPU-only use
• Running a local model on Apple Silicon
• Comparing model size and quality across quantization levels

Convert language models to GGUF and reduce their size through quantization, which stores model weights at lower precision. Guides cover compression choices, local inference on CPUs and Apple Silicon, and integration with tools such as Ollama.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Running a language model on a CPU-only computer
• Choosing model compression for available memory
• Configuring Metal acceleration on Apple Silicon

Diagnose errors and incorrect inference results in MNN's QNN backend for Qualcomm HTP processors. Compare intermediate calculation data against CPU results to locate the first faulty operator, distinguish bugs from precision limits, and guide operator support and performance fixes.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate QNN runtime and model conversion errors
• Find the first operator producing incorrect results
• Distinguish implementation bugs from numerical precision limits

Improve MNN's Vulkan backend by separating CPU scheduling bottlenecks from GPU computation costs. It guides operator and shader changes, new feature integration with compatibility fallbacks, and device testing to verify both correctness and end-to-end performance.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Finding CPU or GPU bottlenecks in Vulkan inference
• Optimizing a specific operator or shader
• Integrating cooperative matrix or subgroup features

Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Preprocessing datasets larger than memory
• Running batch predictions across multiple GPUs
• Preparing image and tabular data for training

Build data processing pipelines for machine learning, from preprocessing to batch predictions. Uses Ray Data to handle datasets larger than memory, distribute work across CPUs and GPUs, and feed data into training frameworks.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Preprocessing datasets larger than memory
• Running batch predictions across multiple GPUs
• Preparing image and tabular data for training

Connect applications to NEAR AI Cloud for AI responses generated in protected computing environments. Use its OpenAI-compatible API and follow the verification flow for hardware proofs and response signatures to check which protected environment signed a response.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Connecting a chat app to the OpenAI-compatible API
• Verifying GPU and CPU execution environment proofs
• Checking response signatures against verified signing addresses

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Create or update Kubernetes YAML configuration files for GKE Autopilot and Standard. Translates application requirements into manifests covering container permissions, CPU and memory allocation, health checks, storage, external routing, and GPU resources for AI model serving.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Creating GKE application deployment manifests
• Reviewing container security and resource settings
• Configuring external routes with GKE Gateway API

Guide MNN Metal backend development and optimization, from finding performance bottlenecks to changing GPU computation, combining operators, and adjusting execution scheduling. A task-based document index directs work toward relevant implementation rules, benchmarks, and correctness checks.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Finding bottlenecks in Metal inference
• Adding or modifying Metal kernels
• Combining operators into one GPU dispatch

Executa inferência de LLM em CPU, Apple Silicon e GPUs consumer sem hardware NVIDIA. Use para edge deployment, Macs M1/M2/M3, GPUs AMD/Intel ou quando CUDA não está disponível. Suporta quantização GGUF (1,5-8 bits) para redução de memória e aceleração de 4-10× vs PyTorch em CPU.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

긴 입력을 다루는 Transformer의 학습과 추론에 Flash Attention을 적용해 속도와 GPU 메모리 사용량을 개선합니다. 설치부터 성능 측정과 출력 비교까지 안내합니다.

• 긴 입력을 사용하는 학습 속도 개선
• 긴 텍스트 추론의 메모리 사용량 절감
• 처리 속도와 모델 출력 비교

Set up machine learning training and inference on NVIDIA DGX Spark. Get guidance on choosing containers and packages, resolving CUDA compatibility errors, and checking that PyTorch can access the GPU before starting a workload.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up a new DGX Spark for training or inference
• Diagnosing CUDA compatibility and import errors
• Choosing between containers and direct pip installation

Guides you through setting up RWKV, generating text, processing long documents in chunks, and fine-tuning models. Explains how to carry context between steps and compare memory use with Transformer models.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build streaming text generation
• Process long documents in chunks
• Plan RWKV fine-tuning

GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

NVIDIA GPU에서 대규모 언어 모델의 응답 생성을 가속하고 API로 제공하도록 설정을 돕습니다. 양자화, 일괄 처리, 여러 GPU로의 분산도 다룹니다.

• 언어 모델을 채팅 API로 제공
• 많은 프롬프트를 일괄 처리
• 양자화로 메모리 사용량 감소

Plan memory use and monitor long training jobs on NVIDIA DGX Spark. It guides troubleshooting for out-of-memory failures, tracks temperature and power alongside training logs, and helps assess competition between training and inference workloads.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Estimating available memory before training
• Troubleshooting out-of-memory failures
• Investigating slowdowns during long training jobs

Investigate SGLang runs that freeze across multiple GPUs. It uses stack traces, communication logs, GPU memory dumps, and comparisons between processes to locate state differences, trace their causes, and guide fixes.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Locate where a multi-GPU inference run freezes
• Compare logs from distributed processes
• Trace the first difference in process state

Guides the use of RWKV, a language model architecture that carries information forward in a compact state. Covers streaming text generation, long document processing, fine-tuning, and memory comparisons with Transformers.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Build streaming text generation
• Process long documents in chunks
• Compare memory needs with Transformers

Helps reduce GPU memory use by loading large language models in 4-bit or 8-bit formats, a technique called quantization. Covers bitsandbytes configuration, memory checks, QLoRA fine-tuning, and training with memory-saving optimizers.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Load a larger model within available GPU memory
• Compare 4-bit and 8-bit quantization settings
• Fine-tune a quantized model with QLoRA

Optimize transformer attention, the calculation that relates tokens to one another, using Flash Attention. The skill guides PyTorch integration, GPU memory reduction, performance measurement, and output comparisons for training or inference with long sequences.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reducing GPU memory use with long sequences
• Integrating Flash Attention into PyTorch models
• Comparing runtime and outputs before and after changes