“llama.cpp”的搜索结果

Find and run local AI models with llama.cpp. Choose GGUF model files and quantization levels that fit available memory, launch a command line session or local server, and test an OpenAI-compatible API on your computer.

中文简介正在准备中。当前显示英语简介。

• Find models to run on a local computer
• Choose quantization for available memory
• Launch and test a local OpenAI-compatible API

Find language models on Hugging Face Hub and run them locally with llama.cpp. The skill helps compare GGUF model files, choose compressed versions that fit available memory, and prepare commands for a local chat server or command-line session.

中文简介正在准备中。当前显示英语简介。

• Find a model that fits local RAM or GPU memory
• Compare available GGUF files and sizes
• Prepare a local chat server command

Convert LLMs to GGUF and reduce model size through quantization, which stores weights at lower numerical precision. Prepare models for CPU or Apple Silicon inference, compare compression levels, and integrate them with local AI tools.

中文简介正在准备中。当前显示英语简介。

• Preparing an LLM for CPU-only use
• Running a local model on Apple Silicon
• Comparing model size and quality across quantization levels

Convert language models to GGUF and reduce their size through quantization, which stores model weights at lower precision. Guides cover compression choices, local inference on CPUs and Apple Silicon, and integration with tools such as Ollama.

中文简介正在准备中。当前显示英语简介。

• Running a language model on a CPU-only computer
• Choosing model compression for available memory
• Configuring Metal acceleration on Apple Silicon

Run language models locally with llama.cpp on CPUs, Apple Silicon, or AMD and Intel GPUs. The skill guides model setup, memory-saving GGUF format choices, interactive chat, and serving models through an OpenAI-compatible API.

中文简介正在准备中。当前显示英语简介。

• Running a language model on a CPU-only computer
• Setting up local chat on an Apple Silicon Mac
• Choosing model quantization to fit available memory

Convert chat messages into template strings for local model runtimes such as Ollama or llama.cpp. Optionally route queries among a few registered intents using embeddings, numerical representations of meaning, without calling a model.

中文简介正在准备中。当前显示英语简介。

• Preparing prompt strings for Ollama
• Formatting conversations for a local model
• Routing queries among a few intents

Guides you through running large language models on CPUs, Apple Silicon, and AMD or Intel GPUs. Covers choosing GGUF model files, reducing memory through quantization, and starting an OpenAI-compatible chat server.

中文简介正在准备中。当前显示英语简介。

• Running a local LLM on an Apple Silicon Mac
• Trying LLM inference on a CPU-only machine
• Choosing a model format that fits available memory

Estimate the cost per million tokens, units of model text, on hardware you own using power consumption, electricity rates, hardware costs, and measured generation speed. Compare utilization levels to find when local operation costs less than a hosted model.

中文简介正在准备中。当前显示英语简介。

• Compare local model costs with a hosted API
• Estimate costs at different utilization levels
• Include electricity and hardware depreciation in costs

Export an approved fine-tuned model into a format suited to its hardware and runtime. Choose weight packaging and quantization, which reduces numerical precision, then compare outputs before and after export to check for regressions.

中文简介正在准备中。当前显示英语简介。

• Exporting a checkpoint approved for deployment
• Choosing a model format for a target GPU or laptop
• Comparing outputs before and after export

Use when imp outputs differ between two paths that should agree - chunk sizes, prefill vs decode, prefix-cache resend, two engines (imp vs llama.cpp vs HF), two builds or kernels - "output changes with chunk size", "first token differs", "logprobs moved", "which layer diverges", "who is right, imp or llama.cpp", "near-tie". Do NOT use for repetition loops or garbage output (check-degeneration), a new arch that loads wrong (add-model-arch), or perf (benchmark-cuda).

中文简介正在准备中。当前显示原文说明。

Run GGUF models directly, load LoRA adapters, benchmark inference speed, and serve models via llama-server using llama.cpp. Includes Qwen 3.5 serve scripts (9B dense + F16, 35B MoE) with asymmetric KV cache and thinking mode. Secondary to Ollama; use when needing direct model control or LoRA hot-loading. Triggers on 'llama.cpp', 'GGUF', 'LoRA adapter', 'benchmark inference', 'llama-server'.

中文简介正在准备中。当前显示原文说明。

Analyze and tune a running llama.cpp server for local coding agents: propose config (ctx, prompt cache, MTP), benchmark before/after, report deltas. Use to optimize llama-server for Claude Code or Aider. Don't use for vLLM, ollama, or non-llama.cpp.

中文简介正在准备中。当前显示原文说明。

Formato GGUF e quantização llama.cpp para inferência eficiente em CPU/GPU. Use ao implantar modelos em hardware de consumidor, Apple Silicon, ou quando precisar de quantização flexível de 2-8 bits sem requisitos de GPU.

中文简介正在准备中。当前显示原文说明。

GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.

中文简介正在准备中。当前显示原文说明。

Generate model responses in defined formats for data extraction and classification. Use JSON schemas, Pydantic data models, regular expressions, or fixed choices to produce structured results that other programs can parse and validate.

中文简介正在准备中。当前显示英语简介。

• Extracting company or product details from text
• Classifying reviews into predefined labels
• Turning profile text into structured fields

Set up and customize Pi, a terminal coding agent framework. This skill guides model configuration, extension development, terminal interfaces, and integration with other applications through its SDK, a software development kit, or process messaging.

中文简介正在准备中。当前显示英语简介。

• Install Pi and configure model access
• Create custom tools and terminal interfaces
• Embed Pi in another application

Configure permission checks for unattended Antigravity CLI runs. The setup combines blocking rules, automatic approval for scoped operations, AI classification for ambiguous calls, and single-use approval tokens to control tool access within a verified workspace.

中文简介正在准备中。当前显示英语简介。

• Setting up permission checks for unattended Antigravity CLI runs
• Defining rules for shell commands, file changes, and network access
• Configuring a local model to classify ambiguous tool calls

Build Python workflows that constrain language model output using text patterns and grammar rules. Generate structured JSON, restrict classifications to allowed labels, extract information, and combine reusable generation steps with loops and tool selection.

中文简介正在准备中。当前显示英语简介。

• Generate JSON with a defined structure
• Constrain date, email, and identifier formats
• Classify text using a fixed set of labels

Use Outlines to make language models produce data in a defined format. The skill covers extracting information into JSON, restricting classification labels, and applying Pydantic models, which define fields and data types, with local model backends.

中文简介正在准备中。当前显示英语简介。

• Extracting company or profile details into JSON
• Classifying text using a fixed set of labels
• Generating text that matches a specified pattern

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

中文简介正在准备中。当前显示英语简介。

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

中文简介正在准备中。当前显示英语简介。

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Build Python workflows that constrain language model output using text patterns and grammar rules. Generate structured JSON, restrict classifications to allowed labels, extract information, and combine reusable generation steps with loops and tool selection.

中文简介正在准备中。当前显示英语简介。

• Generate JSON with a defined structure
• Constrain date, email, and identifier formats
• Classify text using a fixed set of labels

Use Outlines to make language models produce data in a defined format. The skill covers extracting information into JSON, restricting classification labels, and applying Pydantic models, which define fields and data types, with local model backends.

中文简介正在准备中。当前显示英语简介。

• Extracting company or profile details into JSON
• Classifying text using a fixed set of labels
• Generating text that matches a specified pattern

Pick the serving stack and per-runtime memory flags (vLLM, SGLang, llama.cpp, TensorRT Edge-LLM) for an LLM/VLM workload on any NVIDIA Jetson.

中文简介正在准备中。当前显示原文说明。