“llama.cpp”的搜索结果

Pick the serving stack and per-runtime memory flags (vLLM, SGLang, llama.cpp, TensorRT Edge-LLM) for an LLM/VLM workload on any NVIDIA Jetson.

中文简介正在准备中。当前显示原文说明。

[omh] Self-hosted LLM serving on GPUs: choose the serving engine and quantization from decision tables, prepare deployment as an idempotent runbook with observed-only verification, and measure the endpoint with the standard TTFT/TPOT/goodput protocol. Use when the user says: inference-serving, inference serving, serve this model, serve the model, model serving, serving endpoint, vllm, llama.cpp.

中文简介正在准备中。当前显示原文说明。

Delegate a coding task to Aider (`aider`) as a background implementer, then review its diff and land it yourself. Use this whenever the user wants to hand implementation work to Aider - phrasings like "have Aider do X", "delegate this to aider", "run it through Aider", or "use Aider to implement/fix/refactor" - or wants to run a queue of coding tasks through Aider while staying the reviewer. This includes asking Aider to drive a local or self-hosted OpenAI-compatible endpoint ("have Aider use my local model", "run Aider against llama.cpp / Ollama / vLLM / LM Studio"), which Aider reaches via `--api-base`. DO NOT USE for local-model or coding requests that do not name Aider, for tasks small enough to do inline, or when the user wants the code written directly without delegating.

中文简介正在准备中。当前显示原文说明。

Uninstall or hard-clean OpenClaw Companion, Windows node, native Gateway/MXC, WSL Gateway, and managed llama.cpp/Local AI state for a clean retest. Choose the existing dev CLI uninstall or the reviewed hard-clean procedure, inventory exact targets, and obtain destructive confirmation before acting.

中文简介正在准备中。当前显示原文说明。

Build private, on-device AI features on iPhone, iPad, and Mac with Foundation Models, Core ML, MLX Swift, or llama.cpp. Use when choosing an Apple-local model runtime, building an Apple Intelligence chatbot or tool-calling feature, running an LLM on Apple Silicon, converting or compressing a Python model for Core ML, or comparing on-device inference backends. For Swift Core ML loading and prediction code, use the coreml skill.

中文简介正在准备中。当前显示原文说明。

Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact GGUF file lookup, conversion, and OpenAI-compatible local serving.

中文简介正在准备中。当前显示原文说明。

Cross-engine decision rubric for self-hosting or recommending an LLM serving stack. Picks among vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX as a function of (hardware × workload × constraint), not "which is fastest". Activates whenever a coder-agent must choose, defend, or migrate a serving runtime.

中文简介正在准备中。当前显示原文说明。

Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.

中文简介正在准备中。当前显示原文说明。

在本机 Mac 或 Apple Silicon 上部署 Gemma 4 12B。本地安装/升级 llama.cpp，下载 GGUF 量化模型，用 llama-server 暴露 OpenAI-compatible API，或用 Ollama 暴露本地模型服务；按用户需求在默认 Q4_K_M、64K/128K 长上下文、QAT Q4_0 @ 256K、左右对比演示之间选择，配置 tmux 后台运行，验证健康检查、问答接口、资源占用和常见故障。当用户说部署 Gemma 4、Gemma 4 12B、本地大模型、长上下文、QAT、量化、llama-server、Ollama、GGUF、Mac 本地模型服务时使用。

中文简介正在准备中。当前显示原文说明。

Optimize Ollama configuration for the current machine's hardware. Use when asked to speed up Ollama, tune local LLM performance, or pick models that fit available GPU/RAM. Don't use for LM Studio, llama.cpp, vLLM, or hosted-API LLM providers.

中文简介正在准备中。当前显示原文说明。

Runs local and self-hosted LLMs with Ollama, LM Studio, MLX, llama.cpp, and Open WebUI. Use when choosing GGUF quants, sizing VRAM, or running models privately.

中文简介正在准备中。当前显示原文说明。

Run a GGUF model locally with llama.cpp, and find one on the HF Hub.

中文简介正在准备中。当前显示原文说明。

Set up and run local LLMs with Harbor. Use when the user wants to run models locally, install Harbor, start Open WebUI, llama.cpp, Ollama, vLLM, Docker Model Runner, MLX, or oMLX, pull GGUF or HuggingFace models, add SearXNG web search, Speaches TTS/STT, or Open Terminal code execution, launch Codex/Claude/Grok/OpenCode against a Harbor backend, or troubleshoot GPU, VRAM, and service startup.

中文简介正在准备中。当前显示原文说明。

llama.cpp local GGUF inference + HF Hub model discovery.

中文简介正在准备中。当前显示原文说明。

Executa inferência de LLM em CPU, Apple Silicon e GPUs consumer sem hardware NVIDIA. Use para edge deployment, Macs M1/M2/M3, GPUs AMD/Intel ou quando CUDA não está disponível. Suporta quantização GGUF (1,5-8 bits) para redução de memória e aceleração de 4-10× vs PyTorch em CPU.

中文简介正在准备中。当前显示原文说明。

llama.cpp local GGUF inference + HF Hub model discovery.

中文简介正在准备中。当前显示原文说明。

Master local LLM inference, model selection, VRAM optimization, and local deployment using Ollama, llama.cpp, vLLM, and LM Studio. Expert in quantization formats (GGUF, EXL2) and local AI privacy.

中文简介正在准备中。当前显示原文说明。

Use to select models to run locally with llama.cpp and GGUF on CPU, Mac Metal, CUDA, or ROCm. Covers finding GGUFs, quant selection, running servers, exact GGUF file lookup, conversion, and OpenAI-compatible local serving.

中文简介正在准备中。当前显示原文说明。

Master local LLM inference, model selection, VRAM optimization, and local deployment using Ollama, llama.cpp, vLLM, and LM Studio. Expert in quantization formats (GGUF, EXL2) and local AI privacy.

中文简介正在准备中。当前显示原文说明。

Integrate on-device AI using Foundation Models framework, Core ML, and open-source LLM runtimes on Apple Silicon. Covers Foundation Models (LanguageModelSession, @Generable, @Guide, SystemLanguageModel, structured output, tool calling), Core ML (coremltools, model conversion, quantization, palettization, pruning, Neural Engine, MLTensor), MLX Swift (transformer inference, unified memory), and llama.cpp (GGUF, cross-platform LLM). Use when building tool-calling AI features, working with guided generation schemas, converting models, or running on-device inference.

中文简介正在准备中。当前显示原文说明。

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

中文简介正在准备中。当前显示原文说明。

Hermes Agent (Nous Research の AI CLI エージェント)。インストール、 CLI コマンド (chat, model, gateway, config, cron, webhook, mcp, skills, profile, hermes pm)、 Configuration (Terminal Backends, Memory, TTS / STT, Auxiliary Models, Compression, Display, Local Models llama.cpp, Egress / Secrets)、 Features (Tools, Skills, Hooks, Delegation, Kanban, Cron, Voice Mode, Personality, Context Files)、 Messaging Gateway (Platform Support, Bot Mode, Connectors)、Security、Architecture、 Hermes Desktop、Plugin Catalog、Profiles、Developer Guide、Nous Portal (Tool Gateway)。

中文简介正在准备中。当前显示原文说明。

GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.

中文简介正在准备中。当前显示原文说明。