'quantization' 검색 결과

Tune AgentDB memory use and search speed through quantization, which stores vectors more compactly, search indexes, caching, and batch operations. Includes configuration examples and benchmarks to help balance accuracy, speed, and storage as datasets grow.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reduce memory use in an AgentDB application
• Tune search speed while preserving accuracy
• Plan settings for millions of vectors

Export an approved fine-tuned model into a format suited to its hardware and runtime. Choose weight packaging and quantization, which reduces numerical precision, then compare outputs before and after export to check for regressions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Exporting a checkpoint approved for deployment
• Choosing a model format for a target GPU or laptop
• Comparing outputs before and after export

Find and run local AI models with llama.cpp. Choose GGUF model files and quantization levels that fit available memory, launch a command line session or local server, and test an OpenAI-compatible API on your computer.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Find models to run on a local computer
• Choose quantization for available memory
• Launch and test a local OpenAI-compatible API

Use this skill for setting up vector similarity search with pgvector for AI/ML embeddings, RAG applications, or semantic search. **Trigger when user asks to:** - Store or search vector embeddings in PostgreSQL - Set up semantic search, similarity search, or nearest neighbor search - Create HNSW or IVFFlat indexes for vectors - Implement RAG (Retrieval Augmented Generation) with PostgreSQL - Optimize pgvector performance, recall, or memory usage - Use binary quantization for large vector datasets **Keywords:** pgvector, embeddings, semantic search, vector similarity, HNSW, IVFFlat, halfvec, cosine distance, nearest neighbor, RAG, LLM, AI search Covers: halfvec storage, HNSW index configuration (m, ef_construction, ef_search), quantization strategies, filtered search, bulk loading, and performance tuning.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Deploy machine learning models to edge devices using Google AI Edge Gallery, TensorFlow Lite, ONNX Runtime, and MediaPipe. Covers model quantization (INT8/INT4), on-device inference with Gemma 4 models, Android/iOS deployment via AI Edge Gallery, hardware delegate selection (GPU/NPU/DSP), and performance benchmarking on constrained devices. Use when deploying models to mobile phones, IoT devices, or embedded systems where cloud inference is impractical due to latency, cost, or connectivity constraints.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

对 DiT（扩散/多模态生成）模型执行 W8A8 动态（data-free）或 W8A8 MXFP8（data-free）量化。覆盖 HunyuanVideo、Wan2.2-T2V/I2V/TI2V、FLUX.1-dev（已迁移）、SD3、Sana、HunyuanDiT、CogViewX 等。YAML apiversion 固定 multimodal_sd_modelslim_v1，统一 MultimodalPipelineInterface 重构路径（不再保留 LegacyMultimodalPipelineInterface）。模板与字段以 msmodelslim/model/hunyuan_video/model_adapter.py（唯一已迁移到重构路径且 lab_practice YAML 已 validated 的 DiT）为参考；其它 DiT 需按各自推理仓的 `parse_args` 调整字段集合。data-free 指不引入外部激活校准数据，spec.dataset（校准集短名，短名如 wan2_2_t2v；enable_dump: false 时不参与 dump）仍必填；精度调优通过下游经验库 L2 §7 + 直接调 `quantization-expert-experience-tuning-rules/scripts/apply_rollback.py`（整层回退）扩展 exclude 列表，闭环由 `quantization-accuracy-tuning-orchestrator` 调度。

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Formato GGUF e quantização llama.cpp para inferência eficiente em CPU/GPU. Use ao implantar modelos em hardware de consumidor, Apple Silicon, ou quando precisar de quantização flexível de 2-8 bits sem requisitos de GPU.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Quantização de pesos consciente da ativação para compressão de LLM de 4-bit com aceleração 3x e perda mínima de acurácia. Use ao implantar modelos grandes (7B-70B) em memória GPU limitada, quando precisa de inferência mais rápida que GPTQ com melhor preservação de acurácia, ou para modelos com instruções ajustadas e multimodais. Vencedor do Prêmio Melhor Artigo MLSys 2024.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Quantização Half-Quadratic para LLMs sem dados de calibração. Use ao quantizar modelos com precisão 4/3/2-bit sem necessidade de conjuntos de calibração, para workflows de quantização rápida, ou ao fazer deploy com vLLM ou HuggingFace Transformers.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Quantização pós-treinamento em 4-bits para LLMs com perda mínima de precisão. Use para implantar modelos grandes (70B, 405B) em GPUs de consumo, quando você precisa de redução de memória 4× com <2% de degradação de perplexidade, ou para inferência mais rápida (aceleração de 3-4×) vs FP16. Integra com transformers e PEFT para fine-tuning QLoRA.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Model Quantization Guide: Run 70B LLMs in 4 Bits — … — 12. Okt. 2025 · Run 70B parameter LLMs on con

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Expert en IA embarquée (TensorFlow Lite, ONNX, quantization, deployment on edge devices)

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Build, debug, optimize, and deploy ComfyUI workflows for diffusion image, video, and audio generation. Activate on: ComfyUI workflow, ComfyUI custom node, ComfyUI API, ComfyUI Manager, KSampler, FLUX in ComfyUI, Wan 2.2 ComfyUI, Hunyuan video ComfyUI, LTX video ComfyUI, IPAdapter, ControlNet ComfyUI, Kijai wrapper, ComfyDeploy, RunComfy, ComfyUI security, GGUF quantization, TeaCache, Nunchaku, ACE-Step ComfyUI, F5-TTS ComfyUI, subgraphs ComfyUI. NOT for: A1111/Forge/Invoke (different UIs), training pipelines from scratch, non-diffusion ML models, or general image-API integrations (use generative-video-2026 / generative-music-audio / media-gen-deployment).

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Search document collections by meaning using embeddings, numerical representations of text. Compare text similarity, route queries among a few predefined patterns, and use a compressed search index when memory is limited.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Finding documents with similar meaning
• Comparing the similarity of two pieces of text
• Reducing memory use for large vector collections

Tune vector indexes, which organize data for similarity searches, to balance speed, memory use, and missed matches. Choose index types, adjust HNSW settings, and assess compression options using benchmarks with real queries.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reduce latency in similarity searches
• Tune HNSW settings to reduce missed matches
• Compare compression options for lower memory use

Quantize diffusion transformers with NVIDIA ModelOpt, using FP8 or NVFP4 lower-precision numbers, and prepare the saved weights for SGLang Diffusion. Follow a workflow for loading, checking output quality, and comparing speed under matching conditions.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Preparing ModelOpt FP8 weights for SGLang Diffusion
• Checking quality after diffusion model quantization
• Comparing BF16 and quantized generation speed

Build similarity search with FAISS to find nearby matches in large collections of numeric vectors. The skill guides index selection, memory reduction, GPU use, and saving indexes while balancing search speed and accuracy.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Find similar items in large vector datasets
• Choose an index for search speed and accuracy
• Reduce memory use for vector search

Reduce GPU memory use by loading language models in 8-bit or 4-bit formats with bitsandbytes and HuggingFace Transformers. Includes configuration examples, QLoRA fine-tuning of compressed models, and optimizers that reduce memory needed during training.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Load larger models within limited GPU memory
• Compare 8-bit and 4-bit model settings
• Fine-tune a model with QLoRA

Triage incorrect MNN outputs and regressions by matching symptoms to focused debugging guides. Use backend comparisons, minimal reproductions, and controlled experiments to investigate memory issues, quantization, concurrency, numeric limits, caches, and inconsistent results across runs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating garbled or degraded MNN output
• Comparing results across execution backends
• Tracking regressions after optimization changes

Build similarity search with FAISS to find nearby matches in large collections of numeric vectors. The skill guides index selection, memory reduction, GPU use, and saving indexes while balancing search speed and accuracy.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Find similar items in large vector datasets
• Choose an index for search speed and accuracy
• Reduce memory use for vector search

Estimate inference memory for models on Hugging Face Hub without downloading their weights. Check GPU capacity for Safetensors or GGUF files, with optional estimates for the KV cache, memory used to retain attention data during generation.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Check whether a model fits on a GPU
• Compare memory estimates for GGUF quantizations
• Estimate KV cache memory for a chosen context length