Results for “hqq”

Compress LLM weights using HQQ quantization, which stores weights with fewer bits without calibration datasets. Follow workflows for precision settings, model saving, inference integration, and LoRA fine-tuning of quantized models.

• Compressing LLMs without calibration datasets
• Serving quantized models with vLLM or HuggingFace Transformers
• Comparing bit widths and inference backends

Apply HQQ quantization, which stores language model weights using fewer bits without calibration examples. Covers precision settings, inference backends, saving compressed models, deployment with HuggingFace Transformers or vLLM, and further training with LoRA.

• Compressing a model without calibration data
• Comparing 4-bit and 2-bit quantization settings
• Running quantized models with HuggingFace Transformers or vLLM

Half-Quadratic Quantization for LLMs without calibration data. Use when quantizing models to 4/3/2-bit precision without needing calibration datasets, for fast quantization workflows, or when deploying with vLLM or HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Quantização Half-Quadratic para LLMs sem dados de calibração. Use ao quantizar modelos com precisão 4/3/2-bit sem necessidade de conjuntos de calibração, para workflows de quantização rápida, ou ao fazer deploy com vLLM ou HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Half-Quadratic Quantization for LLMs without calibration data. Use when quantizing models to 4/3/2-bit precision without needing calibration datasets, for fast quantization workflows, or when deploying with vLLM or HuggingFace Transformers.

The English summary is being prepared. Showing the original description.