Results for “4-bit”

Reduce GPU memory use by loading language models in 8-bit or 4-bit formats with bitsandbytes and HuggingFace Transformers. Includes configuration examples, QLoRA fine-tuning of compressed models, and optimizers that reduce memory needed during training.

• Load larger models within limited GPU memory
• Compare 8-bit and 4-bit model settings
• Fine-tune a model with QLoRA

Compress large language models to 4-bit weights to reduce GPU memory needs and accelerate text generation. Covers loading compressed models, converting your own models, choosing settings, and preparing memory-efficient fine-tuning with transformers and PEFT.

• Run language models with limited GPU memory
• Convert your own model to 4-bit weights
• Compare compression settings for quality and speed

Compress large language models to 4-bit weights to reduce GPU memory use and accelerate inference. Covers loading compressed models, configuring quantization, and using Transformers or vLLM, with guidance on hardware requirements and backend choices.

• Running large models with limited GPU memory
• Compressing a model with domain-specific sample text
• Serving AWQ models with vLLM

Helps reduce GPU memory use by loading large language models in 4-bit or 8-bit formats, a technique called quantization. Covers bitsandbytes configuration, memory checks, QLoRA fine-tuning, and training with memory-saving optimizers.

• Load a larger model within available GPU memory
• Compare 4-bit and 8-bit quantization settings
• Fine-tune a quantized model with QLoRA

Compress large language model weights to four bits with AWQ, a method that uses model activation patterns to limit quality loss. Follow workflows for quantization, loading compressed models, and serving them with vLLM using less GPU memory.

• Running large language models with limited GPU memory
• Quantizing a model with domain-specific text
• Serving an AWQ model with vLLM

Reduce the memory needed to run large language models with GPTQ quantization, which stores model weights at lower precision. Covers loading compressed models, quantizing your own models, choosing execution backends, and connecting to memory-efficient fine-tuning.

• Run a language model within limited GPU memory
• Load a pre-quantized model for text generation
• Quantize and save your own trained model

Quantização pós-treinamento em 4-bits para LLMs com perda mínima de precisão. Use para implantar modelos grandes (70B, 405B) em GPUs de consumo, quando você precisa de redução de memória 4× com <2% de degradação de perplexidade, ou para inferência mais rápida (aceleração de 3-4×) vs FP16. Integra com transformers e PEFT para fine-tuning QLoRA.

The English summary is being prepared. Showing the original description.

Quantização de pesos consciente da ativação para compressão de LLM de 4-bit com aceleração 3x e perda mínima de acurácia. Use ao implantar modelos grandes (7B-70B) em memória GPU limitada, quando precisa de inferência mais rápida que GPTQ com melhor preservação de acurácia, ou para modelos com instruções ajustadas e multimodais. Vencedor do Prêmio Melhor Artigo MLSys 2024.

The English summary is being prepared. Showing the original description.

Quantiza LLMs para 8-bit ou 4-bit com redução de memória de 50-75% e perda mínima de acurácia. Use quando a memória GPU é limitada, precisa ajustar modelos maiores ou quer inferência mais rápida. Suporta formatos INT8, NF4, FP4, treinamento QLoRA e otimizadores 8-bit. Funciona com HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Post-training 4-bit quantization for LLMs with minimal accuracy loss. Use for deploying large models (70B, 405B) on consumer GPUs, when you need 4× memory reduction with <2% perplexity degradation, or for faster inference (3-4× speedup) vs FP16. Integrates with transformers and PEFT for QLoRA fine-tuning.

The English summary is being prepared. Showing the original description.

Activation-aware weight quantization for 4-bit LLM compression with 3x speedup and minimal accuracy loss. Use when deploying large models (7B-70B) on limited GPU memory, when you need faster inference than GPTQ with better accuracy preservation, or for instruction-tuned and multimodal models. MLSys 2024 Best Paper Award winner.

The English summary is being prepared. Showing the original description.

Quantizes LLMs to 8-bit or 4-bit for 50-75% memory reduction with minimal accuracy loss. Use when GPU memory is limited, need to fit larger models, or want faster inference. Supports INT8, NF4, FP4 formats, QLoRA training, and 8-bit optimizers. Works with HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Apply HQQ quantization, which stores language model weights using fewer bits without calibration examples. Covers precision settings, inference backends, saving compressed models, deployment with HuggingFace Transformers or vLLM, and further training with LoRA.

• Compressing a model without calibration data
• Comparing 4-bit and 2-bit quantization settings
• Running quantized models with HuggingFace Transformers or vLLM

Diagnoses and resolves Amazon RDS for Oracle connectivity, authentication, networking, and driver troubleshooting. Applicable to any RDS-for-Oracle question including connecting a Python Lambda to RDS Oracle in a VPC with pooling and cold-start optimization, EKS pods to RDS Oracle via the Secrets Manager CSI driver with IRSA and SecretProviderClass, ORA-12170 cross-VPC timeouts from EC2, DPI-1047 cannot-locate-64-bit-Oracle-Client errors, and Oracle Connection Manager (CMAN) on EC2 as a proxy with HA across two AZs. Covers python-oracledb thin vs thick mode, init_oracle_client, RDS Proxy does NOT support RDS Oracle, port 1521, VPC peering, Transit Gateway, Kerberos with AWS Managed Microsoft AD, SSL/TLS/NNE, SSM port forwarding, EC2/ECS Fargate/EKS/Lambda, SQL Developer/DBeaver/Toad/SQLcl, and Secrets Manager.

The English summary is being prepared. Showing the original description.

Identify, exploit, and write custom payloads for classic Stack-Based Buffer Overflows in 32-bit and 64-bit applications. Use this skill when conducting exploit development, reverse engineering custom network protocols, or preparing for advanced certifications (OSCP, OSEP). Covers fuzzing, controlling EIP/RIP, identifying bad characters, generating shellcode, finding return/JMP instructions, and gaining reverse shells.

The English summary is being prepared. Showing the original description.

- Track aircraft and vessels from public ADS-B and AIS broadcasts using ADS-B Exchange, Flightradar24, FlightAware,… Use when following a tail number or flight, looking up an ICAO 24-bit hex code, registration or callsign, tracing a ship by IMO number or MMSI, checking a flag of convenience or port-call history, finding who owns a private jet or vessel

The English summary is being prepared. Showing the original description.

Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs Published in arXiv , 2025 Abstract : Efficient deployment of 1-bit Large Language

The English summary is being prepared. Showing the original description.