Results for “efficient inference”

Comprehensive toolkit for protein language models including ESM3 (generative multimodal protein design across sequence, structure, and function) and ESM C (efficient protein embeddings and representations). Use this skill when working with protein sequences, structures, or function prediction; designing novel proteins; generating protein embeddings; performing inverse folding; or conducting protein engineering tasks. Supports both local model usage and cloud-based Forge API for scalable inference.

The English summary is being prepared. Showing the original description.

High-throughput inference on FriendliAI — optimized serving engine for low-latency, cost-efficient LLMs.

The English summary is being prepared. Showing the original description.

Build with DeepSeek's models — efficient open-weight reasoning models and low-cost API inference.

The English summary is being prepared. Showing the original description.

Modelo de espaço de estados com complexidade O(n) versus O(n²) dos Transformers. Inferência 5× mais rápida, sequências de milhão de tokens, sem cache KV. SSM seletivo com design aware de hardware. Mamba-1 (d_state=16) e Mamba-2 (d_state=128, multi-head). Modelos 130M-2.8B no HuggingFace.

The English summary is being prepared. Showing the original description.

Híbrido RNN+Transformer com inferência O(n). Tempo linear, contexto infinito, sem cache KV. Treina como GPT (paralelo), infere como RNN (sequencial). Projeto Linux Foundation AI. Produção em Windows, Office, NeMo. RWKV-7 (março de 2025). Modelos até 14B parâmetros.

The English summary is being prepared. Showing the original description.

Quantiza LLMs para 8-bit ou 4-bit com redução de memória de 50-75% e perda mínima de acurácia. Use quando a memória GPU é limitada, precisa ajustar modelos maiores ou quer inferência mais rápida. Suporta formatos INT8, NF4, FP4, treinamento QLoRA e otimizadores 8-bit. Funciona com HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.

The English summary is being prepared. Showing the original description.

GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.

The English summary is being prepared. Showing the original description.