Results for “inference optimization”

Interactively gather parameters for `python -m cli.inference.throughput_optimizer`, generate a deployment simulation command, explain assumptions, ask for execution confirmation, then run the simulation and summarize the best parallel strategy. Use when the user wants to evaluate a model on one or more hardware profiles, compare hardware targets, compare aggregation versus disaggregation versus PD ratio optimization, choose TP/EP/MOE-DP search ranges, or obtain recommended concurrency, throughput, TTFT, and TPOT results from throughput modeling.

The English summary is being prepared. Showing the original description.

Formato GGUF e quantização llama.cpp para inferência eficiente em CPU/GPU. Use ao implantar modelos em hardware de consumidor, Apple Silicon, ou quando precisar de quantização flexível de 2-8 bits sem requisitos de GPU.

The English summary is being prepared. Showing the original description.

Quantização Half-Quadratic para LLMs sem dados de calibração. Use ao quantizar modelos com precisão 4/3/2-bit sem necessidade de conjuntos de calibração, para workflows de quantização rápida, ou ao fazer deploy com vLLM ou HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Master local LLM inference, model selection, VRAM optimization, and local deployment using Ollama, llama.cpp, vLLM, and LM Studio. Expert in quantization formats (GGUF, EXL2) and local AI privacy.

The English summary is being prepared. Showing the original description.

Use when working with Baseten — baseten ML deployment platform management covering model inventory, deployment status, autoscaling configuration, inference call history, GPU allocation, environment management, and performance metrics. Use for comprehensive Baseten workspace assessment and ML serving optimization.

The English summary is being prepared. Showing the original description.

Master local LLM inference, model selection, VRAM optimization, and local deployment using Ollama, llama.cpp, vLLM, and LM Studio. Expert in quantization formats (GGUF, EXL2) and local AI privacy.

The English summary is being prepared. Showing the original description.

Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.

The English summary is being prepared. Showing the original description.

Half-Quadratic Quantization for LLMs without calibration data. Use when quantizing models to 4/3/2-bit precision without needing calibration datasets, for fast quantization workflows, or when deploying with vLLM or HuggingFace Transformers.

The English summary is being prepared. Showing the original description.

Master TypeScript with advanced types, generics, and strict type safety. Handles complex type systems, decorators, and enterprise-grade patterns. Use PROACTIVELY for TypeScript architecture, type inference optimization, or advanced typing patterns.

The English summary is being prepared. Showing the original description.

Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring. Use PROACTIVELY for ML model deployment, inference optimization, or production ML infrastructure.

The English summary is being prepared. Showing the original description.