'inference optimization' 검색 결과

Interactively gather parameters for `python -m cli.inference.throughput_optimizer`, generate a deployment simulation command, explain assumptions, ask for execution confirmation, then run the simulation and summarize the best parallel strategy. Use when the user wants to evaluate a model on one or more hardware profiles, compare hardware targets, compare aggregation versus disaggregation versus PD ratio optimization, choose TP/EP/MOE-DP search ranges, or obtain recommended concurrency, throughput, TTFT, and TPOT results from throughput modeling.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Quantização Half-Quadratic para LLMs sem dados de calibração. Use ao quantizar modelos com precisão 4/3/2-bit sem necessidade de conjuntos de calibração, para workflows de quantização rápida, ou ao fazer deploy com vLLM ou HuggingFace Transformers.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Formato GGUF e quantização llama.cpp para inferência eficiente em CPU/GPU. Use ao implantar modelos em hardware de consumidor, Apple Silicon, ou quando precisar de quantização flexível de 2-8 bits sem requisitos de GPU.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Master local LLM inference, model selection, VRAM optimization, and local deployment using Ollama, llama.cpp, vLLM, and LM Studio. Expert in quantization formats (GGUF, EXL2) and local AI privacy.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Use when working with Baseten — baseten ML deployment platform management covering model inventory, deployment status, autoscaling configuration, inference call history, GPU allocation, environment management, and performance metrics. Use for comprehensive Baseten workspace assessment and ML serving optimization.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Master local LLM inference, model selection, VRAM optimization, and local deployment using Ollama, llama.cpp, vLLM, and LM Studio. Expert in quantization formats (GGUF, EXL2) and local AI privacy.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Half-Quadratic Quantization for LLMs without calibration data. Use when quantizing models to 4/3/2-bit precision without needing calibration datasets, for fast quantization workflows, or when deploying with vLLM or HuggingFace Transformers.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scaling model capacity without proportional compute increase. Covers MoE architectures, routing mechanisms, load balancing, expert parallelism, and inference optimization.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Master TypeScript with advanced types, generics, and strict type safety. Handles complex type systems, decorators, and enterprise-grade patterns. Use PROACTIVELY for TypeScript architecture, type inference optimization, or advanced typing patterns.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring. Use PROACTIVELY for ML model deployment, inference optimization, or production ML infrastructure.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.