'inference' 검색 결과

Guides migration of existing AI inference workloads to self-hosted GKE, Google’s managed Kubernetes service. Covers requirements, GPU sizing, deployment configuration, model storage, validation, and traffic cutover using gcloud and kubectl.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Migrating an existing Cloud Run AI service to GKE
• Planning a self-hosted replacement for Gemini API inference
• Sizing GPUs and preparing deployment configuration

Lists all inference providers offered during NemoClaw onboarding. Use when explaining which providers are available, what the onboard wizard presents, or how inference routing works. Trigger keywords - nemoclaw inference options, nemoclaw onboarding providers, nemoclaw inference routing, switch nemoclaw inference model, change inference runtime, nemoclaw local inference, ollama nemoclaw, vllm nemoclaw, local model server, openai compatible endpoint.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Diagnose failed model requests from OpenShell sandboxes to hosted APIs or local servers such as Ollama and vLLM. Check provider attachment, access rules, credentials, network layout, and client settings to identify the narrowest configuration change needed.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Diagnose failed requests to hosted model APIs
• Troubleshoot connections to Ollama or LM Studio
• Investigate credential and endpoint mismatch errors

Monitor requests to an AI prediction API for signs of model copying or training-data exposure. Test an authorized model with simulated attacks, measure the risk, and evaluate defenses such as request limits and reduced prediction detail.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Monitoring a public or partner prediction API
• Measuring model copying risk before deployment
• Testing request limits and reduced prediction detail

Generate RuVLLM settings for local language model inference and create MicroLoRA or SONA instances. Adjust their small internal weights using quality scores as feedback, with instances held in server memory for subsequent adaptation calls.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Configure local language model inference
• Prepare inference tools for an air-gapped environment
• Adjust MicroLoRA using quality feedback

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Reduce LLM weight usage through pruning, which removes selected model parameters. Explore Wanda and SparseGPT for compression without retraining, choose hardware-friendly patterns for faster inference, and compare the compressed model's accuracy with the original.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Compressing an LLM without retraining
• Comparing pruning methods
• Exploring faster inference on supported hardware

Helps speed up LLM text generation by comparing methods that predict and verify multiple tokens at once. Covers draft model selection, Medusa prediction heads, Lookahead Decoding settings, and deployment examples for inference services.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reducing chatbot response latency
• Comparing methods for faster code generation
• Selecting a small draft model for a larger LLM

Deploy AI models on GKE with suitable GPUs or TPUs and model servers. Generate deployment configuration, adjust automatic scaling, and tune inference—the process of producing model responses—for memory use, throughput, and response time.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Deploying Gemma or Llama on GKE
• Choosing GPUs or TPUs for model serving
• Configuring automatic scaling for LLM requests

Configure LLM inference and serving on NVIDIA GPUs with TensorRT-LLM. Covers quantization, which reduces numerical precision, batching requests during generation, and distributing models across GPUs to improve processing capacity and response times.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up an LLM server on NVIDIA GPUs
• Configuring FP8 or INT4 quantization
• Distributing large models across multiple GPUs

Managing privacy risks from AI-driven inferences about individuals including derived data classification, profiling under GDPR Art. 22, inference accuracy obligations, and controlling automated personality/behaviour predictions. Keywords: AI inference, derived data, profiling, automated predictions, GDPR.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Conducts privacy auditing of AI models including training data extraction testing, membership inference attacks, model inversion testing, and attribute inference assessment. Uses ML Privacy Meter and related tools to quantify privacy leakage. Keywords: model audit, membership inference, privacy meter, model inversion, training data extraction.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Guides the migration of existing AI workloads (Cloud Run, Gemini API, Gemini Enterprise Agent Platform) to self-hosted GKE inference using gcloud and kubectl. Use when the user has an existing AI inference workload (on Cloud Run, the Gemini API, Gemini Enterprise Agent Platform, or a custom VM) and wants to move it to self-hosted inference on GKE, or asks follow-up questions during such a migration (hardware sizing, model staging, manifest generation, validation, traffic cutover). DO NOT use for brand new GKE inference deployments with no existing workload to migrate (use gke-inference instead). DO NOT use if the user intends to automate the migration via the Gemini Cloud Assist MCP server.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

This skill should be used when the user asks to "implement a DiD regression", "run a staggered difference-in-differences", "set up an event study", "implement IV / 2SLS", "run a regression discontinuity design", "build synthetic control", "do propensity score matching", "test parallel trends", "run Honest DiD", "wild cluster bootstrap", "Callaway-Sant'Anna", "Sun-Abraham", "Bacon decomposition", "double machine learning", "debiased machine learning", "DML / DoubleML", "cross-fitting", "causal forest", "heterogeneous treatment effects", "ATE with ML controls", "which causal design fits my data", or needs causal-inference code, diagnostics, or identification writing in Python (StatsPAI), R, or Stata. Based on Scott Cunningham's Causal Inference: The Mixtape, with runnable validated templates, ML-based causal inference patterns, and Journal of Finance applications.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Design traffic controls for AI inference services using Istio configuration examples. Covers mutual authentication, model version routing, gradual releases, retry limits, failure containment, and monitoring, with guidance for validating changes on the target cluster.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Secure communication between inference services
• Gradually route requests to a new model version
• Review retries and failure containment for GPU services

Set up LLM serving with structured responses, including JSON and outputs restricted to specified patterns. Covers reusing shared prompt calculations, tool calling, and multi-turn conversations through SGLang, with examples for server setup and application integration.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extract information into JSON with a defined schema
• Build agents that call tools
• Reuse shared prompts across chat requests

Plan and troubleshoot large language and vision model training and inference. This guide uses hardware measurements to identify bottlenecks, address memory shortages and unstable training, and improve checkpointing, recovery, and performance across multiple GPUs and machines.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Diagnosing slow training and GPU memory shortages
• Investigating loss spikes and invalid numerical values
• Debugging communication hangs across GPUs and machines

Run model training, fine-tuning, inference, and batch jobs on Modal’s cloud GPUs. Estimate hardware needs and cost, generate Python launch scripts, run or deploy workloads, monitor logs, and retrieve results without managing remote servers.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Test CUDA code without a local GPU
• Run model training or LoRA fine-tuning on cloud GPUs
• Deploy a model inference API or vLLM service

Find why MNN CPU inference is slow by examining threads, scheduling, memory layout, and the processor instruction path actually used. Operator-level measurements and comparisons with other inference frameworks help identify which layer needs improvement across ARM, x86_64, and RISC-V.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate slow CPU operators or LLM inference
• Diagnose thread count and memory usage anomalies
• Locate the cause of a performance regression

Build LLM serving workflows that produce JSON or other required formats. The skill covers reusing computation for shared prompts, tool calls, and ongoing conversations, with examples for server setup and structured output.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Extracting article details into JSON
• Building agents that call tools
• Reusing shared prompts across conversations

Reduce LLM generation delays with speculative decoding, which proposes tokens ahead of verification. This skill compares small draft models, Medusa, and Lookahead Decoding, with implementation examples, tuning guidance, and deployment using vLLM.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Reducing response delays in chatbots
• Speeding up code generation
• Choosing a small model to draft candidate tokens

Guides LLM compression through pruning, which removes selected model weights. Covers Wanda and SparseGPT methods without retraining, accuracy comparisons, and structured weight patterns that can accelerate inference on compatible hardware.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Trying model compression without retraining
• Comparing accuracy before and after pruning
• Choosing between Wanda and SparseGPT

Guides you through running large language models on CPUs, Apple Silicon, and AMD or Intel GPUs. Covers choosing GGUF model files, reducing memory through quantization, and starting an OpenAI-compatible chat server.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Running a local LLM on an Apple Silicon Mac
• Trying LLM inference on a CPU-only machine
• Choosing a model format that fits available memory

Deploy SageMaker endpoints, the services that answer model requests, with automatic capacity scaling, CloudWatch alarms, and resource tags. Supports real-time and asynchronous inference, scaling down to zero instances, deployment checks, and resource cleanup.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Deploying a real-time model endpoint
• Scaling a sparse workload down to zero instances
• Running long inference jobs asynchronously