'llm ops' 검색 결과

Connect an application's LLM calls through the Caveman Cloud gateway to measure usage and cost without changing request content. The skill updates connection settings and verifies the integration with one small, real provider request.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Start measuring an LLM application's usage and cost
• Route existing LLM calls through the gateway
• Verify a connection with a real request

Choose a serving container for models deployed on SageMaker. Match the model type to a compatible image, retrieve its address from the AWS catalog, and check regional availability, CPU or GPU variants, and required host settings.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Choose a container for an LLM deployment
• Select images for embeddings and rerankers
• Check regional image addresses and GPU compatibility

Build and operate LLM applications with guidance on RAG, which uses retrieved documents to inform answers. Covers document indexing, vector database selection, prompt improvement, response evaluation, caching, and estimates of model usage costs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Building an assistant that answers from documents
• Choosing a vector database for document search
• Comparing answer quality after prompt changes

Design LLM applications using patterns for answers grounded in documents, agents that use tools, and prompt testing. Implementation examples also cover monitoring response quality, latency, costs, and failures during operation.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Building an assistant that answers from internal documents
• Choosing an architecture for an agent that uses tools
• Comparing and versioning prompts

Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating failures in LLM applications
• Evaluating answers against test datasets
• Monitoring production latency and costs

Use LangSmith to record LLM application inputs, outputs, and timing, investigate failures, evaluate answers against test datasets, and monitor production usage. It also guides automated quality checks and analysis of token usage and costs.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigating failures in LLM applications
• Evaluating answers against test datasets
• Monitoring production latency and costs

Deploy language model APIs and run batch generation with vLLM. Configure GPU memory use, quantization—lower precision model storage—and distribution across GPUs, then test response times and monitor how many requests the server handles.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Deploying an OpenAI-compatible language model API
• Generating responses for a large set of prompts
• Fitting a large model into limited GPU memory

Prepare trained machine learning models for production with deployment, monitoring, and retraining workflows. Build RAG systems that retrieve documents for answers, and integrate language model APIs with retries, fallback providers, and cost tracking.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Deploying a trained model to production
• Setting up model monitoring and retraining workflows
• Building a RAG pipeline for document-based answers

Evaluate LLM application quality using automated scores, human reviews, and another language model as a judge. Compare models or prompts, establish baseline results, and detect quality drops through repeatable tests and statistical comparisons.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Comparing models and prompts
• Checking quality before deployment
• Evaluating search quality in RAG applications

Diagnose failed model requests from OpenShell sandboxes to hosted APIs or local servers such as Ollama and vLLM. Check provider attachment, access rules, credentials, network layout, and client settings to identify the narrowest configuration change needed.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Diagnose failed requests to hosted model APIs
• Troubleshoot connections to Ollama or LM Studio
• Investigate credential and endpoint mismatch errors

Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate AI application problems using execution traces
• Evaluate answer quality across a test dataset
• Compare prompts and models through experiments

Use Phoenix to record execution traces, detailed records of an AI application's operations, and investigate problems. Evaluate answers on datasets, compare prompts and models, and set up monitoring through a self-hosted server.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Investigate AI application problems using execution traces
• Evaluate answer quality across a test dataset
• Compare prompts and models through experiments

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when "langfuse, llm observability, llm tracing, prompt management, llm evaluation, monitor llm, debug llm, langfuse, observability, tracing, llm-monitoring, evaluation, prompt-management, debugging, analytics" mentioned.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Erstellt oder überarbeitet einen Skill FÜR DIESES ÖKOSYSTEM: delegiert die generische Interview-/Entwurfs-/Eval-Schleife an das offizielle skill-creator-Plugin (Anthropic, Apache-2.0), setzt aber zusätzlich die ellmos-Anforderungen durch, die das offizielle Plugin nicht kennt — Haus-Frontmatter (9 Felder), Kategorie/Sichtbarkeits-Regeln, anbieter-/nutzer-/ellmos-neutrale aber ellmos-sensitive Erkennung via grounding-seed, mitgelieferte config.json (inkl. Lernvertrag) + connections.json, Anbieter-/Vendor-Bewusstsein (State-of-the-Art-Alternativen inkl. eigener ellmos-Komponenten vorschlagen), einen Selbstaktualisierungs-Abschnitt UND einen wiederverwendbaren Lern-/Abdeckungs-Dateisatz (user-usecases.json, usecase-gaps.json, known-providers-and-abilities.json, gap-closing-analysis.json — an skill-creators evals.json und A2A/AgentSkill angelehnt, kein Parallelstandard). IMMER nutzen, wenn ein NEUER Skill für dieses System entstehen oder ein bestehender auf ellmos-Konformität nachgerüstet werden soll — auch bei "erstelle einen Skill", "bau mir einen Skill für X", "skill-creator nutzen", "neuen Skill anlegen", "diesen Skill ellmos-konform machen", "Lernvertrag/Verhaltenscodex für einen Skill anlegen", "Usecase-Abdeckung eines Skills prüfen". Für reines Skill-FINDEN/ROUTEN stattdessen skill-finder, für Landschafts-Audit skill-explorer.

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

AI基础设施实战——从GPU集群调度到模型推理部署的全栈实操指南。覆盖：昇腾/GPU统一调度、PyTorch分布式训练、vLLM推理部署、模型量化实战、LLMOps、GPU故障诊断、HuggingFace生态、MLOps流水线。触发词：AI基础设施、GPU、昇腾、训练、推理、vLLM、PyTorch分布式、模型部署、LLMOps、MLOps、集群调度、HuggingFace

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Plan AI application development by selecting relevant phases and companion skills. The workflow covers language model integration, retrieval from documents, agents, machine learning pipelines, monitoring, and security, with prompts and quality checklists.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Plan AI features and language model integration
• Organize RAG retrieval design and implementation
• Map agent roles, tools, and memory

인공지능 에이전트의 문제를 지시, 기억, 도구 실행, 화면 표시 등 12개 계층에서 조사합니다. 코드와 로그를 근거로 원인을 정리하고 심각도에 따른 지적 사항과 수정안을 제시합니다.

• 앱 안에서만 발생하는 문제 조사
• 이전 대화가 섞이는 원인 확인
• 도구 실행 누락과 허위 실행 보고 확인

Design workflows that evaluate and refine AI outputs through self-review, separate generation and evaluation roles, tests, or weighted scoring criteria. Includes patterns for structured feedback, iteration limits, and stopping when quality stops improving.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Adding self-review and revision loops to AI outputs
• Separating generation from evaluation
• Refining generated code using test results

Plan how to serve a model on Amazon SageMaker AI. The skill uses model type, traffic, response time, and cost constraints to recommend a deployment route, check instance quotas, and coordinate specialized setup and deployment skills.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Plan AWS hosting for a language model
• Choose a serving route for an embedding model
• Plan asynchronous inference for image generation

LLM agent 基础设施 (LLM agent infra) Master OS — automated mastery of LLM agent infra: top builders' mental models, tool stack, current workflows, jargon, and where to keep up. Trigger this skill when the user works on LLM agent infra problems and wants industry-grade thinking, tool selection, or workflow guidance. 触发词：「agent framework」「LLM agent」「agent infra」「multi-agent orchestration」「agent runtime」

한국어 요약은 준비 중입니다. 원문 설명을 표시하고 있습니다.

Set up ruvector@0.3.3 with supporting packages, register its MCP server for agent access, and verify installation with diagnostic commands and smoke tests. Helps prepare the dependencies needed for initial use of /vector subcommands.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up ruvector for the first time
• Resolving missing dependencies for Brain, SONA, or LLM commands
• Registering the ruvector MCP server with Claude

Develop incident response procedures for LLM outages, declining answer quality, spending spikes, and unsafe outputs. Use example monitoring alerts, severity levels, recovery playbooks, and postmortem guidance to organize detection, response, and follow-up work.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Plan fallback procedures for model provider outages
• Investigate declining answer quality after a deployment
• Prepare emergency controls for token spending spikes

Design scheduled AI workflows for document ingestion, batch processing, and model training using Prefect, Airflow, or Dagster. Compare orchestration tools and plan task dependencies, retries, caching, and monitoring through implementation fragments.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Schedule document ingestion and RAG index refreshes
• Design batch workflows for LLM requests
• Connect model training with quality evaluation

Guides setup and operation of Lambda Labs GPU instances for machine learning training and inference. Covers SSH access, persistent storage, API automation, and multi-GPU or multi-node training, including how to retain checkpoints and terminate idle instances.

한국어 요약은 준비 중입니다. 영어 요약을 표시하고 있습니다.

• Setting up a GPU instance for LLM fine-tuning
• Keeping datasets and checkpoints between sessions
• Automating instance launch and termination through the API