「llm ops」の検索結果

Build automated evaluation suites for AI agents using golden datasets, rubrics, and regression gates. Use when shipping agent features, validating prompt changes, or gating deployments on quality.

日本語の概要は準備中です。原文の説明を表示しています。

Instrument AI agents with tracing, token metrics, latency, and cost visibility. Use for reliability and debugging.

日本語の概要は準備中です。原文の説明を表示しています。

Schedule AI agent runs with cron, loops, or external clocks while avoiding unsafe tight autonomous timers.

日本語の概要は準備中です。原文の説明を表示しています。

Use when working with Aws Control Tower — aWS Control Tower landing zone and account management. Covers landing zone status, enrolled accounts, guardrails (controls), organizational units, baseline configurations, and drift detection. Use when auditing landing zone health, reviewing guardrail compliance, inspecting account enrollment, or troubleshooting Control Tower drift.

日本語の概要は準備中です。原文の説明を表示しています。

Busabase App-in-Skill dashboard visualizing a generated MOCK fleet of LLM agents running behind a shared AI gateway for a generic organization. Use when the user invokes $kelly-agent-observability or /kelly-agent-observability, wants to review agent health, call volume, latency, error rates, cost, or trace-level chain breaks across an agent fleet. This is a demo/reference dashboard over a generated mock fleet snapshot seeded into Busabase by a trusted script — never a live monitoring integration, never a real gateway or real agents.

日本語の概要は準備中です。原文の説明を表示しています。

Deploying Cloudflare Access with Cloudflare Tunnel to provide zero trust access to self-hosted and private applications, configuring identity-aware access policies, device posture checks, and WARP client enrollment for VPN replacement.

日本語の概要は準備中です。原文の説明を表示しています。

Focused Signals scout for PostHog projects using AI observability. Rotates through a set of lenses — cost, latency, errors, volume, eval performance, eval/enrichment config, clusters, and tool usage — watching each for trends and spikes sliced by the dimensions it discovers over time. Leans on the sandbox's bundled `exploring-llm-*` deep-dive skills for the actual queries. Emits findings only when they clear the confidence bar; otherwise writes durable memory and closes out empty. Self-contained peer in the signals-scout-* fleet — no dependencies on other scouts.

日本語の概要は準備中です。原文の説明を表示しています。

Instrument a shipped product's AI agents with tracing and observability so you can see what they did, why outputs happened, and what each run cost. Plain-language primer plus free-tier-first backend selection (Langfuse, Phoenix, LangSmith, Braintrust) and OpenTelemetry/OpenInference instrumentation. Load when the user asks to add observability, add tracing, instrument my agents, see what my agent is doing in production, set up Langfuse or Phoenix or LangSmith, debug why my agent gave a bad answer, or track LLM cost per request. Also fires when agent-system-architecture or setup-evaluation requires an observability plan for an agent-chain product. NOT for tracing the coding agent itself — that is run-trace. Precondition for runtime-learning-loop.

日本語の概要は準備中です。原文の説明を表示しています。

Improve Coval trace quality after basic ingestion works. Use when traces are sparse, missing useful STT/LLM/TTS/tool spans, missing attributes needed for Coval built-in metrics, or when a customer wants maximum debugging and observability value from agent traces.

日本語の概要は準備中です。原文の説明を表示しています。

Recommend, create, preview, and attach Coval trace-based metrics from OpenTelemetry spans. Use when a user has Coval traces and wants custom trace metrics, trace-aware LLM judge metrics with include_traces, latency, token usage, provider failures, tool behavior, vertical-specific workflow signals, or production monitoring based on trace attributes.

日本語の概要は準備中です。原文の説明を表示しています。

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

日本語の概要は準備中です。原文の説明を表示しています。

Security review and threat work — threat modeling, vulnerability assessment, penetration test, incident response, OWASP Top 10, LLM Top 10, API Top 10, CVE triage, STRIDE, NIST CSF, SBOM. Auto-loops on any task touching PII, auth, payments or regulated data.

日本語の概要は準備中です。原文の説明を表示しています。

OpenInference semantic conventions and instrumentation for Phoenix AI observability. Use when implementing LLM tracing, creating custom spans, or deploying to production.

日本語の概要は準備中です。原文の説明を表示しています。

INVOKE THIS SKILL when creating, reading, updating, or deleting Arize AI integrations. Covers listing integrations, creating integrations for any supported LLM provider (OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Vertex AI, Gemini, NVIDIA NIM, custom), updating credentials or metadata, and deleting integrations using the ax CLI.

日本語の概要は準備中です。原文の説明を表示しています。

Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality

日本語の概要は準備中です。原文の説明を表示しています。

Deploying Cloudflare Access with Cloudflare Tunnel to provide zero trust access to self-hosted and private applications, configuring identity-aware access policies, device posture checks, and WARP client enrollment for VPN replacement.

日本語の概要は準備中です。原文の説明を表示しています。

Complete reference for 26 web2 bug classes with root causes, detection patterns, bypass tables, exploit techniques, and real paid examples. Covers IDOR, auth bypass, XSS, SSRF (11 IP bypass techniques), SQLi, business logic, race conditions, OAuth/OIDC, file upload (10 bypass techniques), GraphQL, LLM/AI (ASI01-ASI10 agentic framework), API misconfig (mass assignment, JWT attacks, prototype pollution, CORS), ATO taxonomy (9 paths), SSTI (Jinja2/Twig/Freemarker/ERB/Spring), subdomain takeover, cloud/infra misconfigs, HTTP smuggling (CL.TE/TE.CL/H2.CL), cache poisoning, MFA bypass (7 patterns), SAML attacks (XSW/comment injection/signature stripping), error disclosure / debug endpoints (stack trace regex per framework, chain templates), CSS injection (attribute-selector exfiltration, opacity clickjacking, @import). LFI / file inclusion -> RCE (php://filter source disclosure, iconv filter-chain RCE with no upload, log/environ poisoning, .user.ini/.htaccess auto_prepend, data:// + expect:// wrappers, session inclusion, traversal bypass table). Insecure deserialization (PHP __wakeup bypass / phar:// POP chains, Java ysoserial CommonsCollections gadgets + magic bytes, Python pickle __reduce__ + signed-cookie forgery, Node node-serialize). Dependency confusion / supply chain (internal package-name discovery, unclaimed-name confirmation, callback-only PoC, npm/pip/Maven/RubyGems variants). Use when hunting a specific vuln class or studying what makes bugs pay.

日本語の概要は準備中です。原文の説明を表示しています。

Active Directory attack methodology for internal network red team engagements. Covers reconnaissance (BloodHound, PowerView, ADExplorer), credential abuse (Kerberoasting, ASREProasting, NTLM relay, LLMNR/NBT-NS poisoning), privilege escalation (ACL abuse, GPO abuse, unconstrained/constrained delegation), lateral movement (Pass-the-Hash, Pass-the-Ticket, Overpass-the-Hash, WMI/WinRM/PsExec), persistence (Golden/Silver/Diamond Tickets, DCSync, DCShadow, AdminSDHolder, Skeleton Key), forest trust attacks, ADCS abuse (ESC1-ESC15), and modern MDI/Defender for Identity evasion. Use when assessing on-prem AD, hybrid AD/Entra ID environments, or ADCS deployments.

日本語の概要は準備中です。原文の説明を表示しています。

Prepare for Anthropic-specific technical interviews covering Constitutional AI, RLHF, interpretability, scaling laws, MCP, agentic systems, and AI safety. Activate on "anthropic interview", "constitutional AI prep", "alignment interview", "safety interview", "RLHF opinion", "interpretability prep", "anthropic technical". NOT for general ML interview prep, system design interviews, coding challenges, or behavioral interview practice.

日本語の概要は準備中です。原文の説明を表示しています。