Skip to content
cccskills
FreePublished on GitHub

cloud-trace-querying

Investigate slow or failing requests in Google Cloud using Cloud Trace. Filter traces by duration or error status, inspect spans representing individual operations, and retrieve related Cloud Logging entries to investigate bottlenecks and exceptions.

OriginalShow the original description

Query Cloud Trace spans, filter by latency thresholds or error status, correlate distributed traces with Cloud Logging, and diagnose latency bottlenecks across Google Cloud services. Use when investigating slow requests, analyzing trace hierarchies, or resolving latency regressions. Do NOT use for querying non-GCP telemetry or database query optimization outside Cloud Trace.

google/skills
See how to install

Good for

  • Find requests exceeding a latency threshold
  • Inspect operation hierarchies within a trace
  • Investigate failures using related application logs
  • Generate direct links to traces in Cloud Console

Files included (89)

  • SKILL.md6.7 KB
  • assets/formatting/a2ui/single_span/display.a2ui.json1.1 KB
  • assets/formatting/a2ui/trace_chart/display.a2ui.json1.1 KB
  • assets/formatting/a2ui/trace_flow/display.a2ui.json767 B
  • assets/formatting/a2ui/traces_heat_map/display.a2ui.json1.8 KB
  • assets/formatting/ascii/single_span/display.txt377 B
  • assets/formatting/ascii/trace_chart/display.txt517 B
  • assets/formatting/ascii/trace_flow/display.txt430 B
  • assets/formatting/ascii/traces_heat_map/display.txt887 B
  • assets/queries/billing_errors/query.txt73 B
  • assets/queries/billing_errors/SCENARIO.md1.4 KB
  • assets/queries/checkout_latency/query.txt47 B
  • assets/queries/checkout_latency/SCENARIO.md416 B
  • assets/queries/customer_spans/query.txt24 B
  • assets/queries/customer_spans/SCENARIO.md307 B
  • assets/queries/prod_outliers/query.txt51 B
  • assets/queries/prod_outliers/SCENARIO.md979 B
  • assets/queries/slow_db/query.txt35 B
  • assets/queries/slow_db/SCENARIO.md440 B
  • references/api-limits.md1.6 KB
  • references/concepts-span.md869 B
  • references/concepts-trace.md1.3 KB
  • references/console-deep-links.md1.1 KB
  • references/conventions-gcp-legacy.md3.4 KB
  • references/conventions-gcp.md2.7 KB
  • references/conventions-otel-genai.md6.9 KB
  • references/conventions-otel-grpc.md2.4 KB
  • references/conventions-otel-http.md1.9 KB
  • references/conventions-otel.md4.2 KB
  • references/conventions.md2.5 KB
  • references/correlated-logs.md2.5 KB
  • references/decision-tree.md2.1 KB
  • references/filter-syntax.md4.0 KB
  • references/index.md2.7 KB
  • references/knowledge-overview.md3.2 KB
  • references/logs-trace-correlation.md1.7 KB
  • references/observability-scopes.md3.0 KB
  • references/querying-mcp.md2.0 KB
  • references/querying-observability-analytics.md4.9 KB
  • references/troubleshoot-latency.md2.1 KB
  • references/workflows-overview.md881 B
  • scripts/fetch_entire_trace/run.sh231 B
  • scripts/fetch_entire_trace/src/main.py1.8 KB
  • scripts/fetch_entire_trace/test.sh279 B
  • scripts/fetch_entire_trace/tests/main_test.py3.9 KB
  • scripts/fetch_related_logs/run.sh231 B
  • scripts/fetch_related_logs/src/log_fetcher.py7.4 KB
  • scripts/fetch_related_logs/src/main.py2.4 KB
  • scripts/fetch_related_logs/src/observability_scopes.py3.0 KB
  • scripts/fetch_related_logs/src/timestamp_utils.py577 B
  • scripts/fetch_related_logs/test.sh422 B
  • scripts/fetch_related_logs/tests/main_test.py10.6 KB
  • scripts/fetch_related_logs/tests/timestamp_utils_test.py1.1 KB
  • scripts/fetch_trace_span/run.sh231 B
  • scripts/fetch_trace_span/src/main.py2.6 KB
  • scripts/fetch_trace_span/test.sh279 B
  • scripts/fetch_trace_span/tests/main_test.py5.1 KB
  • scripts/generate_links/run.sh138 B
  • scripts/generate_links/src/main.py1.7 KB
  • scripts/generate_links/test.sh279 B
  • scripts/generate_links/tests/main_test.py2.7 KB
  • scripts/get_trace_project/run.sh231 B
  • scripts/get_trace_project/src/main.py402 B
  • scripts/get_trace_project/src/project_resolver.py2.0 KB
  • scripts/get_trace_project/test.sh279 B
  • scripts/get_trace_project/tests/integration_test.py2.7 KB
  • scripts/get_trace_project/tests/project_resolver_test.py2.8 KB
  • scripts/requirements.txt92 B
  • scripts/run_python_with_deps/run.sh2.0 KB
  • scripts/run_python_with_deps/src/cleanup_env.py2.1 KB
  • scripts/run_python_with_deps/src/lock_manager.py2.5 KB
  • scripts/run_python_with_deps/src/manage_venv.py1.7 KB
  • scripts/run_python_with_deps/src/path_resolver.py3.4 KB
  • scripts/run_python_with_deps/src/venv_cleaner.py800 B
  • scripts/run_python_with_deps/src/venv_executor.py1.2 KB
  • scripts/run_python_with_deps/src/venv_setup.py2.1 KB
  • scripts/run_python_with_deps/test.sh364 B
  • scripts/run_python_with_deps/tests/cleanup_env_test.py2.5 KB
  • scripts/run_python_with_deps/tests/lock_manager_test.py2.0 KB
  • scripts/run_python_with_deps/tests/path_resolver_test.py5.4 KB
  • scripts/run_python_with_deps/tests/venv_setup_test.py1.8 KB
  • scripts/search_traces/run.sh231 B
  • scripts/search_traces/src/main.py2.1 KB
  • scripts/search_traces/src/timestamp_utils.py1.5 KB
  • scripts/search_traces/src/trace_searcher.py16.0 KB
  • scripts/search_traces/test.sh279 B
  • scripts/search_traces/tests/main_test.py2.7 KB
  • scripts/search_traces/tests/timestamp_utils_test.py1.7 KB
  • scripts/search_traces/tests/trace_searcher_test.py17.4 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

Cloud Trace Querying Skill (cloud-trace-querying)

This skill equips the agent to interact with Cloud Trace by providing standard query patterns, diagnostic workflows for latency and errors, and tools to fetch and inspect trace hierarchies.

[!IMPORTANT] AUTONOMOUS TOOL EXECUTION MANDATE: You MUST execute the tool scripts directly in bash to query telemetry. Do NOT just describe query strings or write Python scripts. Run the provided bash tools directly and report the exact results.

Prerequisites & Setup

  1. Google Cloud SDK: Verify gcloud is installed (Installation Guide).
  2. Authentication: Authenticate CLI and application default credentials: 
    gcloud auth login && gcloud auth application-default login
  3. Project & Billing: Identify the target <PROJECT_ID> (from the user's prompt or via bash scripts/get_trace_project/run.sh), ensure an active billing account is attached, and configure the project if not already set: 
    gcloud config set project <PROJECT_ID>
  4. API Enablement: Verify Cloud Trace and Cloud Logging APIs are enabled: 
    gcloud services enable cloudtrace.googleapis.com logging.googleapis.com

Quickstart Workflows

1. Identify Google Cloud Project ID

If the user specified a Google Cloud project ID in their prompt, use it.

If they did not, ALWAYS determine the Google Cloud project ID by running:

bash scripts/get_trace_project/run.sh

2. Search Traces (Latency & Errors)

To find traces matching specific criteria, such as a root span of /v1/booking/book with a duration of at least 1.5s:

bash scripts/search_traces/run.sh --filter 'root:/v1/booking/book latency:1.5s' --projects '<PROJECT_ID>'

Note: --filter is required. To list traces without specific filtering, pass --filter '' or --filter 'root:'. Filter syntax: The Cloud Trace API uses latency:<duration> (such as latency:1.5s for duration >= 1.5s), which corresponds to "Span duration" in Trace Explorer. The script outputs JSON where each cluster's examples array contains sample trace objects with trace_id, duration_seconds, span count, and error status.

3. Fetch Entire Trace Hierarchy

To view all spans and status codes/labels within a trace:

bash scripts/fetch_entire_trace/run.sh --trace-id '<TRACE_ID>' --project '<PROJECT_ID>'

4. Fetch Correlated Application Logs

To retrieve application log entries correlated with a trace ID:

bash scripts/fetch_related_logs/run.sh --trace-id '<TRACE_ID>' --trace-projects '<PROJECT_ID>'

Application log entries are useful when inspecting stack traces, exception messages, and log events emitted during span execution. When diagnosing errors or service crashes, inspect the returned log entries for severity: "ERROR" to confirm root causes or uncover unhandled exceptions. Note: fetch_related_logs queries the default log scope (_Default), searching across configured log views in the target project. An empty response can occur if: (1) no log entries match the criteria, (2) the caller lacks IAM permissions to view the matching entries, or (3) matching entries reside in log views outside the searched log scope.

5. Fetch Specific Trace Span

To inspect attributes and metadata of a single span within a trace:

bash scripts/fetch_trace_span/run.sh --trace-id '<TRACE_ID>' --span-id '<SPAN_ID>' --project '<PROJECT_ID>'

6. Generate Cloud Console Deep Links

To generate direct Cloud Console URLs for browser inspection of traces or correlated logs:

bash scripts/generate_links/run.sh --trace-id '<TRACE_ID>' --project '<PROJECT_ID>' [--span-id '<SPAN_ID>']

7. Format Output

  • Terminal / CLI: Render span trees as indented ASCII timelines using |--.
  • UI Presentation: Render as A2UI JSON graph structures when requested for UI.

8. Upstream Service Error Handling

If the Cloud Trace API or Cloud Logging API returns an upstream HTTP service error (such as 500 Internal Server Error, 503 Service Unavailable, or GoogleAPICallError), report the upstream service outage directly to the user. Do not attempt to inspect virtual environments, modify internal scripts, or debug local libraries.

Reference Documentation

Operational Workflows

Query Syntax & Tools

Semantic Conventions

Concepts & Architecture

Templates & Assets

Official Documentation Links

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

agent-platform-alert-configuration

FreeSummary

Create Terraform files that define alerts for AI agent latency, errors, token use, safety, security, and response quality. Check telemetry and existing policies first, then validate the generated configuration. Quality alerts require Vertex AI.

  • Configure alerts for agent latency and error rates
  • Monitor rapid increases in token usage
  • Monitor safety triggers and permission denials
google/skills21.4KUpdated Oct 17, 2026

agent-platform-deploy

FreeSummary

Deploy Model Garden models to Agent Platform endpoints, the addresses used for model requests. Check supported hardware and estimated hourly costs, inspect deployment progress, test predictions, and remove deployed resources after use.

  • Choose a deployable model and supported hardware
  • Estimate hourly deployment costs before approval
  • Check deployment progress and test predictions
google/skills21.4KUpdated Oct 17, 2026

agent-platform-endpoint-management

FreeSummary

Manages Agent Platform endpoints, the stable URLs used to serve online model predictions. It supports listing, inspecting, creating, updating, and deleting endpoints, plus troubleshooting permission errors, quota limits, and resources that are busy.

  • Listing endpoints and checking their configuration
  • Creating an endpoint before model deployment
  • Updating endpoint display names or labels
google/skills21.4KUpdated Oct 17, 2026

agent-platform-eval-flywheel

FreeSummary

Evaluate AI models and agents on Google Cloud using test datasets and scoring criteria. Generate simulated user scenarios, analyze failures, and compare results across changes to guide improvements to prompts and code.

  • Build evaluation datasets from conversation logs
  • Generate simulated user scenarios
  • Choose scoring criteria for responses and tool use
google/skills21.4KUpdated Oct 17, 2026

agent-platform-inference

FreeSummary

Call Gemini and OpenMaaS models such as Llama, DeepSeek, and Qwen through Google Cloud Agent Platform. Prepare authentication and SDK code, run prompts, configure connection endpoints, and troubleshoot common request errors.

  • Send a test prompt to Gemini
  • Call Llama or DeepSeek with the OpenAI SDK
  • Write code to call an existing custom endpoint
google/skills21.4KUpdated Oct 17, 2026

agent-platform-migrate-from-ai-studio

FreeSummary

Guides migration from Google AI Studio’s Gemini API to Gemini Enterprise Agent Platform on Google Cloud, covering billing, access permissions, authentication, and connection settings for SDKs and supported agent tools.

  • Moving a Gemini application to Google Cloud
  • Using Google Cloud credits for Gemini API calls
  • Unifying agent authentication and billing
google/skills21.4KUpdated Oct 17, 2026

See all skills by google

Report a problem with this skill