Grafana — Setup & Verification Guide

This skill teaches setup-gen and reward-gen how to create, manipulate, and verify Grafana state for monitoring/observability tasks. Grafana tasks involve HTTP state management through a shared Grafana instance.

• Libraries: requests, json, uuid, copy
• Grafana instance: https://cua-gym-grafana.xlang.ai
• Admin credentials: admin / cua-gym-admin-2024
• Grafana version: 12.4.1

---

0. Architecture Overview

Admin creates session org + user  →  User logs in via browser  →  CUA agent interacts with Grafana UI
         ↓                                    ↓                              ↓
  Inject dashboards/alerts            setup-gen configures state      reward-gen verifies via API
         ↓                                    ↓                              ↓
  TestData datasource               initial_setup.py runs              reward.py reads back state
         ↓                                                                   ↓
  csv_content = controlled data                                   DELETE org + user (cleanup)

Key design: Each training session gets its own Grafana Organization for complete isolation. Dashboards, datasources, alerts, and folders are org-scoped — a user in org N cannot see org M's resources.

Key difference from mock_websites: Grafana is a real application (not a mock). State is injected via Grafana's native REST API, not a custom /post?sid= endpoint. Verification reads state back via the same API.

---

1. Session Isolation via Organizations

Every training episode uses a dedicated Grafana Organization with its own user. This provides complete state isolation.

1.1 Session Setup Flow

import requests
import uuid
import json

GRAFANA_URL = 'https://cua-gym-grafana.xlang.ai'
ADMIN_AUTH = ('admin', 'cua-gym-admin-2024')

session_id = str(uuid.uuid4())[:8]
org_name = f'session_{session_id}'
user_login = f'agent_{session_id}'
user_password = f'agent_pass_{session_id}'
user_email = f'{user_login}@cua-gym.local'

# 1. Create organization
resp = requests.post(
    f'{GRAFANA_URL}/api/orgs',
    json={'name': org_name},
    auth=ADMIN_AUTH, timeout=15
)
assert resp.status_code == 200, f'Org creation failed: {resp.text}'
org_id = resp.json()['orgId']

# 2. Create user
resp = requests.post(
    f'{GRAFANA_URL}/api/admin/users',
    json={
        'name': f'Agent {session_id}',
        'login': user_login,
        'password': user_password,
        'email': user_email,
    },
    auth=ADMIN_AUTH, timeout=15
)
assert resp.status_code == 200, f'User creation failed: {resp.text}'
user_id = resp.json()['id']

# 3. Add user to org as Editor
resp = requests.post(
    f'{GRAFANA_URL}/api/orgs/{org_id}/users',
    json={'loginOrEmail': user_login, 'role': 'Editor'},
    auth=ADMIN_AUTH, timeout=15
)
assert resp.status_code == 200

# 4. Switch user's active org
resp = requests.post(
    f'{GRAFANA_URL}/api/users/{user_id}/using/{org_id}',
    auth=ADMIN_AUTH, timeout=15
)
assert resp.status_code == 200

# 5. Create TestData datasource in the new org
resp = requests.post(
    f'{GRAFANA_URL}/api/datasources',
    json={
        'name': 'TestData',
        'type': 'grafana-testdata-datasource',
        'access': 'proxy',
        'isDefault': True,
    },
    auth=ADMIN_AUTH,
    headers={'X-Grafana-Org-Id': str(org_id)},
    timeout=15
)
assert resp.status_code == 200
ds_uid = resp.json()['datasource']['uid']

1.2 Session Cleanup

def cleanup_session(org_id, user_id):
    """Delete org and user. Order matters: org first, then user."""
    requests.delete(f'{GRAFANA_URL}/api/orgs/{org_id}', auth=ADMIN_AUTH, timeout=15)
    requests.delete(f'{GRAFANA_URL}/api/admin/users/{user_id}', auth=ADMIN_AUTH, timeout=15)

1.3 Persisting Session Info

# Save session info for golden_patch.py and reward.py
session_info = {
    'grafana_url': GRAFANA_URL,
    'org_id': org_id,
    'org_name': org_name,
    'user_id': user_id,
    'user_login': user_login,
    'user_password': user_password,
    'ds_uid': ds_uid,
}
with open('/tmp/task_grafana_session', 'w') as f:
    json.dump(session_info, f)

---

2. Dashboard API

The primary setup and verification surface. Dashboards are JSON documents containing panels, variables, and layout.

2.1 Create Dashboard

def create_dashboard(dashboard_json, org_id, folder_uid=''):
    """Create or overwrite a dashboard."""
    payload = {
        'dashboard': dashboard_json,
        'overwrite': True,
        'folderUid': folder_uid,
    }
    resp = requests.post(
        f'{GRAFANA_URL}/api/dashboards/db',
        json=payload,
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    assert resp.status_code == 200, f'Dashboard creation failed: {resp.text}'
    return resp.json()  # {'uid', 'url', 'id', 'status', 'version'}

2.2 Read Dashboard (for reward verification)

def get_dashboard(uid, org_id):
    """Read back a dashboard. Returns full JSON model."""
    resp = requests.get(
        f'{GRAFANA_URL}/api/dashboards/uid/{uid}',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    assert resp.status_code == 200, f'Dashboard read failed: {resp.text}'
    result = resp.json()
    return result['dashboard'], result['meta']
    # dashboard = full JSON model (panels, templating, time, tags, etc.)
    # meta = {folderUid, folderTitle, slug, url, version, ...}

2.3 Delete Dashboard

def delete_dashboard(uid, org_id):
    resp = requests.delete(
        f'{GRAFANA_URL}/api/dashboards/uid/{uid}',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.status_code == 200

2.4 List Dashboards (search)

def search_dashboards(org_id, query='', tag=''):
    params = {}
    if query:
        params['query'] = query
    if tag:
        params['tag'] = tag
    resp = requests.get(
        f'{GRAFANA_URL}/api/search',
        params=params,
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()  # [{uid, title, url, tags, type, folderUid, ...}]

---

3. Controlled Data via csv_content

The TestData datasource's csv_content scenario embeds exact data inline in the dashboard JSON. This makes data fully deterministic and verifiable.

3.1 CSV Content Target

def make_csv_target(ref_id, csv_data):
    """Create a panel target with exact inline data.

    csv_data: str with header row + data rows.
    First column should be 'time' for timeseries panels.
    """
    return {
        'refId': ref_id,
        'scenarioId': 'csv_content',
        'csvContent': csv_data,
    }

# Example: CPU metrics for 3 servers over 8 hours
csv = (
    "time,web-server-01,web-server-02,db-primary\n"
    "2024-01-15T00:00:00Z,25,30,45\n"
    "2024-01-15T01:00:00Z,28,32,48\n"
    "2024-01-15T02:00:00Z,35,29,52\n"
    "2024-01-15T03:00:00Z,72,31,55\n"
    "2024-01-15T04:00:00Z,95,33,90\n"  # anomaly!
    "2024-01-15T05:00:00Z,88,35,85\n"
    "2024-01-15T06:00:00Z,42,30,50\n"
    "2024-01-15T07:00:00Z,30,28,47"
)
target = make_csv_target('A', csv)

3.2 Random Walk Target (for visual richness, non-verifiable data)

def make_random_walk_target(ref_id, alias, start_value=50, spread=10):
    """Random walk — produces live-updating data. NOT deterministic."""
    return {
        'refId': ref_id,
        'scenarioId': 'random_walk',
        'alias': alias,
        'seriesCount': 1,
        'startValue': start_value,
        'spread': spread,
    }

3.3 When to Use Which

Scenario	Use Case	Verifiable?
csv_content	Task requires specific data patterns (anomalies, trends)	Yes — data is in the JSON
random_walk	Visual richness, background panels	No — data changes on each load

---

4. Panel Types & Configuration

4.1 Supported Panel Types

Type Key	Display Name	Typical Use
timeseries	Time Series	Metrics over time (default)
stat	Stat	Single big number with sparkline
gauge	Gauge	Dial showing value vs threshold
bargauge	Bar Gauge	Horizontal/vertical bar
table	Table	Tabular data display
barchart	Bar Chart	Categorical comparison
piechart	Pie Chart	Proportional breakdown
heatmap	Heatmap	2D density visualization
histogram	Histogram	Value distribution
state-timeline	State Timeline	State changes over time
status-history	Status History	Periodic state display
geomap	Geomap	Geographic data
candlestick	Candlestick	Financial data
xychart	XY Chart	Arbitrary x/y plotting
canvas	Canvas	Free-form layout
text	Text	Markdown/HTML content
logs	Logs	Log data display
nodeGraph	Node Graph	Network/graph visualization
alertlist	Alert List	Active alerts display
dashlist	Dashboard List	Dashboard links
news	News	RSS feed

4.2 Panel JSON Structure

def make_panel(panel_id, title, panel_type, grid_x, grid_y, grid_w, grid_h,
               targets, ds_uid, field_config=None, options=None, transformations=None):
    """Build a complete panel JSON object."""
    panel = {
        'id': panel_id,
        'title': title,
        'type': panel_type,
        'gridPos': {'x': grid_x, 'y': grid_y, 'w': grid_w, 'h': grid_h},
        'datasource': {
            'type': 'grafana-testdata-datasource',
            'uid': ds_uid,
        },
        'targets': targets,
        'fieldConfig': field_config or {
            'defaults': {},
            'overrides': [],
        },
        'options': options or {},
    }
    if transformations:
        panel['transformations'] = transformations
    return panel

4.3 Field Config — Units, Thresholds, Overrides

# Standard field config with thresholds
field_config = {
    'defaults': {
        'unit': 'percent',        # bytes, ms, reqps, short, etc.
        'decimals': 1,
        'min': 0,
        'max': 100,
        'color': {'mode': 'palette-classic'},
        'custom': {
            'lineWidth': 2,
            'fillOpacity': 10,
            'stacking': {'mode': 'none'},  # 'normal' for stacked
        },
        'thresholds': {
            'mode': 'absolute',    # or 'percentage'
            'steps': [
                {'color': 'green', 'value': None},   # base (always None)
                {'color': 'yellow', 'value': 60},
                {'color': 'red', 'value': 80},
            ],
        },
        'links': [
            # data links
            {
                'title': 'View Details',
                'url': 'https://example.com/details?server=${__field.name}',
                'targetBlank': True,
            },
        ],
    },
    'overrides': [
        {
            'matcher': {'id': 'byName', 'options': 'web-server-01'},
            'properties': [
                {'id': 'color', 'value': {'fixedColor': 'red', 'mode': 'fixed'}},
                {'id': 'custom.lineWidth', 'value': 3},
            ],
        },
    ],
}

4.4 Panel Options (per visualization type)

# Time series options
timeseries_options = {
    'tooltip': {'mode': 'multi'},           # 'single', 'multi', 'none'
    'legend': {
        'displayMode': 'table',             # 'list', 'table', 'hidden'
        'placement': 'bottom',              # 'bottom', 'right'
        'calcs': ['mean', 'max', 'last'],   # legend calculations
    },
}

# Stat options
stat_options = {
    'graphMode': 'area',        # 'area', 'none'
    'colorMode': 'background',  # 'none', 'value', 'background'
    'textMode': 'value',        # 'auto', 'value', 'value_and_name', 'name', 'none'
    'orientation': 'auto',      # 'auto', 'horizontal', 'vertical'
}

# Gauge options
gauge_options = {
    'showThresholdLabels': True,
    'showThresholdMarkers': True,
    'orientation': 'auto',
}

# Pie chart options
pie_options = {
    'pieType': 'pie',           # 'pie', 'donut'
    'tooltip': {'mode': 'single'},
    'legend': {'displayMode': 'list', 'placement': 'right'},
}

# Bar chart options
bar_options = {
    'orientation': 'auto',      # 'auto', 'horizontal', 'vertical'
    'stacking': 'none',         # 'none', 'normal', 'percent'
    'showValue': 'auto',        # 'auto', 'always', 'never'
    'barWidth': 0.97,
    'groupWidth': 0.7,
}

# Table options
table_options = {
    'showHeader': True,
    'footer': {'show': False},
    'cellHeight': 'sm',         # 'sm', 'md', 'lg'
}

4.5 Transformations

# Organize fields (rename, reorder, hide)
transform_organize = {
    'id': 'organize',
    'options': {
        'excludeByName': {'Time': True},
        'renameByName': {'web-server-01': 'Web Server 1'},
        'indexByName': {'web-server-01': 0, 'web-server-02': 1},
    },
}

# Filter by value
transform_filter = {
    'id': 'filterByValue',
    'options': {
        'filters': [
            {
                'fieldName': 'web-server-01',
                'config': {
                    'id': 'greater',
                    'options': {'value': 50},
                },
            },
        ],
        'type': 'include',
        'match': 'any',
    },
}

---

5. Template Variables

Dashboard variables enable dynamic filtering via dropdown menus.

5.1 Custom Variable (static options)

templating = {
    'list': [
        {
            'type': 'custom',
            'name': 'server',
            'label': 'Server',
            'query': 'web-server-01,web-server-02,web-server-03',
            'current': {'text': 'web-server-01', 'value': 'web-server-01'},
            'options': [
                {'text': 'web-server-01', 'value': 'web-server-01', 'selected': True},
                {'text': 'web-server-02', 'value': 'web-server-02', 'selected': False},
                {'text': 'web-server-03', 'value': 'web-server-03', 'selected': False},
            ],
            'multi': False,          # True for multi-select
            'includeAll': False,     # True to add "All" option
            'hide': 0,              # 0=show, 1=hide label, 2=hide entirely
        },
        {
            'type': 'interval',
            'name': 'interval',
            'label': 'Interval',
            'query': '1m,5m,15m,30m,1h',
            'current': {'text': '5m', 'value': '5m'},
            'auto': False,
            'hide': 0,
        },
    ],
}

---

6. Alerting API

6.1 Prerequisites

Alert rules require a folder. Always create the folder first.

def create_folder(title, uid, org_id):
    resp = requests.post(
        f'{GRAFANA_URL}/api/folders',
        json={'title': title, 'uid': uid},
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    assert resp.status_code == 200, f'Folder creation failed: {resp.text}'
    return resp.json()

6.2 Create Alert Rule

def create_alert_rule(title, folder_uid, ds_uid, org_id,
                      threshold=80, pending_for='5m',
                      labels=None, annotations=None):
    """Create a threshold-based alert rule."""
    rule = {
        'title': title,
        'ruleGroup': 'cua-gym-alerts',
        'folderUID': folder_uid,
        'condition': 'C',
        'data': [
            {
                'refId': 'A',
                'relativeTimeRange': {'from': 600, 'to': 0},
                'datasourceUid': ds_uid,
                'model': {
                    'scenarioId': 'random_walk',
                    'seriesCount': 1,
                    'startValue': 50,
                    'spread': 20,
                    'refId': 'A',
                },
            },
            {
                'refId': 'C',
                'relativeTimeRange': {'from': 600, 'to': 0},
                'datasourceUid': '__expr__',
                'model': {
                    'type': 'threshold',
                    'expression': 'A',
                    'conditions': [
                        {
                            'evaluator': {
                                'type': 'gt',
                                'params': [threshold],
                            },
                        },
                    ],
                    'refId': 'C',
                },
            },
        ],
        'for': pending_for,
        'noDataState': 'NoData',
        'execErrState': 'Error',
        'labels': labels or {},
        'annotations': annotations or {},
    }
    resp = requests.post(
        f'{GRAFANA_URL}/api/v1/provisioning/alert-rules',
        json=rule,
        auth=ADMIN_AUTH,
        headers={
            'X-Grafana-Org-Id': str(org_id),
            'X-Disable-Provenance': 'true',
        },
        timeout=15,
    )
    assert resp.status_code == 201, f'Alert rule creation failed: {resp.text}'
    return resp.json()  # includes 'uid'

6.3 Read Alert Rule (for reward verification)

def get_alert_rule(rule_uid, org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/v1/provisioning/alert-rules/{rule_uid}',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    assert resp.status_code == 200
    return resp.json()
    # Fields: uid, title, condition, data, for, labels, annotations, isPaused, ...

6.4 List All Alert Rules

def list_alert_rules(org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/v1/provisioning/alert-rules',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()  # list of alert rule objects

6.5 Delete Alert Rule

def delete_alert_rule(rule_uid, org_id):
    requests.delete(
        f'{GRAFANA_URL}/api/v1/provisioning/alert-rules/{rule_uid}',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )

---

7. Contact Points & Notification Policies

7.1 Create Contact Point

def create_contact_point(name, cp_type, settings, org_id):
    """Create a contact point.

    cp_type: 'email', 'webhook', 'slack', etc.
    settings: type-specific dict (see examples below).
    """
    resp = requests.post(
        f'{GRAFANA_URL}/api/v1/provisioning/contact-points',
        json={
            'name': name,
            'type': cp_type,
            'settings': settings,
            'disableResolveMessage': False,
        },
        auth=ADMIN_AUTH,
        headers={
            'X-Grafana-Org-Id': str(org_id),
            'X-Disable-Provenance': 'true',
        },
        timeout=15,
    )
    assert resp.status_code == 202, f'Contact point creation failed: {resp.text}'
    return resp.json()

# Email settings
email_settings = {'addresses': 'oncall@example.com;alerts@example.com'}

# Webhook settings
webhook_settings = {
    'url': 'https://hooks.example.com/grafana',
    'httpMethod': 'POST',
}

7.2 Read Contact Points (for verification)

def list_contact_points(org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/v1/provisioning/contact-points',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()  # list of {uid, name, type, settings, ...}

7.3 Notification Policy Tree

def get_notification_policies(org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/v1/provisioning/policies',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()  # full routing tree

def update_notification_policies(policy_tree, org_id):
    """WARNING: This replaces the ENTIRE policy tree.
    Always GET first to preserve the root receiver."""
    resp = requests.put(
        f'{GRAFANA_URL}/api/v1/provisioning/policies',
        json=policy_tree,
        auth=ADMIN_AUTH,
        headers={
            'X-Grafana-Org-Id': str(org_id),
            'X-Disable-Provenance': 'true',
        },
        timeout=15,
    )
    assert resp.status_code == 202

# Example: route critical alerts to a specific contact point
policy_tree = {
    'receiver': 'grafana-default-email',  # keep existing default
    'group_by': ['alertname'],
    'routes': [
        {
            'receiver': 'oncall-webhook',
            'object_matchers': [
                ['severity', '=', 'critical'],
            ],
            'continue': False,
        },
    ],
}

---

8. Mute Timings

def create_mute_timing(name, time_intervals, org_id):
    resp = requests.post(
        f'{GRAFANA_URL}/api/v1/provisioning/mute-timings',
        json={'name': name, 'time_intervals': time_intervals},
        auth=ADMIN_AUTH,
        headers={
            'X-Grafana-Org-Id': str(org_id),
            'X-Disable-Provenance': 'true',
        },
        timeout=15,
    )
    assert resp.status_code == 201
    return resp.json()

# Weekend mute
weekend_intervals = [
    {
        'weekdays': ['saturday', 'sunday'],
        'times': [{'start_time': '00:00', 'end_time': '23:59'}],
    },
]

# Maintenance window
maintenance_intervals = [
    {
        'weekdays': ['wednesday'],
        'times': [{'start_time': '02:00', 'end_time': '06:00'}],
    },
]

def list_mute_timings(org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/v1/provisioning/mute-timings',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()

---

9. Annotations

def create_annotation(text, tags, org_id, dashboard_uid=None, panel_id=None,
                      time_ms=None, time_end_ms=None):
    body = {'text': text, 'tags': tags}
    if dashboard_uid:
        body['dashboardUID'] = dashboard_uid
    if panel_id:
        body['panelId'] = panel_id
    if time_ms:
        body['time'] = time_ms
    if time_end_ms:
        body['timeEnd'] = time_end_ms
    resp = requests.post(
        f'{GRAFANA_URL}/api/annotations',
        json=body,
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    assert resp.status_code == 200
    return resp.json()  # {'id', 'message'}

def query_annotations(org_id, tags=None, dashboard_uid=None):
    """Query annotations. Tags use repeated params: tags=a&tags=b."""
    params = {}
    if dashboard_uid:
        params['dashboardUID'] = dashboard_uid
    resp = requests.get(
        f'{GRAFANA_URL}/api/annotations',
        params=params,
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    results = resp.json()
    if tags:
        results = [a for a in results if set(tags).issubset(set(a.get('tags', [])))]
    return results

---

10. Folders

def create_folder(title, uid, org_id, parent_uid=None):
    body = {'title': title, 'uid': uid}
    if parent_uid:
        body['parentUid'] = parent_uid
    resp = requests.post(
        f'{GRAFANA_URL}/api/folders',
        json=body,
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    assert resp.status_code == 200, f'Folder creation failed: {resp.text}'
    return resp.json()

def get_folder(uid, org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/folders/{uid}',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()

def set_folder_permissions(uid, org_id, items):
    """WARNING: This replaces ALL permissions.
    Always include admin user: {'userId': 1, 'permission': 4}
    or you lose access to the folder."""
    resp = requests.post(
        f'{GRAFANA_URL}/api/folders/{uid}/permissions',
        json={'items': items},
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()

def get_folder_permissions(uid, org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/folders/{uid}/permissions',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()

---

11. Playlists

def create_playlist(title, interval, dashboard_uids, org_id):
    items = [
        {'type': 'dashboard_by_uid', 'value': uid}
        for uid in dashboard_uids
    ]
    resp = requests.post(
        f'{GRAFANA_URL}/api/playlists',
        json={'name': title, 'interval': interval, 'items': items},
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    assert resp.status_code == 200
    return resp.json()

def get_playlist(uid, org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/playlists/{uid}',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()

---

12. Snapshots

def create_snapshot(dashboard_json, name, org_id, expires=3600):
    resp = requests.post(
        f'{GRAFANA_URL}/api/snapshots',
        json={
            'dashboard': dashboard_json,
            'name': name,
            'expires': expires,
        },
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    assert resp.status_code == 200
    return resp.json()  # {'key', 'deleteKey', 'url'}

def get_snapshot(key, org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/snapshots/{key}',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()  # includes full 'dashboard' JSON

---

13. Library Panels

def create_library_panel(name, model, org_id, folder_uid=''):
    resp = requests.post(
        f'{GRAFANA_URL}/api/library-elements',
        json={
            'name': name,
            'kind': 1,  # 1 = panel
            'model': model,
            'folderUid': folder_uid,
        },
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    assert resp.status_code == 200
    return resp.json()['result']  # includes 'uid'

def get_library_panel(uid, org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/library-elements/{uid}',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()['result']

def get_library_panel_connections(uid, org_id):
    """Check which dashboards use this library panel."""
    resp = requests.get(
        f'{GRAFANA_URL}/api/library-elements/{uid}/connections/',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()['result']

---

14. Dashboard Version History

def get_dashboard_versions(uid, org_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/dashboards/uid/{uid}/versions',
        auth=ADMIN_AUTH,
        headers={'X-Grafana-Org-Id': str(org_id)},
        timeout=15,
    )
    return resp.json()['versions']  # list of {id, version, message, created, data, ...}

---

15. User & Team Management

These APIs are for tasks where the CUA agent is expected to manage users/teams in the Grafana UI.

def create_team(name, email, org_id):
    # Admin must be in the target org to create a team
    requests.post(
        f'{GRAFANA_URL}/api/user/using/{org_id}',
        auth=ADMIN_AUTH, timeout=15,
    )
    resp = requests.post(
        f'{GRAFANA_URL}/api/teams',
        json={'name': name, 'email': email},
        auth=ADMIN_AUTH, timeout=15,
    )
    assert resp.status_code == 200
    return resp.json()  # {'teamId', 'message'}

def get_team(team_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/teams/{team_id}',
        auth=ADMIN_AUTH, timeout=15,
    )
    return resp.json()

def add_team_member(team_id, user_id):
    resp = requests.post(
        f'{GRAFANA_URL}/api/teams/{team_id}/members',
        json={'userId': user_id},
        auth=ADMIN_AUTH, timeout=15,
    )
    return resp.json()

def get_team_members(team_id):
    resp = requests.get(
        f'{GRAFANA_URL}/api/teams/{team_id}/members',
        auth=ADMIN_AUTH, timeout=15,
    )
    return resp.json()

def set_org_preferences(org_id, theme=None, timezone=None, home_dashboard_uid=None):
    """Set org preferences. Must switch admin to target org first."""
    requests.post(
        f'{GRAFANA_URL}/api/user/using/{org_id}',
        auth=ADMIN_AUTH, timeout=15,
    )
    prefs = {}
    if theme:
        prefs['theme'] = theme
    if timezone:
        prefs['timezone'] = timezone
    if home_dashboard_uid:
        prefs['homeDashboardUID'] = home_dashboard_uid
    resp = requests.patch(
        f'{GRAFANA_URL}/api/org/preferences',
        json=prefs,
        auth=ADMIN_AUTH, timeout=15,
    )
    return resp.json()

def get_org_preferences(org_id):
    requests.post(
        f'{GRAFANA_URL}/api/user/using/{org_id}',
        auth=ADMIN_AUTH, timeout=15,
    )
    resp = requests.get(
        f'{GRAFANA_URL}/api/org/preferences',
        auth=ADMIN_AUTH, timeout=15,
    )
    return resp.json()

---

16. initial_setup.py Template

"""
Initial Setup: <task_description>
Task ID: <task_id>
Domain: grafana
"""
import json
import os
import shlex
import subprocess
import time
import uuid

import requests

# --- Config ---
GRAFANA_URL = 'https://cua-gym-grafana.xlang.ai'
ADMIN_AUTH = ('admin', 'cua-gym-admin-2024')
session_id = str(uuid.uuid4())[:8]

# --- Create isolated session ---
org_name = f'session_{session_id}'
user_login = f'agent_{session_id}'
user_password = f'agent_pass_{session_id}'

# Create org
resp = requests.post(f'{GRAFANA_URL}/api/orgs', json={'name': org_name},
                     auth=ADMIN_AUTH, timeout=15)
assert resp.status_code == 200
org_id = resp.json()['orgId']

# Create user
resp = requests.post(f'{GRAFANA_URL}/api/admin/users', json={
    'name': f'Agent {session_id}', 'login': user_login,
    'password': user_password, 'email': f'{user_login}@cua-gym.local',
}, auth=ADMIN_AUTH, timeout=15)
assert resp.status_code == 200
user_id = resp.json()['id']

# Add user to org + switch
requests.post(f'{GRAFANA_URL}/api/orgs/{org_id}/users',
              json={'loginOrEmail': user_login, 'role': 'Editor'},
              auth=ADMIN_AUTH, timeout=15)
requests.post(f'{GRAFANA_URL}/api/users/{user_id}/using/{org_id}',
              auth=ADMIN_AUTH, timeout=15)

# Create TestData datasource
resp = requests.post(f'{GRAFANA_URL}/api/datasources', json={
    'name': 'TestData', 'type': 'grafana-testdata-datasource',
    'access': 'proxy', 'isDefault': True,
}, auth=ADMIN_AUTH, headers={'X-Grafana-Org-Id': str(org_id)}, timeout=15)
assert resp.status_code == 200
ds_uid = resp.json()['datasource']['uid']

# --- Build dashboard ---
dashboard = {
    'uid': f'dash-{session_id}',
    'title': 'Production API Monitoring',
    'tags': ['production', 'api'],
    'time': {'from': '2024-01-15T00:00:00Z', 'to': '2024-01-15T08:00:00Z'},
    'refresh': '',
    'panels': [
        # ... build panels using make_panel() / make_csv_target() ...
    ],
    'templating': {'list': []},
}

# Inject dashboard
resp = requests.post(f'{GRAFANA_URL}/api/dashboards/db', json={
    'dashboard': dashboard, 'overwrite': True,
}, auth=ADMIN_AUTH, headers={'X-Grafana-Org-Id': str(org_id)}, timeout=15)
assert resp.status_code == 200
print(f'Dashboard created: {resp.json()["url"]}')

# --- Persist session info ---
session_info = {
    'grafana_url': GRAFANA_URL,
    'org_id': org_id, 'org_name': org_name,
    'user_id': user_id, 'user_login': user_login,
    'user_password': user_password,
    'ds_uid': ds_uid,
    'dashboard_uid': f'dash-{session_id}',
}
with open('/tmp/task_grafana_session', 'w') as f:
    json.dump(session_info, f)

# --- Launch browser ---
def launch_gui(command, delay_sec=1.0):
    env = os.environ.copy()
    env['DISPLAY'] = ':0'
    subprocess.Popen(
        shlex.split(command),
        stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, env=env,
    )
    time.sleep(delay_sec)

# Login URL with org context
login_url = f'{GRAFANA_URL}/login'
launch_gui(f'google-chrome "{login_url}"', delay_sec=2.0)
print(f'GUI_READY: launched browser at {login_url}')
print(f'Login: {user_login} / {user_password}')

---

17. golden_patch.py Template

"""
Golden Patch: <task_description>
Task ID: <task_id>
Domain: grafana
Changes: <brief list of what this patch does>
"""
import copy
import json

import requests

# --- Load session ---
with open('/tmp/task_grafana_session') as f:
    session = json.load(f)

GRAFANA_URL = session['grafana_url']
ADMIN_AUTH = ('admin', 'cua-gym-admin-2024')
org_id = session['org_id']
headers = {'X-Grafana-Org-Id': str(org_id)}

# --- Read current state ---
resp = requests.get(
    f'{GRAFANA_URL}/api/dashboards/uid/{session["dashboard_uid"]}',
    auth=ADMIN_AUTH, headers=headers, timeout=15,
)
dashboard = resp.json()['dashboard']

# --- Apply golden changes ---
# Example: change panel title
# dashboard['panels'][0]['title'] = 'Updated Panel Title'

# Example: add a threshold
# dashboard['panels'][0]['fieldConfig']['defaults']['thresholds'] = {
#     'mode': 'absolute',
#     'steps': [
#         {'color': 'green', 'value': None},
#         {'color': 'red', 'value': 90},
#     ],
# }

# --- Save updated dashboard ---
resp = requests.post(f'{GRAFANA_URL}/api/dashboards/db', json={
    'dashboard': dashboard, 'overwrite': True,
}, auth=ADMIN_AUTH, headers=headers, timeout=15)
assert resp.status_code == 200
print('Golden state applied')

---

18. reward.py Template

"""
Reward: <task_description>
Task ID: <task_id>
Domain: grafana
"""
import json
import sys

import requests

# --- Load session ---
with open('/tmp/task_grafana_session') as f:
    session = json.load(f)

GRAFANA_URL = session['grafana_url']
ADMIN_AUTH = ('admin', 'cua-gym-admin-2024')
org_id = session['org_id']
headers = {'X-Grafana-Org-Id': str(org_id)}

score = 0.0
checks_passed = 0
total_checks = 0

def check(name, condition, weight=1.0):
    global score, checks_passed, total_checks
    total_checks += 1
    if condition:
        checks_passed += 1
        score += weight
        print(f'  PASS: {name}')
    else:
        print(f'  FAIL: {name}')

# --- Read current state ---
resp = requests.get(
    f'{GRAFANA_URL}/api/dashboards/uid/{session["dashboard_uid"]}',
    auth=ADMIN_AUTH, headers=headers, timeout=15,
)
if resp.status_code != 200:
    print(f'Dashboard not found: {resp.status_code}')
    print(f'reward: 0.0')
    sys.exit(0)

dashboard = resp.json()['dashboard']
meta = resp.json()['meta']

# --- Verification checks ---
# Example: verify panel title was changed
# check('Panel title updated',
#        dashboard['panels'][0]['title'] == 'Updated Panel Title')

# Example: verify threshold was added
# thresholds = dashboard['panels'][0].get('fieldConfig', {}).get('defaults', {}).get('thresholds', {})
# steps = thresholds.get('steps', [])
# check('Threshold has 2 steps', len(steps) == 2)
# check('Red threshold at 90', any(s.get('value') == 90 and s.get('color') == 'red' for s in steps))

# --- Final score ---
final_score = score / total_checks if total_checks > 0 else 0.0
print(f'\nChecks: {checks_passed}/{total_checks}')
print(f'reward: {final_score:.2f}')

# --- Cleanup ---
try:
    requests.delete(f'{GRAFANA_URL}/api/orgs/{org_id}', auth=ADMIN_AUTH, timeout=15)
    requests.delete(f'{GRAFANA_URL}/api/admin/users/{session["user_id"]}',
                    auth=ADMIN_AUTH, timeout=15)
    print('Session cleaned up')
except Exception:
    pass

---

19. Task Design Space

Category 1: Dashboard Editing

Difficulty	Task	Setup State	Reward Check
Easy	Rename panel title	Panel with old title	panels[i].title == new_title
Easy	Change time range to "Last 24h"	Dashboard with default time	time.from == 'now-24h'
Easy	Add a tag to the dashboard	Dashboard without tag	tag in dashboard.tags
Easy	Change refresh interval to 30s	Dashboard with no refresh	refresh == '30s'
Medium	Change panel type (timeseries → gauge)	Timeseries panel	panels[i].type == 'gauge'
Medium	Rearrange panel layout	Panels at positions A	panels[i].gridPos == expected
Medium	Add threshold (green<60, red>=80)	Panel without thresholds	Check thresholds.steps
Medium	Configure legend as table with calcs	Default legend	Check options.legend
Hard	Add field override (color per series)	Multi-series panel	Check fieldConfig.overrides
Hard	Add data link to panel	Panel without links	Check fieldConfig.defaults.links
Hard	Add transformation (organize fields)	Panel with raw data	Check transformations array

Category 2: Template Variables

Difficulty	Task	Setup State	Reward Check
Medium	Create custom variable with options	No variables	Check templating.list
Medium	Enable multi-select on variable	Single-select variable	variable.multi == True
Hard	Create interval variable	No variables	Check type + query

Category 3: Alerting

Difficulty	Task	Setup State	Reward Check
Easy	Pause an alert rule	Active alert rule	isPaused == True
Medium	Create threshold alert (CPU > 80%)	Dashboard + folder	Check rule condition
Medium	Change alert threshold 80→90	Rule with threshold=80	Check evaluator.params
Medium	Add alert label (severity: critical)	Rule without labels	Check labels dict
Hard	Create notification policy route	Contact points exist	Check policy tree
Hard	Create mute timing (weekends)	No mute timings	Check time_intervals

Category 4: Contact Points

Difficulty	Task	Setup State	Reward Check
Medium	Create email contact point	None	Check type + addresses
Medium	Create webhook contact point	None	Check type + url
Hard	Configure notification routing	Multiple CPs	Check policy routes

Category 5: Organization & Folders

Difficulty	Task	Setup State	Reward Check
Easy	Create folder	None	Check folder title/uid
Easy	Move dashboard to folder	Dashboard in root	Check meta.folderUid
Medium	Create nested folder structure	None	Check parentUid
Medium	Set folder permissions	Folder exists	Check permissions list
Medium	Create playlist of 3 dashboards	3 dashboards	Check playlist items

Category 6: User & Team Admin

Difficulty	Task	Setup State	Reward Check
Easy	Create new user	None	Check user exists
Medium	Create team and add member	Users exist	Check team members
Medium	Change user role in org	User in org	Check role field
Medium	Set org dark theme	Default preferences	Check theme preference

---

20. Bitter Lessons

1. X-Grafana-Org-Id header is essential. Without it, all operations go to org 1 (Main Org). Always include it when operating on session orgs.

2. X-Disable-Provenance: true for alerting resources. Without this header, resources created via API get a provenance lock and cannot be modified in the UI — defeating the purpose of training tasks.

3. Notification policy PUT replaces the entire tree. Always GET first to read the current root receiver, then build the new tree on top. Otherwise you break the default routing.

4. Folder permissions POST replaces ALL permissions. Always include {'userId': 1, 'permission': 4} (admin) in the items array, or you'll lock yourself out.

5. for field format inconsistency. Creating with "5m" may read back as "5m0s". Reward verification should normalize: for_value.replace('0s', '') == '5m'.

6. Alert rules require a folder first. Create folder → create alert rule. Never try to create an alert rule without a valid folderUID.

7. Annotation tag filtering uses repeated params. Not tags=a,b but tags=a&tags=b. In Python requests: params=[('tags', 'a'), ('tags', 'b')].

8. Org preferences need context switch. Use /api/org/preferences (current org), NOT /api/orgs/:id/preferences (returns 404). Admin must switch to target org first via POST /api/user/using/{org_id}.

9. Dashboard save requires overwrite: true for updates. Without it, updating an existing dashboard UID returns 412 Precondition Failed.

10. TestData datasource type is grafana-testdata-datasource. Not testdata. When verifying, match on substring testdata.

11. Cleanup order matters. Delete org first, then user. Deleting user first leaves orphaned org memberships.

12. csv_content is the key to verifiable data. For any task that involves reading or interpreting data (find anomaly, compare trends), always use csv_content so reward scripts can verify the data was actually present and correct.

13. Session info goes to /tmp/task_grafana_session. This file links initial_setup.py, golden_patch.py, and reward.py to the same Grafana session (org_id, user credentials, dashboard UIDs, etc.).