jaeger-trace-tests

Jaeger ingests traces over OTLP and exposes a query API for verification. Per the Jaeger getting-started docs, the all-in-one image "combines collector and query components in a single process and uses a transient in-memory storage for trace data" - perfect for CI.

When to use

• E2E or integration test exercises multiple services and you need to verify the full distributed trace shape (not just per-process spans).
• Production observability stack uses Jaeger; tests should reflect the same query API your alerts/SLOs depend on.
• Smoke test after instrumentation changes - confirm spans actually reach Jaeger (not just the SDK exporter).

How to use

1. Start Jaeger all-in-one in CI as a Docker service (Step 1) - it exposes OTLP ingest on :4317/:4318 and the query API on :16686.
2. Point the app's OpenTelemetry SDK at the collector's OTLP endpoint (Step 2).
3. Exercise the flow, force_flush() the span processor, and let the ingest pipeline settle before querying (Worked example).
4. Query GET /api/traces?service=X&operation=Y and assert on the returned span set and tags (Worked example); parent-child + duration assertions and the full query API live in references/query-api-and-ci-wiring.md.
5. Scope each test to a unique service.name so shared-CI trace data does not cross-contaminate (see references).

Step 1 - Run Jaeger all-in-one in CI

Per the Jaeger getting-started docs:

docker run --rm --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 5778:5778 \
  -p 9411:9411 \
  cr.jaegertracing.io/jaegertracing/jaeger:2.17.0

Port	Purpose
16686	Jaeger UI + query HTTP API
4317	OTLP/gRPC ingest
4318	OTLP/HTTP ingest

The full port map (sampling :5778, Zipkin :9411) and the GitHub Actions service block are in references/query-api-and-ci-wiring.md.

Step 2 - Configure SDK to ship to Jaeger

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
provider.add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter(endpoint="http://localhost:4317", insecure=True))
)
trace.set_tracer_provider(provider)

BatchSpanProcessor defers shipping, so the Worked example flushes manually before querying.

Worked example

Exercise the flow, force a flush, then query Jaeger and assert the span set plus a tag value end to end:

def test_order_trace_visible_in_jaeger():
    with use_tracer():
        create_order(items=[item])

    # Flush all spans to Jaeger before query
    trace.get_tracer_provider().force_flush(timeout_millis=5000)

    # Allow Jaeger ingest pipeline a moment
    time.sleep(0.5)

    resp = requests.get(
        "http://localhost:16686/api/traces",
        params={"service": "orders", "operation": "order.create", "lookback": "1m", "limit": 1},
    )
    traces = resp.json()["data"]
    assert len(traces) == 1
    span = next(s for s in traces[0]["spans"] if s["operationName"] == "order.create")
    tag = next(t for t in span["tags"] if t["key"] == "order.item_count")
    assert tag["value"] == 1

The force_flush + brief sleep is mandatory: BatchSpanProcessor batches exports, so an immediate query races the ingest pipeline and misses the span. For parent-child links and duration ceilings, see references/query-api-and-ci-wiring.md.

Other backends - Zipkin and Tempo

The same workflow (backend in Docker, ship OTLP, flush, query, assert) applies to the other two mainstream OSS backends; only the query surface changes:

• Zipkin (Spring Cloud Sleuth heritage; B3 propagation) - REST API on :9411, string-typed tags, dependency-graph assertions, B3 single/multi-header tests: references/zipkin.md.
• Grafana Tempo - TraceQL span selectors with structural operators (>> descendant, > child) that Jaeger's flat span list cannot express, /api/search + /api/traces/{id}: references/tempo.md.

Anti-patterns

Anti-pattern	Why it fails	Fix
Query Jaeger immediately after exercise	Spans may not have shipped yet	force_flush() + brief sleep (Worked example)
Use prod Jaeger from CI	Test traces pollute prod data	Always Docker all-in-one (Step 1)
Hard-code service name across tests	Cross-test contamination on shared CI	Unique service.name per test (references)
Assume long retention	All-in-one is in-memory; old traces evicted	Restart container or shorten test runs
Skip flushing pipeline	BatchSpanProcessor defers ship; queries miss spans	Always flush before query (Worked example)

Limitations

• Jaeger v2 changed deployment + binary names from v1; verify current image tag at the Jaeger getting-started docs.
• Storage backends (Cassandra, Elasticsearch, OpenSearch, Badger) matter for production but Docker all-in-one is sufficient for CI.
• Jaeger UI is for humans; only query HTTP API in tests (no scraping HTML).

References

• Jaeger getting-started docs - Docker run, ports, OTLP ingest
• references/query-api-and-ci-wiring.md - full query API endpoints + trace JSON shape, parent-child + duration assertions, GitHub Actions service, per-test isolation, retention
• references/zipkin.md - Zipkin REST API, B3 propagation, dependency graph
• references/tempo.md - Tempo TraceQL + HTTP API
• opentelemetry-trace-assertions - in-process unit pattern