Skip to main content

OpenTelemetry MCP Server Examples & Scenarios

These real-world operational scenarios demonstrate how AI assistants (Claude, Cursor, Cline, or TalkOps autonomous controllers) use the OpenTelemetry MCP Server to solve complex observability tasks on Kubernetes.

Scenario 1: Intent-Driven Namespace Telemetry Provisioning​

User Prompt:

"I need to collect traces and metrics in the payments namespace and export them to Tempo and Mimir in monitoring. Set up an OpenTelemetry collector with best practices."

What Happens:

  1. The AI calls otel_provision_collector with namespace="payments", signals=["traces", "metrics"], and enable_spanmetrics=true.
  2. The MCP server queries the Kubernetes cluster, auto-discovers active backend services (tempo.monitoring.svc:4317 and mimir.monitoring.svc:9009), and generates a production-grade collector configuration.
  3. The server ensures strict processor sequencing (memory_limiter → k8sattributes → resourcedetection → batch).
  4. The server creates the required ClusterRole and ClusterRoleBinding for k8sattributes pod metadata enrichment.
  5. The AI presents the complete manifest diff to the engineer before applying the changes.

Result: A fully functional OpenTelemetryCollector is deployed to the payments namespace, with resource limits properly sized and telemetry streaming securely to your backends.

Scenario 2: Zero-Touch Polyglot Auto-Instrumentation​

User Prompt:

"Onboard the order-service Python FastAPI app in production to OpenTelemetry with 15% trace sampling."

What Happens:

  1. The AI calls otel_lookup_instrumentation(language="python", framework="fastapi") to verify SDK compatibility and obtain the standard injection annotation (instrumentation.opentelemetry.io/inject-python).
  2. It calls otel_patch_instrumentation to ensure an Instrumentation CRD exists in production configured with sampler_type="parentbased_traceidratio" and sampler_argument="0.15".
  3. The AI executes otel_annotate_deployment(deployment_name="order-service", namespace="production", language="python").
  4. The tool scans existing container environment variables to verify that no hardcoded OTEL_EXPORTER_OTLP_ENDPOINT conflicts exist that would override the Operator's configuration.
  5. The deployment pod template is annotated, triggering a smooth rolling restart with the OTel Python init container injected.

Result: All inbound HTTP requests and outbound database calls from order-service are automatically traced and exported without touching application source code.

Scenario 3: Investigating High-Cardinality SpanMetrics​

User Prompt:

"Prometheus memory usage is spiking after enabling SpanMetrics on otel-gateway. Audit the dimensions and fix any high-cardinality attributes."

What Happens:

  1. The AI calls otel_detect_cardinality(collector_name="otel-gateway", namespace="monitoring").
  2. The audit reveals that http.url (containing raw UUID path parameters) and user.id are active dimensions, generating over 45,000 distinct time series.
  3. The AI calls otel_gen_drop_attribute_rules(signal="metrics", attributes_to_drop=["http.url", "user.id"]) to generate an OTTL remediation snippet:
processors:
  transform/strip_high_cardinality:
    metric_statements:
      - context: datapoint
        statements:
          - delete_key(attributes, "http.url")
          - delete_key(attributes, "user.id")
  1. The AI presents the proposed collector patch to the operator and applies it via otel_patch_collector(dry_run=false).

Result: Active time-series counts drop by 92%, stabilizing Prometheus memory without losing latency distribution insights.

Scenario 4: Aligning Multi-Tier Sampling Policies​

User Prompt:

"We're missing critical error traces in Tempo. Check our sampling configuration to see why 500 errors are getting dropped."

What Happens:

  1. The AI calls otel_inspect_sampling_configuration(collector_name="otel-traces", namespace="monitoring").
  2. The tool flags a major architectural conflict: the SDK Instrumentation CRD has a 5% probabilistic head sampler (traceidratio: 0.05), while the collector has a 100% error tail-sampling policy. Because the head sampler drops 95% of all spans before they ever reach the collector, the tail sampler never gets a chance to evaluate the failed requests.
  3. The AI calls otel_toggle_sampling_strategy to apply a hybrid strategy: switching SDK head sampling to always_on and delegating sampling decisions entirely to the collector's tail-sampling processor.

Result: 100% of error traces and slow requests (> 1.5s) are preserved in Tempo, while healthy background requests are sampled at a cost-effective 10% rate.