OpenTelemetry MCP Server

OpenTelemetry is the undisputed standard for cloud-native telemetry, but running it in production on Kubernetes is notoriously tricky. Getting multi-language auto-instrumentation right across Java, Python, Node.js, and Go, sequencing collector processor pipelines without introducing silent data loss, sizing buffer memory, and stopping runaway SpanMetrics cardinality before Prometheus melts requires seasoned platform engineering skills. When generic AI assistants try to help, they routinely hallucinate CRD fields, place batch processors in the wrong position, or suggest unvalidated configs that crash collector pods.

The TalkOps OpenTelemetry MCP Server replaces that guesswork with structured, Kubernetes-native automation. It connects your AI agents (Claude, Cline, Cursor, or TalkOps autonomous controllers) directly to the OpenTelemetry Operator API with strict validation, dry-run safety gates, and battle-tested defaults:

1. Intent-Driven Collector Provisioning. Tell the agent "Deploy a telemetry pipeline for traces and metrics in the payments namespace", and it discovers active backend endpoints (Tempo, Mimir, Jaeger, Prometheus), builds the collector config with validated processor ordering, sets up required RBAC for k8sattributes, and sizes memory limits to match your cluster topology.
2. Zero-Touch Application Onboarding. The AI identifies workload runtimes using a 4-tier inspection strategy, generates or patches Instrumentation CRDs, and applies non-conflicting injection annotations to Deployments.
3. Pipeline Architecture & Validation. Deep-inspect any collector's runtime config, verify strict processor ordering (memory_limiter → k8sattributes → resourcedetection → transform → batch), and audit filelog receivers to eliminate self-collection feedback loops.
4. SpanMetrics & Cardinality Governance. Detect runaway label combinations from SpanMetrics connectors, generate OpenTelemetry Transformation Language (OTTL) statements to drop high-cardinality attributes, and calculate time-series explosion risk before applying changes.
5. Sampling Strategy Alignment. Cross-examine SDK head sampling against collector tail sampling to eliminate dropped error traces and prevent conflicting sampling policies.
6. eBPF Security Auditing. Audit cluster-wide eBPF telemetry agents (OTel eBPF, Grafana Beyla) for CAP_SYS_ADMIN, privileged containers, and hostPID exposures.

---

Key Features

Intent-Driven Collector Provisioning

• Declare desired signals (traces, metrics, logs) and target namespaces — the server auto-discovers endpoints and wires the pipeline.
• Built-in discovery for 10 backend patterns: Jaeger, Tempo, Zipkin, Prometheus, Thanos, Mimir, VictoriaMetrics, OpenSearch, Elasticsearch, Loki.
• Automatic RBAC provisioning (ClusterRole + ClusterRoleBinding) enabling k8sattributes processor enrichment.
• Workload-aware deployment modes: DaemonSet for host log collection, StatefulSet for Target Allocator scraping, and Deployment for OTLP gateways.

Polyglot Workload Auto-Instrumentation

• Support matrix lookup for Java, Python, Node.js, .NET, Go, and Rust runtimes.
• 4-tier language detection: Pod annotations → Container image patterns → Container names → Runtime environment variables (JAVA_HOME, PYTHONPATH).
• Conflict detection preventing hardcoded OTEL_EXPORTER_OTLP_ENDPOINT env vars from breaking Operator-injected agents.
• Automated Instrumentation CRD generation with custom propagators (tracecontext, baggage, b3) and sampler ratios.

Pipeline Safety & Validation

• Best-practice processor chain verification (memory_limiter first to prevent OOMs, batch last to optimize network transport).
• Filelog receiver safety audits checking for checkpoint storage (file_storage extension) and self-log exclusion patterns.
• Prometheus Target Allocator state and strategy verification (consistent-hashing vs. per-node).
• Sizing recommendations scaling memory buffers and CPU limits based on namespace workload density.

Cardinality & Sampling Governance

• SpanMetrics dimension analysis and histogram bucket multiplier auditing.
• Automated OTTL snippet generation (delete_key(attributes, "http.url")) for transform processors.
• Multi-layer sampling coordination: head sampling in the SDK and tail sampling policies (error retention, latency thresholds) in the collector.

eBPF Security Auditing

• Cluster-wide discovery of eBPF kernel agents.
• Security context auditing: privileged mode, host PID sharing, and capability sets (CAP_BPF vs CAP_SYS_ADMIN).
• Prioritized least-privilege hardening recommendations for compliance audits.

---

Architecture

The server translates high-level MCP tool requests into Kubernetes API operations and OpenTelemetry Operator CRD mutations through a structured, multi-layer service architecture.

How it works:

1. Your AI assistant connects over HTTP, SSE, or stdio and initializes the MCP session.
2. The agent reads otel://system/health to confirm Kubernetes cluster reachability and OTel Operator CRD availability.
3. Every mutating tool call (otel_provision_collector, otel_patch_instrumentation, otel_annotate_deployment) defaults to dry_run=True so the human operator can review the generated manifest diff before applying.
4. The service layer parses live configs, audits processor order, runs cardinality checks, and applies changes directly through the Kubernetes API.

---

Tech Stack

Category	Technologies
Language	Python 3.12+
MCP Framework	FastMCP ≥ 2.13.3
Protocol	Model Context Protocol (MCP)
Kubernetes	Python K8s Client · OpenTelemetry Operator CRDs (v1beta1, v1alpha1)
Telemetry Standard	OpenTelemetry Specification 1.30+ · OTLP/gRPC · OTLP/HTTP
Supported Backends	Tempo, Jaeger, Prometheus, Mimir, Thanos, VictoriaMetrics, Loki, OpenSearch
Transport Modes	stdio · http · sse · streamable-http
Packaging	Docker · uv

---

Available Tools Summary

Tool	Category	Description
otel_provision_collector	Collector Management	Intent-driven provisioning with backend auto-discovery, RBAC generation, and resource sizing.
otel_patch_collector	Collector Management	Expert-level CRD creation and patch with full YAML validation and dry-run safety.
otel_list_collectors	Discovery	List OpenTelemetryCollector CRDs across namespaces with label selectors and pagination.
otel_get_collector	Discovery	Inspect full collector configuration, pipeline topology, and raw YAML specs.
otel_query_a2ui	Discovery	Retrieve real-time collector pipeline health and metrics for A2UI status tables.
otel_list_instrumented_services	Discovery	Workload auto-instrumentation status, annotation verification, and 4-tier language detection.
otel_lookup_instrumentation	Instrumentation	Language and framework support matrix lookup for auto-instrumentation capabilities.
otel_patch_instrumentation	Instrumentation	Create or patch Instrumentation CRDs with propagators, samplers, and custom images.
otel_annotate_deployment	Instrumentation	Annotate Deployments for auto-instrumentation with conflicting env var detection.
otel_validate_k8sattributes_order	Validation	Validate processor sequence against OpenTelemetry best practices.
otel_check_filelog_safety	Validation	Check filelog receivers for feedback loops, checkpoint storage, and resource detection.
otel_inspect_target_allocator_state	Validation	Inspect Target Allocator allocation strategies, selectors, and Prometheus CR integration.
otel_recommend_collector_topology	Validation	Generate collector architecture recommendations based on cluster scale.
otel_detect_cardinality	Governance	Detect high-cardinality SpanMetrics dimensions and estimate time-series explosion risk.
otel_gen_drop_attribute_rules	Governance	Generate transform processor YAML rules to drop noisy telemetry attributes.
otel_analyze_ebpf_footprint	Governance	Audit eBPF agent security contexts, privileged containers, and capabilities.
otel_inspect_sampling_configuration	Sampling	Cross-reference head vs. tail sampling configurations and detect conflicts.
otel_toggle_sampling_strategy	Sampling	Generate config patches to switch or adjust head and tail sampling policies.
otel_inspect_spanmetrics_config	SpanMetrics	Inspect SpanMetrics connector configuration and pipeline wiring.
otel_enable_spanmetrics_for_service	SpanMetrics	Generate SpanMetrics enablement YAML with custom dimensions and histogram buckets.

---

Quick Start

Running with Docker

Run the MCP server locally with access to your active kubeconfig:

docker run --rm -it \
  -p 8771:8771 \
  -e MCP_TRANSPORT=http \
  -e K8S_IN_CLUSTER=false \
  -v ~/.kube/config:/root/.kube/config:ro \
  talkopsai/opentelemetry-mcp-server:latest

Connect to Claude Desktop, Cursor, or Cline

Add the server to your MCP client configuration (claude_desktop_config.json or .cursor/mcp.json):

{
  "mcpServers": {
    "opentelemetry": {
      "url": "http://localhost:8771/mcp",
      "description": "TalkOps OpenTelemetry MCP Server for Kubernetes Observability"
    }
  }
}

Or connect directly using uvx over stdio:

{
  "mcpServers": {
    "opentelemetry": {
      "command": "uvx",
      "args": ["opentelemetry-mcp-server"],
      "env": {
        "K8S_ENABLED": "true",
        "MCP_LOG_LEVEL": "INFO"
      }
    }
  }
}

---

Security Considerations

• Least-Privilege RBAC: The MCP server requires RBAC permissions to inspect workloads and manage OTel CRDs. Review Kubernetes RBAC Requirements before granting cluster-wide roles.
• Dry-Run by Default: All mutating operations (otel_provision_collector, otel_patch_instrumentation, otel_annotate_deployment) default to dry_run=True. We recommend keeping dry-run enabled during automated agent evaluations.
• Sensitive Attribute Scrubbing: Use otel_gen_drop_attribute_rules to generate transform processor rules that redact PII, Authorization headers, and internal tokens before telemetry leaves your network perimeter.

---

Project Layout

opentelemetry-mcp-server/
├── opentelemetry_mcp_server/
│   ├── tools/           # 19 MCP Tools across 7 functional groups
│   ├── resources/       # 9 MCP Resources (otel:// URIs)
│   ├── prompts/         # 5 Guided workflow prompts
│   ├── services/        # K8s client and collector config parser
│   ├── server/          # FastMCP server initialization & middleware
│   ├── utils/           # Language registry and topology analyzer
│   ├── models/          # Pydantic data schemas
│   ├── config.py        # Environment variables and CRD version config
│   └── main.py          # Server entry point
├── tests/               # Unit and integration test suites
├── docs/                # Workflow guides and test fixtures
├── Dockerfile
├── pyproject.toml
└── README.md

---

Next Steps

• Configuration — Environment variables, CRD versions, and Kubernetes RBAC manifests.
• Tools Reference — Complete parameter specifications and sample invocations for all 19 tools.
• Resources Reference — 9 read-only otel:// URIs for live cluster and pipeline introspection.
• Workflows — Step-by-step guides for onboarding, pipeline validation, cardinality audits, and sampling optimization.
• Examples & Scenarios — Real-world conversational prompts and step-by-step agent executions.