Why OpenTelemetry Is the Foundation for AI-Powered Observability

The next wave of observability tooling is AI-powered: anomaly detection, root cause analysis, predictive alerting, and eventually autonomous remediation. All of these capabilities share one requirement: structured, semantically consistent telemetry data.

OpenTelemetry isn’t just about vendor flexibility—it’s the foundation for AI observability.

The AI Observability Landscape in 2026

Every major observability vendor is shipping AI features:

Vendor AI Capabilities
Datadog Watchdog anomaly detection, root cause analysis
New Relic AIOps, predictive alerting
Grafana ML-based anomaly detection, smart thresholds
Dynatrace Davis AI, autonomous problem detection

But here’s what they don’t advertise: AI features work best with standardized data.

Why AI Needs OTel

1. Consistent Schema for Model Training

ML models learn patterns from data. Inconsistent data = poor models:

# Training data sample 1 (Service A)
{"http.method": "GET", "duration_ms": 45}

# Training data sample 2 (Service B)  
{"request_method": "GET", "latency": 0.045}

# Training data sample 3 (Service C)
{"HTTP_METHOD": "get", "response_time_seconds": 0.045}

# Result: Model confused, poor anomaly detection

With OTel semantic conventions:

# All services
{"http.request.method": "GET", "http.response.latency": 45}

# Result: Clean training data, accurate models

2. Cross-Service Correlation

Root cause analysis requires tracing requests across services. Without trace context propagation (which OTel standardizes), AI can’t correlate:

Anomaly detected: Checkout service slow
├── Related: Payment service errors? (Can't tell without trace context)
├── Related: Inventory service timeout? (Can't tell without trace context)  
└── Root cause: Unknown

With OTel traces:
Anomaly detected: Checkout service slow
├── Trace shows: Checkout → Payment → Inventory
├── Inventory service: 10s timeout (root cause)
└── Recommendation: Scale inventory service

3. Semantic Understanding

AI needs to understand what data means, not just that it exists:

# OTel semantic conventions give AI context
http.response.status_code: 500
# AI knows: This is an error (server-side)

http.response.status_code: 429
# AI knows: This is rate limiting, different remediation

http.response.status_code: 401  
# AI knows: This is auth failure, security implications

Building AI-Ready Telemetry

The Instrumentation Checklist

from opentelemetry import trace
from opentelemetry.semconv.trace import SpanAttributes

tracer = trace.get_tracer(__name__)

def process_order(order):
    with tracer.start_as_current_span("process_order") as span:
        # Business context for AI correlation
        span.set_attribute("order.id", order.id)
        span.set_attribute("order.value_usd", order.total)
        span.set_attribute("customer.tier", order.customer.tier)
        
        # Semantic conventions for AI understanding
        span.set_attribute(SpanAttributes.CODE_FUNCTION, "process_order")
        span.set_attribute(SpanAttributes.CODE_NAMESPACE, "orders.processing")
        
        # Outcome for AI learning
        try:
            result = execute_order(order)
            span.set_attribute("order.status", "success")
        except Exception as e:
            span.set_attribute("order.status", "failed")
            span.set_attribute("error.type", type(e).__name__)
            span.record_exception(e)
            raise

The AI-Ready Metrics Pipeline

# OTel Collector config for AI backends
processors:
  # Ensure consistent attribute naming
  transform:
    metric_statements:
      - context: datapoint
        statements:
          - set(attributes["service.name"], resource.attributes["service.name"])
  
  # Add derived attributes for ML
  metricstransform:
    transforms:
      - include: http.server.duration
        action: insert
        new_name: http.server.duration.anomaly_score
        operations:
          - action: experimental_scale_value
            experimental_scale: 0.001  # Normalize for ML

The Autonomous SRE Vision

The end goal isn’t just AI-assisted observability—it’s autonomous operations:

┌─────────────────────────────────────────────────────────────┐
│                    Autonomous SRE Loop                       │
│                                                              │
│  OTel Data → AI Detection → Root Cause → Remediation       │
│      ▲                                          │           │
│      │                                          │           │
│      └──────────── Feedback Loop ───────────────┘           │
└─────────────────────────────────────────────────────────────┘

This future requires:

  1. Structured data (OTel provides)
  2. Semantic meaning (OTel conventions provide)
  3. Action context (OTel attributes provide)

The Bottom Line

If you’re evaluating OTel purely for vendor flexibility, you’re underselling it. OTel is the data layer that enables AI observability. Organizations without OTel will struggle to adopt AI tooling—or will pay dearly to retrofit their telemetry.

The question isn’t whether AI observability is coming. It’s whether your data is ready.

MLOps and Model Serving: Where OTel Gets Interesting

Rachel, great framing on AI observability. Let me add the perspective from the other side—using OTel to observe AI/ML workloads themselves.

The ML Telemetry Challenge

Model serving has unique observability requirements that traditional APM doesn’t cover:

# Standard APM gives you:
- Request latency
- Error rate
- Throughput

# ML serving also needs:
- Model inference time (separate from preprocessing)
- Batch size effects
- GPU utilization
- Model version tracking
- Input feature distributions
- Prediction confidence scores

OTel for Model Serving

We instrument our LLM inference pipeline with OTel:

from opentelemetry import trace, metrics

tracer = trace.get_tracer("ml.inference")
meter = metrics.get_meter("ml.inference")

# Custom ML metrics
inference_duration = meter.create_histogram(
    "ml.inference.duration",
    unit="ms",
    description="Time to generate model output"
)

token_throughput = meter.create_counter(
    "ml.tokens.generated",
    description="Number of tokens generated"
)

def generate_completion(prompt, model_id):
    with tracer.start_as_current_span("llm.completion") as span:
        span.set_attribute("ml.model.id", model_id)
        span.set_attribute("ml.model.version", get_model_version(model_id))
        span.set_attribute("ml.input.tokens", count_tokens(prompt))
        
        start = time.time()
        output = model.generate(prompt)
        duration = (time.time() - start) * 1000
        
        span.set_attribute("ml.output.tokens", count_tokens(output))
        span.set_attribute("ml.inference.duration_ms", duration)
        
        inference_duration.record(duration, {"model": model_id})
        token_throughput.add(count_tokens(output), {"model": model_id})
        
        return output

The GPU Utilization Problem

GPU metrics don’t fit cleanly into OTel’s semantic conventions yet:

Metric OTel Coverage Workaround
GPU memory used Not standard Custom gpu.memory.used
GPU utilization % Not standard Custom gpu.utilization
CUDA errors Not standard Custom gpu.cuda.errors
Tensor core usage Not standard Custom gpu.tensor_core.utilization

We’re waiting for OTel GPU semantic conventions to stabilize.

Model Version Tracking

One killer feature: OTel traces let you correlate performance regressions with model deployments:

-- Find latency regression after model update
SELECT 
  span.attributes['ml.model.version'] as version,
  AVG(span.duration) as avg_latency,
  PERCENTILE_CONT(0.99) WITHIN GROUP (ORDER BY span.duration) as p99
FROM traces
WHERE span.name = 'llm.completion'
GROUP BY version
ORDER BY span.start_time DESC;

The AI Observability Loop

Rachel mentioned AI consuming OTel data. We also use AI to analyze OTel data about AI workloads. Meta, but effective:

OTel (ML workload) → AI Analysis → Optimization Recommendations
     ↑                                        ↓
     └────── Apply recommendations ───────────┘

Examples:

  • “Model X latency increased 40% after version 2.3—batch size decreased”
  • “GPU memory pressure on node 5 causing OOM—redistribute inference load”
  • “Token throughput dropped during peak hours—scale GPU pool”

Rachel, this framing of OpenTelemetry as AI infrastructure is exactly the pitch that got our board to approve a 40% increase in observability budget for 2026.

The Strategic Investment Thesis

When I present observability spending to the board, I no longer position it as an operational cost center. It’s now framed as AI infrastructure investment:

Traditional Framing AI Infrastructure Framing
“Monitoring costs” “Training data pipeline for intelligent operations”
“Troubleshooting tools” “Foundation for autonomous incident response”
“Compliance logging” “Audit trail for AI decision-making”

Why Standardization Enables AI

The board’s concern wasn’t whether to invest in AI—it was whether we’d be locked into a single vendor’s AI capabilities. OpenTelemetry answers this:

  1. Model Portability: Our telemetry data can train any AI system, not just one vendor’s black-box algorithms
  2. Competitive Leverage: When vendors know their AI is replaceable, pricing conversations change dramatically
  3. Build vs Buy Flexibility: We can use vendor AI for commodity tasks while building proprietary models for competitive advantage

The 18-Month Roadmap

Our phased approach:

  • Q1-Q2 2026: Complete OTel migration, ensure semantic consistency
  • Q3 2026: Deploy vendor AI for anomaly detection, baseline performance
  • Q4 2026: Begin training internal models on OTel data for domain-specific insights
  • 2027: Evaluate autonomous remediation for low-risk scenarios

Budget Allocation

I’m now allocating observability budget across three categories:

Data Collection & Storage: 40% (OTel infrastructure)
Vendor AI Capabilities:    35% (anomaly detection, AIOps features)
Internal AI Development:   25% (data science team, custom models)

This split ensures we’re not over-dependent on any single approach while building internal capabilities.

The Competitive Moat Question

The executives who understand AI ask: “How do we turn observability into competitive advantage?”

The answer is proprietary context. Every company has unique:

  • Business processes that generate telemetry
  • Failure modes that require specialized detection
  • Customer impact patterns that need custom correlation

OpenTelemetry lets us capture this context in a standard format, then apply AI that understands our specific business—not just generic infrastructure patterns.

Rachel, I’d love to discuss how Anthropic is thinking about this. Are you seeing AI capabilities as a vendor differentiator or a commodity that will be standardized?

Rachel, the AI + OpenTelemetry intersection is where security observability gets genuinely exciting. We’re moving beyond signature-based detection into behavioral anomaly detection that actually works.

AI Security Monitoring Use Cases

Here’s what we’re building on OTel foundations:

1. Anomalous Access Pattern Detection

OTel traces capture the full request path with semantic attributes. Our AI models learn normal patterns:

# Normal pattern learned by AI
user_type: internal_service
endpoint: /api/v2/user-data
time_window: business_hours
request_rate: 10-50/minute
response_size: 1-10KB

# Anomaly detected
user_type: internal_service
endpoint: /api/v2/user-data
time_window: 03:00_AM
request_rate: 500/minute  # 10x normal
response_size: 50MB       # Data exfiltration?

The AI flags this without us writing explicit rules. It learned what “normal” looks like from OTel data.

2. Credential Stuffing Detection

Combining OTel metrics with AI pattern recognition:

# Features extracted from OTel data for ML model
features = {
    'auth_failure_rate': meter.get('auth.failures') / meter.get('auth.attempts'),
    'unique_usernames_per_ip': tracer.get_unique('user.id', group_by='client.ip'),
    'request_timing_variance': tracer.get_stddev('http.duration'),
    'geographic_impossibility': tracer.detect_velocity('user.location'),
}

# AI model trained on OTel data
if credential_stuffing_model.predict(features) > 0.85:
    trigger_adaptive_response()

3. Supply Chain Attack Detection

This is where semantic conventions shine. OTel captures:

  • code.function and code.filepath attributes
  • Dependency call patterns
  • Unusual outbound connections from specific code paths

AI models can detect when a trusted library suddenly makes unexpected network calls—exactly what we’d see in a supply chain compromise.

4. AI Model Abuse Detection

For companies running AI services (Rachel, this is relevant for Anthropic):

# OTel attributes for AI inference monitoring
ai.model.name: claude-3-opus
ai.request.tokens: 150000  # Unusually high
ai.request.pattern: extraction_attempt
ai.user.tier: free
ai.cost.projected: $45.00  # Way over tier limit

AI-on-AI: using ML models to detect abuse of ML services, all built on standardized telemetry.

The SIEM Integration Story

Traditional SIEMs expect proprietary log formats. With OTel:

  1. Consistent schema across all services means AI models transfer between environments
  2. Reduced feature engineering because semantic conventions define the important fields
  3. Cross-service correlation lets AI see attack chains, not isolated events

Why This Wasn’t Possible Before

Proprietary agents meant:

  • Different schemas per vendor
  • AI models trained on one vendor’s data didn’t transfer
  • Security teams maintained multiple detection systems

OTel unification means one AI model can understand telemetry from any service.

The Privacy Balance

One caution: AI on telemetry data needs careful privacy controls. We’re using:

  • Attribute filtering at the collector to strip PII before AI processing
  • Differential privacy techniques for aggregate analysis
  • Audit logs of AI model access to raw telemetry

Rachel, how is Anthropic handling the privacy implications of AI systems analyzing their own telemetry?