AI Agent Observability: Logging, Tracing and Debugging

AI Agent Observability: Logging, Tracing and Debugging

Modern AI agents are complex systems running hundreds of decisions per second across multiple API calls, LLM inferences, and tool executions. Unlike traditional software, they operate in probabilistic environments where the same input can yield different outputs. When your agent starts returning unexpected responses or failing silently, how do you diagnose the issue?

Observability-the ability to understand a system's internal state from its external outputs-isn't optional for production AI agents. Here's how to implement logging, tracing, and debugging that actually works for autonomous systems.

1. Structured Logging: Beyond print()

Most AI agents start with print(response) debugging, but this fails at scale. Structured logging captures:
• Input/Output pairs (with timestamps and session IDs)
• Tool executions (success/failure, latency, retries)
• LLM raw prompts/completions (for drift detection)

# Using LangSmith for LLM logging (works with any framework)
from langsmith import Client
client = Client() run_id = client.create_run(     project_name="CustomerSupportAgent",     inputs={"user_query": "Reset my password"},     run_type="chain" )
Log the LLM call
client.create_llm_run(
    run_id=run_id,
    provider="openai",
    prompt="You are a helpful assistant...",
    completion="Here's how to reset...",
    metadata={"model": "gpt-4-turbo"}
)

Key features to log:
• Session affinity (trace all steps for a single user request)
• Token counts and costs per invocation
• Confidence scores if your agent generates them

2. Distributed Tracing for Multi-Tool Workflows

When your agent chains multiple tools (web search → DB query → LLM), you need traces that connect these steps. OpenTelemetry is the modern standard:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
provider = TracerProvider() trace.set_tracer_provider(provider)
tracer = trace.get_tracer("ai.agent")
with tracer.start_as_current_span("handle_request") as span:     span.set_attribute("user_id", "abc123")     # Tool executions automatically create child spans     search_results = web_search_tool(query)     span.add_event("search_completed", {"result_count": len(search_results)}) 

Trace visualization tools:
• LangSmith (best for LLM-centric agents)
• Datadog/Honeycomb (for infra-heavy deployments)
• Custom solutions using OpenTelemetry Collector

Example of what you should see:

Trace: CustomerSupportSession (1.2s)   ├─ LLM: GenerateSearchQuery (220ms)   ├─ Tool: WebSearch (480ms)   ├─ LLM: SummarizeResults (310ms)   └─ DB: StoreConversation (190ms)   

3. Debugging Non-Deterministic Failures

AI agents fail differently than traditional software. Common issues and how to debug them:

Problem: Agent gets stuck in loops Solution: Implement cycle detection in your orchestration layer:

MAX_STEPS = 10
execution_stack = []
def execute_agent_step(action):     execution_stack.append(action)          if len(execution_stack) > MAX_STEPS:         raise RecursionError(             f"Agent exceeded {MAX_STEPS} steps. "             f"Last actions: {execution_stack[-3:]}"         )          # ... execute action ... 

Problem: Tool validation failures Debugging pattern:
• Log the raw tool input (often malformed JSON/parameters)
• Compare against the tool's OpenAPI schema
• Replay the exact call with manual fixes

Problem: LLM responds off-task Use semantic similarity checks against expected outputs:

from sentence_transformers import SentenceTransformer
encoder = SentenceTransformer('all-MiniLM-L6-v2')
expected_response = "How to reset your password..."  actual_response = "Our company was founded in 2020..."
Compare embeddings
if encoder.similarity(expected_response, actual_response) < 0.6:
    alert(f"Possible off-task response: {actual_response}")

4. Cost-Aware Observability

Logging every LLM call is expensive. Implement sampling:

import random
def should_sample(session_id):     # Always log errors     if session_id.error:         return True     # Sample 10% of successful requests     return random.random() < 0.1 

Optimization tips:
• Compress prompts/completions before storage
• Log embeddings instead of full text for semantic searches
• Use clickhouse/postgresql over MongoDB for analytical queries

Key Takeaways

Observability isn't just about fixing bugs-it's how you turn a prototype into a reliable system. Start small (structured logs), add traces as you scale, and always instrument costs.

--- Published on [agentic.dev](https://devaa-io.github.io/agentic-dev/)