Tracing Agent Loops with OpenTelemetry gen_ai

Why Agent Loops Need Specialized Tracing

A standard HTTP request is straightforward: one request arrives, gets processed, and a response goes out. The start and end are clear, and latency is a single number. Agent loops are different. A single loop execution can involve dozens of model calls, multiple tool invocations, and sub-agent delegations. Which iteration was slow? Which tool call threw an error? How many iterations did the sub-agent take? Without distributed tracing, all of this becomes a black box.

OpenTelemetry addresses this problem by defining gen_ai semantic conventions. Even when the model provider, framework, and tool execution layer are each written by different teams, using the same attribute names lets every span be unified into a single trace tree for inspection.

The gen_ai Span Hierarchy

The OpenTelemetry gen_ai conventions model an agent loop with three span types.

┌─────────────────────────────────────────────────────────────────┐
│  invoke_agent  (root span — entire agent loop lifetime)         │
│  gen_ai.operation.name = "invoke_agent"                         │
│  gen_ai.agent.name     = "research_agent"                       │
│                                                                 │
│  ├── chat  (model call span — created each iteration)           │
│  │   gen_ai.operation.name   = "chat"                           │
│  │   gen_ai.request.model    = "claude-opus-4-5"                │
│  │   gen_ai.usage.input_tokens  = 4200                          │
│  │   gen_ai.usage.output_tokens =  312                          │
│  │                                                              │
│  ├── execute_tool  (tool execution span)                        │
│  │   gen_ai.operation.name = "execute_tool"                     │
│  │   gen_ai.tool.name      = "web_search"                       │
│  │   gen_ai.tool.call.id   = "call_abc123"                      │
│  │                                                              │
│  └── invoke_agent  (sub-agent delegation — recursive)          │
│      gen_ai.agent.name = "summarizer_agent"                     │
└─────────────────────────────────────────────────────────────────┘

The invoke_agent span wraps the entire lifetime of an agent loop. It opens when the loop starts and closes when the termination condition is satisfied or an error occurs. The chat span is created each iteration when the model is called once. Recording gen_ai.usage.input_tokens and gen_ai.usage.output_tokens lets you track the token cost per iteration. The execute_tool span captures the start and end of a tool execution. The gen_ai.tool.call.id attribute links the span back to the specific tool call the model requested.

Core gen_ai Attributes

Attribute	Span Type	Example Value
gen_ai.system	all	"anthropic", "openai"
gen_ai.operation.name	all	"invoke_agent", "chat", "execute_tool"
gen_ai.request.model	chat	"claude-sonnet-4-5"
gen_ai.agent.name	invoke_agent	"research_agent"
gen_ai.agent.description	invoke_agent	human-readable purpose of the agent
gen_ai.tool.name	execute_tool	"bash", "web_search"
gen_ai.tool.call.id	execute_tool	tool call ID generated by the model
gen_ai.usage.input_tokens	chat	input token count for this iteration
gen_ai.usage.output_tokens	chat	output token count for this iteration
error.type	all	exception class name when an error occurs

W3C Trace Context Propagation

When an agent loop calls an external API, communicates with an MCP (Model Context Protocol) server, or delegates to a sub-agent over HTTP, the root trace ID must travel with the request so that all spans end up in a single trace tree. The W3C traceparent header fills this role.

The traceparent header format is {version}-{trace_id}-{parent_span_id}-{flags}. When a tool inside an agent loop makes an outbound HTTP call and includes this header, the remote service can create its own spans with the same trace_id as the parent, stitching everything together.

This is a conceptual pseudocode example; actual API signatures may differ.

from opentelemetry import trace
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator

tracer = trace.get_tracer("agent.loop", "1.0.0")
propagator = TraceContextTextMapPropagator()

def run_agent_loop(task: str, tools: list) -> str:
    with tracer.start_as_current_span(
        "invoke_agent",
        attributes={
            "gen_ai.system": "anthropic",
            "gen_ai.operation.name": "invoke_agent",
            "gen_ai.agent.name": "research_agent",
        }
    ) as agent_span:
        messages = [{"role": "user", "content": task}]
        while True:
            with tracer.start_as_current_span(
                "chat",
                attributes={
                    "gen_ai.operation.name": "chat",
                    "gen_ai.request.model": "claude-sonnet-4-5",
                }
            ) as chat_span:
                response = call_model(messages)
                chat_span.set_attribute(
                    "gen_ai.usage.input_tokens",
                    response.usage.input_tokens
                )
                chat_span.set_attribute(
                    "gen_ai.usage.output_tokens",
                    response.usage.output_tokens
                )

            if response.stop_reason == "end_turn":
                return response.content

            for tool_call in response.tool_calls:
                with tracer.start_as_current_span(
                    "execute_tool",
                    attributes={
                        "gen_ai.operation.name": "execute_tool",
                        "gen_ai.tool.name": tool_call.name,
                        "gen_ai.tool.call.id": tool_call.id,
                    }
                ):
                    # Inject W3C traceparent into headers for outbound calls
                    carrier = {}
                    propagator.inject(carrier)
                    result = execute_tool(tool_call, headers=carrier)
                    messages.append(tool_result(tool_call.id, result))

Tracing MCP Servers

MCP servers often run as separate processes or services from the main agent loop. When an agent calls an MCP tool and includes the traceparent header, the MCP server’s internal processing spans join the same trace tree. The MCP specification provides propagation points for exactly this purpose, enabling you to answer questions like “which MCP tool consumed the most latency across the entire loop run?”

What Distributed Tracing Gives You

Once the distributed tracing infrastructure is in place, you can answer the following questions with data rather than guesswork.

• Bottleneck detection: Which iteration or tool call accounts for most of the total latency?
• Token cost attribution: How quickly does input_tokens grow as iterations accumulate?
• Error propagation tracking: Which downstream spans were affected after a single tool failure?
• Sub-agent performance: How many iterations did a delegated sub-agent consume before returning?

The next chapter builds on this trace data to examine whether evaluating an agent’s final answer is sufficient — or whether we need to evaluate the entire trajectory.