Instrumenting the OpenAI Agents SDK with OpenTelemetry means capturing agent runs, handoffs, guardrail checks and tool calls as spans over OTLP, so an agentic workflow is traced in your own backend, with its cost and behaviour visible, rather than only in the provider's console..
Instrumenting the OpenAI Agents SDK with OpenTelemetry means capturing agent runs, handoffs, guardrail checks and tool calls as spans over OTLP, so an agentic workflow is traced in your own backend, with its cost and behaviour visible, rather than only in the provider's console.
The OpenAI Agents SDK models an agent that reasons, calls tools, hands off to other agents, and passes guardrails. Observing it means capturing that structure: a span for the agent run, spans for each tool call and handoff, and the guardrail checks, with the underlying LLM calls carrying token, model and cost attributes. The behaviours worth watching are cost per run, latency, whether guardrails tripped, and handoff patterns, because agents that hand off in a loop, like any agent loop, burn cost and time without converging.
OpenTelemetry has emerging GenAI semantic conventions for LLM and agent spans, and libraries such as OpenLLMetry and OpenInference instrument the popular frameworks to emit them, with token, model and cost attributes, over OTLP. The OpenAI Agents SDK has built-in tracing that can be exported to an OpenTelemetry backend, and OpenInference-style instrumentation maps its runs, handoffs, guardrails and tool calls to spans. Exporting over OTLP keeps the traces in the same backend as the rest of your telemetry, and, importantly, in your own control rather than only the provider's dashboard, so agent behaviour is observed alongside the services it touches.
The signals that explain most agent incidents:
| Signal | Source | What it tells you / watch for |
|---|---|---|
gen_ai.usage tokens per run | LLM instrumentation | Tokens and cost per agent run; agent workflows chain many calls, so cost adds up. |
handoff count per run | Agents SDK instrumentation | How many times control passed between agents; a high count can signal a handoff loop. |
guardrail interventions | Agents SDK instrumentation | How often a guardrail blocked or redirected the agent, a direct safety and behaviour signal. |
tool call duration and errors | tool instrumentation | Latency and failures of tools the agent calls; a failing tool sends it into retries. |
gen_ai.client.operation.duration | LLM instrumentation | Model call latency within the agent run. |
run success / latency | Agents SDK instrumentation | Whether the run completed and how long it took end to end. |
An agent dashboard should surface cost, safety and loops. Row one: cost and tokens per run, so an expensive workflow stands out. Row two: guardrail interventions and handoff counts, the safety and loop signals. Row three: tool call latency and errors, and run success rate and latency.
# cost per agent run
sum(gen_ai.usage.input_tokens * price_in + gen_ai.usage.output_tokens * price_out) by (agent.run_id)
# handoff loops: runs with an unusually high handoff count
max(agent.handoff.count) by (agent.run_id)
# guardrail intervention rate
rate(agent.guardrail{action="blocked"}[5m])A run's cost and latency spike because agents keep handing off to one another without resolving. The handoff count for the run is far above normal. Bound handoffs, tighten agent instructions, or add a stopping condition; the handoff-count metric is the early tell.
A guardrail repeatedly blocks the agent, so runs fail or produce refusals, because inputs are hitting a safety rule or the agent is attempting disallowed actions. Guardrail intervention rate flags it. Investigate whether the guardrail or the agent behaviour needs adjusting.
Agent runs get more expensive as prompts grow or the agent takes more steps and tool calls. Cost per run and tokens per run show the trend, and the trace shows which step added the tokens. Trim prompts and bound iterations.
Starting thresholds to tune:
| Condition | Signal | Meaning |
|---|---|---|
| cost per run > budget | cost | An agent workflow is over budget. |
| handoff count > baseline x2 | loop | A handoff loop is developing. |
| guardrail block rate rising | safety | Agents are repeatedly hitting guardrails. |
| run failure rate > threshold | reliability | Agent runs are failing. |
Ops Singularity's AI/ML and Agent Ops pillar operates agentic workflows in production, ingesting this OpenTelemetry data, watching cost, guardrails and handoff loops, and taking governed action when an agent misbehaves. Explore AI/ML and Agent Ops and agentic AIOps.
Yes. The SDK has built-in tracing that can be exported to an OpenTelemetry backend, and OpenInference-style instrumentation maps runs, handoffs, guardrails and tool calls to spans over OTLP.
So agent behaviour is observed alongside the rest of your telemetry and stays in your control, rather than only in the provider's dashboard, which is essential for governance and correlation.
Capture prompt and completion tokens per call as gen_ai.usage attributes and multiply by the model's price. Because the tokens are on every span, cost rolls up per model, per feature and per request.
Bring a real incident. We will show you Sentinel investigate, act and verify end to end, with every action reversible and audited.