How to Observe LangChain Apps with OpenTelemetry

What to capture in a LangChain app

A LangChain application is a graph of steps: chains that call other chains, LLM invocations, tool calls, and, in retrieval apps, retrievers that fetch context. The signals worth capturing map onto that structure: a span per chain and step, and, for each LLM call, the model, prompt and completion tokens (which drive cost), latency and any error or rate-limit response. The value of tracing the whole graph is that a slow or expensive request is rarely the LLM alone; it is a particular chain, a retriever, or a tool call, and only the trace shows which.

How to instrument LangChain

OpenTelemetry has emerging GenAI semantic conventions for LLM and agent spans, and libraries such as OpenLLMetry and OpenInference instrument the popular frameworks to emit them, with token, model and cost attributes, over OTLP. For LangChain specifically, the instrumentation hooks into the framework's callback system, so chains, LLM calls, tools and retrievers are captured automatically once the instrumentation is loaded, with an OTLP exporter pointing at your Collector. LangChain has a native tool in LangSmith, but it is proprietary; instrumenting with OpenTelemetry keeps your traces portable and in the same backend as the rest of your telemetry, so an LLM workflow is observed alongside the services around it.

Key metrics to watch

The signals that explain most LangChain incidents:

What to put on a dashboard

A LangChain dashboard keeps cost and experience side by side. Row one is cost: tokens and estimated cost per request by model, with the per-request token distribution so a creeping prompt size is caught before the bill is. Row two is latency: model call latency and total chain latency, so you can see whether the LLM or the surrounding steps dominate. Row three, for retrieval chains and tools: retriever latency and result counts, and tool call errors.

# tokens per request by model (rising = prompt bloat or bigger outputs)
gen_ai.usage.input_tokens + gen_ai.usage.output_tokens  by (gen_ai.request.model)

# chain latency vs model latency (is the LLM or the chain slow?)
histogram_quantile(0.95, chain.span.duration) by (chain.name)

# rate-limit (429) rate by model
rate(gen_ai_errors{type="rate_limit"}[5m]) by (gen_ai.request.model)

Reading the signals: common failure modes

Cost spike from prompt bloat

Spend jumps without a matching jump in traffic, because tokens per request grew, a prompt template expanded or context is being stuffed. The per-request token distribution reveals it, and the trace of one expensive request shows which chain or retriever ballooned the context. Trim prompts and cap retrieved context.

A slow step, not a slow model

End-to-end latency rises but the LLM call is fast; the time is in a retriever, a tool, or a nested chain. The chain span breakdown points at the exact step, which a model-only view would miss entirely.

Rate-limit errors

429s climb as you approach the provider's tokens-per-minute limit, and requests start failing or retrying. The rate-limit rate by model is the signal. Spread load, request higher quota, add backoff, or route overflow to a secondary model.

Example alert conditions

Starting thresholds to tune:

From monitoring to autonomous resolution

Ops Singularity's AI/ML and Agent Ops pillar operates LangChain apps in production, ingesting this OpenTelemetry data, watching cost, latency and retrieval quality, and taking governed action when a chain or model degrades. Explore AI/ML and Agent Ops and TelemetryOps.