How to Monitor Amazon Bedrock with OpenTelemetry

Monitoring Amazon Bedrock with OpenTelemetry means instrumenting your Bedrock model-invocation calls to emit spans with model, token, latency and cost attributes over OTLP, so a managed LLM service is as observable as any other dependency, and its cost and throttling are visible.

What to observe in Bedrock

Amazon Bedrock is AWS's managed service for foundation models, so you call it like any API and it hides the model infrastructure. The signals that matter are per-invocation: the model used, input and output tokens (which drive cost), latency and time-to-first-token, errors, and, importantly, throttling, because Bedrock enforces per-model quotas and a throttled call fails or retries. For applications using guardrails, whether a guardrail intervened is also worth capturing.

How to instrument Bedrock

OpenTelemetry has emerging GenAI semantic conventions for LLM and agent spans, and libraries such as OpenLLMetry and OpenInference instrument the popular frameworks to emit them, with token, model and cost attributes, over OTLP. For Bedrock, the instrumentation wraps the model-invocation SDK calls (the Bedrock runtime InvokeModel and streaming APIs) so each call becomes a span with the gen_ai attributes. Two AWS-native sources complement this: Bedrock publishes invocation, token and latency metrics to CloudWatch, which the OpenTelemetry Collector's awscloudwatch receiver can pull into the same pipeline, and model-invocation logging can capture request detail. The AWS Distro for OpenTelemetry (ADOT) is the supported way to run the Collector in AWS.

Prompts and completions may contain sensitive data, so control whether request and response bodies are captured and redact where needed; keep token, cost and throttle metrics complete for cost and reliability visibility.

Key metrics to watch

The signals that explain most Bedrock incidents:

Signal	Source	What it tells you / watch for
`gen_ai.usage.input_tokens / output_tokens`	Bedrock instrumentation	Tokens per invocation by model; the direct driver of Bedrock cost.
`gen_ai.client.operation.duration`	Bedrock instrumentation	Invocation latency and time-to-first-token per model.
`throttling / ThrottlingException rate`	instrumentation / CloudWatch	Calls throttled against the model's quota; the usual cause of Bedrock request failures under load.
`error rate by model`	instrumentation	Failed invocations, including validation and access errors, per model.
`guardrail interventions`	instrumentation	How often a Bedrock guardrail blocked or modified a response, useful for safety monitoring.
`cost per model / feature`	derived	Tokens mapped to Bedrock pricing, rolled up by model and by application feature.

What to put on a dashboard

A Bedrock dashboard should keep cost and throttling front and centre. Row one: tokens and estimated cost per model, and per application feature, so an expensive model or feature is visible. Row two: invocation latency and time-to-first-token, and error rate by model. Row three: throttling rate against quota and guardrail interventions.

# cost per model (tokens x Bedrock price)
sum(gen_ai.usage.output_tokens * price_out + gen_ai.usage.input_tokens * price_in) by (gen_ai.request.model)

# throttling: hitting per-model quota
rate(bedrock_throttling_exceptions[5m]) by (model_id)

# invocation latency by model
histogram_quantile(0.95, gen_ai.client.operation.duration) by (gen_ai.request.model)

Reading the signals: common failure modes

Throttling under load

Invocations fail or retry because you exceeded the per-model requests-or-tokens-per-minute quota. The throttling rate is the signal. Request a quota increase, spread load, add backoff, or use provisioned throughput for steady high-volume workloads.

Latency from large contexts

Invocation latency and time-to-first-token climb because prompts carry large context or outputs are long. The duration metric rises with token count. Reduce context, cap output length, or stream so users see tokens sooner.

Cost creep by model

Bedrock spend rises because a feature switched to a more expensive model or token usage grew. Cost attributed per model and per feature shows where; the trace of a costly request shows why. Right-size the model choice and trim prompts.

Example alert conditions

Starting thresholds to tune:

Condition	Signal	Meaning
throttling rate > 0 sustained	quota	Hitting a Bedrock model quota; requests failing.
p95 invocation latency > SLO	latency	Large contexts or provider slowdown degrading UX.
cost per feature > budget	cost	A feature's Bedrock spend is over budget.
error rate by model > 2%	errors	Invocations failing for a model.

From monitoring to autonomous resolution

Ops Singularity ingests Bedrock telemetry over OpenTelemetry (and CloudWatch via ADOT), and its AI/ML and Agent Ops and FinOps pillars watch cost, latency and throttling, with Sentinel AI acting on incidents through governed Action Tickets. Explore AI/ML and Agent Ops and monitoring AWS with OpenTelemetry.

Frequently asked questions

Can I monitor Amazon Bedrock with OpenTelemetry?

Yes. Instrument the Bedrock model-invocation SDK calls so each becomes a span with gen_ai token, latency and cost attributes over OTLP, and complement it with Bedrock's CloudWatch metrics via the awscloudwatch receiver.

How do I handle Bedrock throttling?

Watch the throttling rate against your per-model quota, then request a higher quota, spread or back off load, or use provisioned throughput for steady high-volume use.

How do I track LLM cost with OpenTelemetry?

Capture prompt and completion tokens per call as gen_ai.usage attributes and multiply by the model's price. Because the tokens are on every span, cost rolls up per model, per feature and per request.