Monitoring AWS Lambda with OpenTelemetry means using the OpenTelemetry Lambda layer to emit traces and metrics over OTLP, so serverless functions are as observable as long-running services..
Monitoring AWS Lambda with OpenTelemetry means using the OpenTelemetry Lambda layer to emit traces and metrics over OTLP, so serverless functions are as observable as long-running services.
Lambda functions are short-lived and event-driven, which makes cold starts a first-class concern and traditional agents awkward. The signals that matter are invocation count and errors, duration against the configured timeout, cold-start frequency and latency, throttles, and the latency of the downstream calls each invocation makes. Tracing across the event-driven chain is essential, because a Lambda's behaviour only makes sense in the context of what triggered it.
The two readings that predict most serverless incidents are duration creeping toward the timeout and cold-start rate climbing, both of which turn into user-visible failures before the error rate itself moves.
You attach the OpenTelemetry Lambda layer (the AWS Distro for OpenTelemetry provides one) to the function, which auto-instruments the handler and common SDK calls and exports traces and metrics over OTLP, typically through a Collector extension running in the execution environment. Context propagation carries the trace across the event sources that trigger the function, so an SQS or API Gateway trigger links to the function it invokes.
The serverless-specific caution is overhead: the layer and a synchronous exporter add to cold-start time, so use the async or extension-based export path and keep the instrumentation lean, and sample if invocation volume is high.
The signals that explain most serverless incidents:
| Signal | Source | What it tells you / watch for |
|---|---|---|
faas.invoke_duration | OTel Lambda layer | Execution duration; watch it against the configured timeout, not in isolation. |
faas.invocations | OTel Lambda layer | Invocation rate; the throughput baseline and context for errors. |
faas.errors | OTel Lambda layer | Failed invocations; correlate with a downstream span or a timeout. |
cold start count / latency | OTel Lambda layer | Cold starts and their added latency; a rising rate degrades tail latency. |
Lambda Throttles (CloudWatch) | awscloudwatch | Invocations throttled at the concurrency limit; requests are being rejected. |
downstream span duration | OTel Lambda layer | Latency of calls the function makes; often where a slow invocation's time goes. |
A serverless dashboard should keep duration-against-timeout and cold starts visible. Row one: invocation rate and error rate per function. Row two: duration percentiles against the timeout (the panel that predicts timeout failures) and cold-start rate and latency. Row three: throttles and downstream call latency.
# duration approaching the timeout (per function)
faas.invoke_duration / lambda_timeout_ms by (faas.name)
# cold-start rate
rate(faas_coldstarts[5m]) by (faas.name)
# throttling: hitting the concurrency ceiling
rate(AWS/Lambda.Throttles[5m]) by (function_name)Tail latency is high because a large share of invocations start a fresh execution environment, each paying initialisation and layer overhead. Cold-start rate and its latency reveal it. Use provisioned concurrency for latency-sensitive functions, shrink the deployment package, and keep initialisation light.
Invocations that used to finish comfortably start approaching the configured timeout and then fail, usually because a downstream call got slower. Duration-against-timeout is the leading signal, the errors are the lagging one. Trace the slow downstream call and either speed it up or raise the timeout deliberately.
Invocations are rejected because concurrent executions hit the account or function concurrency limit, so events are dropped or delayed. The throttle metric is the tell. Raise reserved concurrency, or smooth the trigger rate so bursts do not exceed the ceiling.
Starting thresholds to tune:
| Condition | Signal | Meaning |
|---|---|---|
| duration / timeout > 0.8 for 10m | latency | Approaching timeout failures. |
| error rate > 2% for 5m | errors | Invocations failing. |
| throttles > 0 sustained | concurrency | Hitting the concurrency ceiling. |
| cold-start rate rising | latency | Tail latency degrading. |
Ops Singularity's TelemetryOps ingests Lambda traces over OpenTelemetry, and Sentinel AI resolves serverless incidents through governed Action Tickets. Explore TelemetryOps.
Attach the OpenTelemetry Lambda layer, such as the one from the AWS Distro for OpenTelemetry, which auto-instruments the handler and exports traces and metrics over OTLP.
Invocation and error rates, throttles, duration against the timeout, cold-start frequency and latency, and downstream call latency.
Bring a real incident. We will show you Sentinel investigate, act and verify end to end, with every action reversible and audited.