Observability Code Instrumentation

Implement X-Ray Annotations/Subsegments, structured logging, custom metric code, and health checks.

The previous post covered reading logs and metrics with CloudWatch and X-Ray. This post covers how to embed observability features directly into your code — preparing in advance so problems are easier to find later.

 

X-Ray Code Instrumentation

Instrumentation means inserting measurement points into code to collect data during execution.

Annotations

Annotations add searchable, indexed key-value pairs to X-Ray trace data.

Example: adding user_id="abc123" as an Annotation lets you search later "show all requests from this user" in X-Ray.

Think of it like creating index cards for library books — you can quickly find them by keyword.

Important note: Metadata is different from Annotations — it is NOT indexed and not searchable. Used only for supplementary reference information. The exam frequently tests this difference.

Subsegments

Subsegments measure the time taken by individual operations within a single service.

Example, inside a Lambda function: DynamoDB query: 50ms S3 upload: 200ms External API call: 500ms

By recording each operation as a subsegment, you can immediately see where time is being spent.

 

Structured Logging

Outputting logs in JSON format instead of plain text.

Plain log: ERROR: DB connection failed

Structured log: {"level":"ERROR","message":"DB connection failed","requestId":"abc-123","userId":"user-456"}

Benefits: Search by specific fields in CloudWatch Logs Insights Easier error statistics and pattern analysis Standardized automatic processing

 

Custom Metrics

Beyond default AWS metrics, you can send your own business-specific measurements: Order processing time Cart abandonment rate Login success/failure ratio

How to send: PutMetricData API: Direct API call sending metric values to CloudWatch EMF: Output structured logs and metrics are extracted automatically

 

Health Checks

Periodic verification that a service is functioning normally — like taking someone's pulse.

ELB Health Checks: Load balancer sends periodic signals to each server's /health address. If it fails, stops sending traffic. ECS Health Checks: HEALTHCHECK command inside the container. If it fails, automatically restarts. Route 53 Health Checks: Monitors endpoints. If failure detected, Failover routing redirects traffic.

 

Alert Configuration

CloudWatch Alarms -> SNS topic -> email, SMS or Lambda EventBridge rules -> notifications for specific events

 

Exam Key Points

"Search specific user requests in X-Ray" -- Annotations (indexed)

"Measure DB query/HTTP call time in X-Ray" -- Subsegments

"JSON format logs, searchable by field" -- Structured logging

"Send application-specific metrics" -- PutMetricData or EMF

"Periodically check if instances are healthy" -- ELB health checks

"Check container health" -- ECS HEALTHCHECK

"Notify when deployment completes" -- EventBridge + SNS

Annotations = searchable (indexed), Metadata = not searchable

Back to blog list