AWS Data Ingestion Services: Complete Guide | DEA-C01
In the AWS Certified Data Engineer - Associate (DEA-C01) exam, Domain 1: Data Ingestion and Transformation (34%) carries the highest weight. The core focus of this domain is understanding the differences between AWS data ingestion services. Knowing each service's characteristics and appropriate use cases allows you to select the right service in exam scenarios.
---
Delivery Semantics
How reliably a service delivers messages is a fundamental criterion for service selection. There are three delivery guarantee models.
| Model | Description | Characteristics | |-------|-------------|-----------------| | At Most Once | Delivered at most once — loss possible | Used for low-value data or when immediate processing is required | | At Least Once | Delivered at least once — duplicates possible | Most common; supports reprocessing via checkpoints | | Exactly Once | Delivered exactly once | Implemented by combining Idempotency with At Least Once |
SQS FIFO and DynamoDB Streams guarantee Exactly Once delivery. Most streaming services, including Kinesis Data Streams and Amazon MSK, follow the At Least Once model.
---
Service Specification Comparison
The following table summarizes delivery guarantee level, ordering support, data retention, parallel consumption, and maximum message size for each service.
| Service | Delivery Guarantee | Ordering | Data Retention | Parallel Consumption | Message Size | |---------|--------------------|----------|----------------|----------------------|--------------| | DynamoDB Streams | Exactly Once | Yes | 24 hours | Yes | 400 KB | | Kinesis Data Streams | At Least Once | Yes | 24 hours (default) | Yes | 1 MB | | Kinesis Firehose | At Least Once | No | N/A | No | Destination size | | Amazon MSK | At Least Once | Yes | Configurable | Yes | Varies | | SQS Standard | At Least Once | No | Up to 14 days | No | 256 KB | | SQS FIFO | Exactly Once | Yes | 4 days default (up to 14 days) | No | 256 KB |
Kinesis Firehose does not support ordering or parallel consumption, but its fully managed auto-scaling makes it optimal for loading data into S3, Redshift, and OpenSearch.
---
Kinesis Data Streams vs Kinesis Firehose
Both services handle streaming data, but serve different purposes. Use Kinesis Data Streams when you need real-time data processing or fine-grained control. Choose Kinesis Firehose when you need automated loading into a data lake or store.
Key characteristics of Kinesis Data Streams:
Shard-based scale-out: 1 shard = 1 MB/s input, 2 MB/s output Throughput must be managed manually, but allows fine-grained control Pricing: per shard-hour + PUT payload units
Key characteristics of Kinesis Firehose:
Fully managed — scales automatically without manual intervention Additional costs apply for data format conversion Direct delivery to S3, Redshift, OpenSearch, Splunk, and other destinations
---
AWS Glue vs AWS DMS
Both AWS Glue and AWS DMS move data, but serve different purposes. Use DMS when migrating a database to AWS, and use Glue when transforming and loading data from S3 or RDS.
| Comparison | AWS Glue | AWS DMS | |------------|----------|---------| | Primary Use | ETL / data transformation and loading | Database migration | | Scaling | Specify number of DPUs (Data Processing Units) | Adjust EC2 replication instance size | | Fault Tolerance | 3 retries by default + SNS notification | Multi-AZ option | | Pricing | Per-second billing for crawlers and ETL jobs | Instance type + storage + data transfer |
---
Amazon MSK vs Apache Kafka on EC2
Amazon MSK is a fully managed Kafka service by AWS. Choose MSK to reduce management overhead, or install Kafka directly on EC2 when you need complete control.
| Item | Amazon MSK | Apache Kafka on EC2 | |------|------------|---------------------| | Managed | Yes (fully managed by AWS) | No (self-operated) | | Broker Count | Up to 30 (increase on request) | Unlimited | | Multi-AZ | Configurable | Configurable | | Cost Structure | Managed service fee | Infrastructure cost + operational cost |
---
Service Keyword Summary
Use the following keywords to quickly identify the right service in exam questions.
| Service | Key Keywords | |---------|--------------| | Kinesis Data Streams | Real-time, shard, At Least Once, 1 MB | | Kinesis Firehose | Auto-scaling, data lake loading, no ordering | | SQS Standard | Queue, At Least Once, no ordering, 256 KB | | SQS FIFO | Queue, Exactly Once, ordering guaranteed, 300 TPS | | Amazon MSK | Managed Kafka, At Least Once, up to 30 brokers | | AWS Glue | ETL, DPU scaling, Spark, 3 retries | | AWS DMS | DB migration, EC2-based, Multi-AZ |
---
The key to succeeding in the data ingestion domain of DEA-C01 is clearly understanding the differences between services. Delivery semantics, ordering guarantees, and data retention periods are frequently tested topics, so make sure to review the tables above thoroughly.