In our MLA-C01 question bank, about 46 questions have Kinesis, Lambda, EventBridge, or Step Functions as a core concept. The exam guide asks about the same skills: Task 1.1 covers ingesting streaming data with Kinesis, Apache Flink, and Apache Kafka, Task 1.2 covers transforming streaming data with Lambda and Spark, Task 3.1 covers choosing an orchestrator between Apache Airflow and SageMaker Pipelines, and Task 3.3 covers configuring training and inference jobs with EventBridge rules. The heart of it is how you receive data that arrives in real time, where you process it, and what wakes up the next step.
This post starts with the difference between batch and streaming, then covers the four streaming services, the details of Firehose, how to run real-time inference on a stream, waking up pipelines with S3 events and EventBridge, the three orchestrators, and the services that were renamed or retired. The S3-based data lake and data formats were covered first in Data Ingestion and Storage.
---
Batch vs. Streaming: What Is the Difference?
A parcel service collects packages and sends them all at once at a set time. Tap water flows continuously as soon as you open the faucet. If batch processing is the parcel service, streaming is the tap.
| Aspect | Batch | Streaming | |------|------|------| | When it runs | Collected and processed together at intervals | Continuously, as soon as data arrives | | Latency | Minutes to hours | Milliseconds to seconds | | Typical tools | Glue ETL, EMR, SageMaker Processing, Batch Transform | Kinesis, Firehose, Managed Service for Apache Flink, MSK | | ML example | Weekly retraining, scoring every customer each night | Fraud detection at the moment of payment, updating recommendations on every click |
Real systems use both. The most common design makes real-time decisions on the stream while landing the same data in S3 for batch retraining.
---
The Four Streaming Services
| Service | Role | Key characteristics | Clue in the question | |------|------|------|------| | Kinesis Data Streams | Real-time ingestion and temporary retention | Throughput per shard; multiple consumers read the same data independently and can replay it | "several applications process the same stream in real time" | | Amazon Data Firehose | Automatic delivery to a destination | No servers or shards to manage; buffers and loads into S3, Redshift, or OpenSearch | "store streaming data in S3 without writing code" | | Managed Service for Apache Flink | Stateful stream processing | Time-window aggregations, stream joins, real-time anomaly detection | "5-minute moving average," "real-time aggregation on the stream" | | Amazon MSK | Managed Apache Kafka | Use existing Kafka applications and tools as they are | "we already use Kafka" |
A single Kinesis Data Streams shard accepts 1 MB or 1,000 records per second and emits 2 MB per second. When throughput is not enough, add shards or use on-demand mode, which manages capacity automatically. Data is retained for 24 hours by default and up to 365 days, so it can be reread after a failure. For video, ingest it with Kinesis Video Streams and analyze it with Rekognition or a SageMaker AI model.
Exam tip: "Just receive the data and land it in S3" is Firehose, "several consumers process the received data in real time" is Kinesis Data Streams, and "time-window aggregations and state on top of the stream" is Managed Service for Apache Flink.
!Streaming ML pipeline in which events received by Kinesis Data Streams are scored in real time by Lambda and a SageMaker endpoint, while Firehose lands them in S3 for retraining
---
A Closer Look at Firehose: Buffering, Transformation, and Formats
The exam often asks about Firehose's specific features.
| Feature | What it does | Clue in the question | |------|------|------| | Buffering | Delivers when either the buffer size (1–128 MiB) or the buffer interval (up to 900 seconds) is reached first | "near real-time is good enough" | | Lambda transformation | Cleans records or adds fields before delivery | "remove personal data fields before loading" | | Source record backup | Keeps the original, untransformed records in a separate S3 path | "keep both the transformed result and the original" | | Format conversion | Converts JSON to Parquet or ORC before storing (using a schema from the Glue Data Catalog) | "columnar format to cut Athena query costs" | | Dynamic partitioning | Splits the S3 path by values inside the record (such as date or customer ID) | "separate into a folder per customer automatically" |
Format conversion expects JSON input. Other formats such as CSV are first converted to JSON with a Lambda transformation. When delivering to Redshift, Firehose first writes the data to S3 and then loads it into Redshift (COPY).
Exam tip: "A delay of about a minute is fine, and the data must land in S3 as Parquet with no operational overhead" is Firehose with format conversion. Because of buffering, Firehose does not fit millisecond-level real-time processing.
---
Running Real-Time Inference on a Stream
To decide whether a payment is fraudulent at the moment it happens, you need to call the model for each event in the stream.
| Stage | Service | What it does | |------|------|------| | Ingestion | Kinesis Data Streams | Receives payment events in real time | | Processing and invocation | Lambda (or Managed Service for Apache Flink) | Reads events in batches, preprocesses them, and calls the SageMaker endpoint | | Inference | SageMaker AI real-time endpoint | Returns the fraud score | | Storing results and alerting | DynamoDB, SNS | Stores the score and sends an alert above the threshold | | Retraining data | Firehose → S3 | Lands the raw events for batch retraining |
When a Kinesis stream is connected to Lambda as an event source, Lambda receives and processes records in batches. Features that need a time window, such as the number of transactions in the last 10 minutes, are better computed in Flink. If you write the features computed in real time to the SageMaker Feature Store online store, inference reads the same values, which keeps training and serving features consistent.
Exam tip: "Score each transaction in the stream in real time and keep the raw data for retraining" is Kinesis Data Streams + Lambda + a SageMaker endpoint, combined with Firehose → S3.
---
Waking Up Pipelines with Events: S3 Events and EventBridge
Event-driven automation means the next step starts automatically when something happens, without someone pressing a button every time.
| Trigger | Example | What it connects to | |------|------|------| | S3 event notification | A new training data file is uploaded | Lambda, SQS, SNS, or EventBridge | | EventBridge schedule rule | Every Sunday at 2 a.m. | A SageMaker pipeline execution | | SageMaker state change events | Training job completed or failed, endpoint status changed | Lambda notifications, starting the next step | | Model Package State Change | A model is approved in the model registry | A deployment pipeline (CodePipeline and so on) | | CloudWatch alarm | Model Monitor detects a drift violation | A retraining pipeline |
EventBridge supports SageMaker Pipelines as a direct target, so when a matching event arrives it can start a pipeline execution (StartPipelineExecution) without going through Lambda. Sending S3 events to EventBridge lets you fan the same event out to several targets and apply finer-grained conditions.
Exam tip: "Automatically run the retraining pipeline when new data lands in S3" is S3 event → EventBridge → SageMaker Pipelines. "Deploy automatically when a model is approved" is an EventBridge rule that receives the Model Package State Change event.
---
Choosing an Orchestrator: SageMaker Pipelines, Step Functions, or MWAA
| Orchestrator | Strengths | Best fit | |------|------|------| | SageMaker Pipelines | ML-specific steps such as processing, training, tuning, evaluation, conditions, and model registration; step caching; lineage tracking; model registry integration | Automating the ML workflow itself with minimal operations | | AWS Step Functions | General-purpose state machine, direct integrations with many AWS services, error retries and branching, parallel processing with the Map state, waiting for human approval | One flow that ties together several services such as Lambda, Glue, SageMaker, and translation | | Amazon MWAA | Managed Apache Airflow, Python DAGs, SageMaker operators | Teams already running data pipelines as Airflow DAGs |
Step Functions supports an integration (.sync) that starts a long-running job, such as a SageMaker training job, and waits until it finishes. The Map state is used to split thousands of files in S3 and process them at the same time (for example, translating thousands of documents in parallel).
Exam tip: "The company already runs hundreds of Airflow DAGs" is MWAA, "a pipeline made only of ML steps with the least operational overhead" is SageMaker Pipelines, and "a flow that mixes Lambda, Glue, SageMaker, and human approval" is Step Functions.
---
Renamed or Retired Services
Older materials and question banks often use the old names.
| Old name | Now | Notes | |------|------|------| | Kinesis Data Firehose | Amazon Data Firehose | Renamed in 2024; same features | | Kinesis Data Analytics (for Apache Flink) | Amazon Managed Service for Apache Flink | Renamed in 2023 | | Kinesis Data Analytics for SQL | Discontinued | No new applications since October 15, 2025; applications deleted and support ended from January 27, 2026 |
The service list in the exam guide (Version 1.0) also uses the new names (Amazon Data Firehose, Amazon Managed Service for Apache Flink). If a question says "aggregate the stream in real time with SQL using Kinesis Data Analytics," read it today as Managed Service for Apache Flink (which includes Flink SQL).
---
Practice with Scenarios
| Situatio