Many people preparing for MLA-C01 come from infrastructure, backend, or data engineering and have never studied ML in depth. Yet Task 2.2 of the exam guide names these concepts directly: elements of the training process such as epochs, steps, and batch size; early stopping; dropout, weight decay, and L1 and L2 regularization; random search and Bayesian optimization; the effect of hyperparameters such as the number of trees or layers; preventing overfitting, underfitting, and catastrophic forgetting; ensembling and stacking; and reducing model size. In our question bank, about 28 questions have these concepts at their core.
This post explains them with analogies and tables instead of formulas. It starts with how training proceeds, then moves through hyperparameters, diagnosing overfitting and regularization, tuning strategies, ensembles, catastrophic forgetting in pre-trained models, and reducing model size. How to apply these concepts to SageMaker training jobs, distributed training, and Spot training is covered in Model Training, Tuning, and Evaluation.
---
How Training Proceeds: Epochs, Batches, Steps, and Learning Rate
Imagine preparing for an exam with a single workbook. You solve a few problems, check the answers, fix what you got wrong, and when you finish the book you start again from the beginning. Training a model has the same structure.
| Term | Meaning | Workbook analogy | |------|------|------| | Batch size | The number of samples seen before the weights are updated once | The number of problems you solve before checking | | Step (iteration) | Seeing one batch and updating the weights once | One round of checking and fixing | | Epoch | Seeing the entire training dataset once | One pass through the whole workbook | | Learning rate | How big each update is | How much you change your approach after a mistake |
For example, training on 10,000 samples with a batch size of 100 gives 100 steps per epoch, and 10 epochs is 1,000 steps.
| Situation | What happens | Response | |------|------|------| | Learning rate too high | The loss jumps around or even grows (fails to converge) | Lower the learning rate | | Learning rate too low | The loss falls very slowly and training takes a long time | Raise the learning rate or use a learning rate schedule | | Batch size increased a lot | Throughput per step and GPU utilization improve, but accuracy can drop with the same learning rate | Increase the learning rate in proportion to the batch size (linear scaling) | | Too many epochs | The model fits the training data too closely | Early stopping |
Exam tip: "The loss oscillates wildly and does not decrease" is solved by lowering the learning rate. "The total batch size was increased eightfold for distributed training and accuracy dropped" is solved by increasing the learning rate as well.
---
Parameters vs. Hyperparameters
Parameters are values the model finds on its own during training (the weights of a neural network, the split points of a tree), while hyperparameters are values a person sets before training (learning rate, batch size, epochs, number of trees). The exam asks how the model changes when you change a hyperparameter.
| Model | Hyperparameter | Increasing the value | |------|------|------| | Tree model (XGBoost) | Number of trees (num_round) | More complexity and longer training | | Tree model (XGBoost) | Tree depth (max_depth) | Finer splits, but a higher risk of overfitting | | Tree model (XGBoost) | Learning rate (eta) | Each tree has more influence, so it fits faster but risks overfitting | | Tree model (XGBoost) | subsample, colsample_bytree | Lowering them below 1 adds randomness and reduces overfitting | | Tree model (XGBoost) | min_child_weight, gamma, alpha, lambda | More conservative splits, less overfitting | | Neural network | Number of layers and neurons | More expressive power and model size, risk of overfitting | | Neural network | Dropout rate | Less overfitting; too high causes underfitting |
Exam tip: If "XGBoost is accurate on the training data but poor on the validation data," candidate answers are lowering max_depth and eta, lowering subsample, and raising min_child_weight, gamma, alpha, and lambda.
---
Diagnosing Overfitting and Underfitting
A student who memorized the answers to past exams gets every repeated question right but fails new ones (overfitting). A student who barely studied gets both old and new questions wrong (underfitting). You diagnose this by putting the error on the training data next to the error on the validation data.
!Learning curves of training and validation loss showing underfitting, a good fit, and overfitting
| State | Training error | Validation error | How to fix it | |------|------|------|------| | Underfitting (high bias) | High | High | More complex model, more features, train longer, relax regularization | | Good fit | Low | Low and close to the training error | Keep it | | Overfitting (high variance) | Low | High (large gap) | More data or augmentation, stronger regularization, dropout, feature selection, early stopping, simpler model |
SageMaker Debugger watches the loss and tensors during training, detects problems with built-in rules such as overfit, overtraining, loss_not_decreasing, vanishing_gradient, and exploding_tensor, and can stop the training job automatically when a condition is met. Debugger has not accepted new customers since July 30, 2026, and AWS points to a combination of SageMaker AI MLflow, TensorBoard, and CloudWatch as the alternative. On MLA-C01 it is still the answer for debugging convergence issues.
Exam tip: "99% training accuracy, 70% validation accuracy" is overfitting, and "60% for both" is underfitting. An option that adds more regularization to an underfitting model is wrong.
---
Preventing Overfitting: Regularization and Other Methods
| Technique | How it works | Characteristics | |------|------|------| | L1 regularization (Lasso) | Penalizes the sum of the absolute values of the weights | Drives some weights to exactly 0, which acts as feature selection | | L2 regularization (Ridge, weight decay) | Penalizes the sum of the squared weights | Keeps all weights small but does not drive them to 0 | | Dropout | Randomly turns off some neurons during training | Keeps the network from relying on specific neurons | | Early stopping | Stops training when validation performance stops improving | Prevents overfitting and saves training time at once | | Data augmentation | Expands the data by rotating or flipping images and paraphrasing sentences | Effective when data is scarce | | Feature selection | Removes features with low importance | A simpler model that is easier to interpret |
Exam tip: "Automatically filter out useless features among hundreds to simplify the model" is L1 regularization. "Keep all features but prevent the weights from growing too large" is L2 regularization.
---
Hyperparameter Tuning Strategies
SageMaker automatic model tuning (AMT) runs many training jobs with different hyperparameter combinations and finds the best one.
| Strategy | How it works | When to choose it | |------|------|------| | Grid search | Tries every combination of the values you specify | When there are only a few categorical values | | Random search | Picks combinations at random within the ranges, without using previous results | When you want to run many jobs in parallel | | Bayesian optimization | Uses previous results to estimate which combination to try next | When you need good results with few training jobs | | Hyperband | Stops poor-performing trials early and shifts resources to promising ones | Training that reports intermediate results, such as deep learning over epochs |
Turning early stopping to Auto in a tuning job ends hopeless training jobs early and reduces cost. Warm start carries over previous tuning results so you do not search from scratch.
Exam tip: "Get close to the optimum with the fewest training jobs" is Bayesian optimization, "as many jobs at once as possible" is random search, and "cut deep learning tuning time by dropping bad combinations quickly" is Hyperband.
---
Combining Several Models: Ensembles
Pooling the opinions of several experts is often more accurate than relying on one. Ensembling combines the predictions of several models.
| Technique | How it works | Mainly reduces | Typical example | |------|------|------|------| | Bagging | Trains several models independently on random samples of the data, then averages or votes | Variance (overfitting) | Random forest | | Boosting | Each model in sequence corrects what the previous ones got wrong | Bias (underfitting) | XGBoost, LightGBM | | Stacking | A meta-model takes the predictions of different models as input and makes the final prediction | Each model's weaknesses | A meta-model on top of XGBoost + a neural network + a linear model |
Exam tip: "Combine the strengths of different kinds of models (trees, neural networks, linear) to improve accuracy" is stacking. "Stack weak models in sequence to reduce errors" is boosting.
---
Adapting Pre-Trained Models: Transfer Learning and Catastrophic Forgetting
Instead of training a model from scratch, transfer learning (fine-tuning) continues training a model that has already learned from a large dataset on your company's smaller dataset. You get high accuracy with less data and time.
The problem is catastrophic forgetting. As you fine-tune on new data, the model forgets what it used to do well. A language model fine-tuned on your company's legal documents might start giving odd answers to general questions.
Mix in old data (replay): train on the new data together with part of the data from the original task Lower the learning rate and reduce epochs: avoid shaking the existing weights too much Freeze layers: keep the early layers fixed and train only the later ones Train only some parameters (PEFT techniques such