"The model doesn't know our business" can be solved with prompt engineering, RAG, fine-tuning, or continued pre-training. On AIP-C01, fine-tuning shows up often in answer options, but far more often as a distractor than as the answer, because the exam expects you to try lighter approaches before expensive ones. This post covers how to choose, the scenarios where fine-tuning is wrong, and how to deploy and manage custom models with Bedrock and SageMaker AI (skill 1.2.4).
Start with the Lightest Option
| Requirement | Choose | |-------------|--------| | Change format or tone; a few examples are enough | Prompt engineering (system prompt, few-shot) | | Answer from frequently changing documents, with sources | RAG (Bedrock Knowledge Bases) | | Consistent format and quality on a task, thousands of examples | Fine-tuning | | Learn domain language from a large unlabeled corpus | Continued pre-training (limited models) | | Large-model quality at small-model cost and speed | Distillation | | Serve an open model trained elsewhere through Bedrock | Custom Model Import |
Separate knowledge from behavior. Facts the model doesn't know (product specs, policies, inventory) call for RAG. Changing how the model behaves (output format, tone, a narrow task) calls for fine-tuning. The two combine well: a model fine-tuned for your report format can still use RAG for current documents.
!Customization spectrum: start from the left
When Fine-Tuning Is the Wrong Answer
Prices, inventory, or policies change daily: you can't retrain that often, so use RAG. Answers must show source documents: weights can't cite sources. Users may only see documents they're permitted to see: knowledge in weights can't be split per user, so use RAG with metadata filters. Minimal operational overhead and fast delivery: prompts or managed RAG come first. Only a few dozen labeled examples: start with few-shot prompting. Reduce hallucinations: ground answers with RAG and verify them with the contextual grounding check.
Fine-tuning is right when output must always follow a specific JSON structure or report template, when you need a brand voice, when a narrow task such as classification or extraction needs higher accuracy, or when long few-shot prompts are driving up token cost and latency.
Customization Methods in Amazon Bedrock
| Method | Data | Purpose | |--------|------|---------| | Supervised fine-tuning | Labeled input-output pairs | Task, format, or tone | | Reinforcement fine-tuning | Prompts + a Lambda reward function | Learn from quality scores instead of labeled answers | | Distillation | Use-case prompts; the teacher generates responses | Smaller, faster, cheaper student model | | Continued pre-training | Unlabeled domain text | Domain language (limited models) | | Custom Model Import | Open model weights trained elsewhere | Serve through the Bedrock API |
Training data is JSONL in S3. Start a job with , or use the Step Functions integration to wait for completion. Training is billed by tokens times epochs plus monthly model storage. To invoke a custom model, buy Provisioned Throughput or, for supported models, deploy it for on-demand inference.
Fine-tuning creates a private copy of the base model, and your data isn't used to train the base model. Encrypt training data and the custom model with your own KMS key, remove PII first with Comprehend or Macie, and compare the custom model with the base model in Bedrock Evaluations before paying for capacity.
Deploying and Managing Custom Models with SageMaker AI
LoRA trains small adapter matrices while the base weights stay frozen. With SageMaker AI inference components, one endpoint hosting a base model can serve many LoRA adapters, and versioned adapters can be swapped or rolled back on their own. SageMaker Model Registry tracks versions in model package groups with approval status (, , ); an approval event can trigger the deployment pipeline through EventBridge. Blue/green deployments with all-at-once, canary, or linear traffic shifting roll back automatically when a CloudWatch alarm fires. Large models need tensor parallelism across GPUs, LLM inference containers, and longer and values.
Embedding Models Are a Separate Decision
Fine-tuning or replacing the embedding model means old document vectors and new query vectors no longer share a space, so every document must be re-embedded and re-indexed. That's why chunking, metadata filters, hybrid search, and reranking are usually the better first answers.
Exam Quick Reference
"Answer from changing information and cite sources" -- RAG "Different users may see different knowledge" -- RAG + metadata filters, not fine-tuning "Few labeled examples, quick results" -- Few-shot prompting "Always the same output format or brand voice" -- Supervised fine-tuning "Train on quality scores instead of labeled answers" -- Reinforcement fine-tuning "Large-model quality at small-model cost" -- Distillation "Serve an externally trained open model serverlessly" -- Custom Model Import "Many domain variants on one base model" -- LoRA adapters + SageMaker inference components "Version models and deploy after approval" -- SageMaker Model Registry "Roll back automatically on errors" -- Blue/green deployment + CloudWatch alarms "Required after changing the embedding model" -- Re-embed and re-index all documents