The same model can cost, respond, and scale very differently depending on how you call it. AIP-C01 asks which inference option fits a requirement throughout Domain 4 (12%) and the deployment skills of Domain 2. This post covers batch inference, cross-Region inference, Intelligent Prompt Routing, prompt caching, and semantic caching.
Bedrock Inference Options at a Glance
| Option | Billing | Choose it when | Watch out for | |--------|---------|----------------|---------------| | On-demand | Per input and output token | Unpredictable traffic, development | Per-model, per-Region request and token quotas | | Provisioned Throughput | Hourly per Model Unit | Steady high volume, guaranteed throughput | Not supported with inference profiles | | Batch inference | Lower price than on-demand for supported models (about 50%) | Large jobs with no real-time need | No tool calling, structured output, or prompt caching | | Cross-Region inference | Source-Region price, no extra routing cost | Peak-hour throttling, more throughput | Requests are processed in other Regions | | Intelligent Prompt Routing | Token price of the model the request is routed to | Mixed easy and hard requests | Two models in the same family; optimized for English | | Prompt caching | Lower rate for tokens read from cache | Long system prompts or documents reused often | Any change to the prefix is a cache miss |
Exam scenarios map to the third column: nightly summaries of 100,000 transcripts are batch inference, lunch-hour throttling is cross-Region inference, and a 500-page manual attached to every question is prompt caching.
On-Demand Quotas and Throttling
On-demand quotas limit requests and tokens per minute per model and Region. For , work through these fixes in order:
Retry with exponential backoff and jitter in the AWS SDK. Buffer requests in Amazon SQS and use Lambda reserved concurrency to control the consumption rate. Use API Gateway usage plans so one tenant can't exhaust the shared quota. Spread load with cross-Region inference. If you exceed the limit consistently, request a quota increase or move to Provisioned Throughput.
Batch Inference
Put requests in a JSONL file in S3, one record per line with a and a in InvokeModel or Converse format, and call . Results land in S3, and EventBridge reports job state changes so you don't have to poll. Records are processed independently, so batch can't do tool use or multi-turn exchanges, and it doesn't work with provisioned models.
Related choices: for long single requests such as video generation, and SageMaker Batch Transform for your own containers.
Cross-Region Inference and Inference Profiles
You call cross-Region inference through an inference profile. Geographic profiles (, , ) keep processing inside that geography; global profiles () can use any commercial Region and cost about 10% less. There is no extra routing charge, data stays on the AWS network, and CloudTrail records the call in the source Region with . If data must not be processed outside the EU, the answer is the EU geographic profile, not the global one. For SCPs, geographic profiles need every destination Region allowed, and global profiles need requests where is "unspecified" to be allowed.
Application inference profiles that you create and tag let you track cost and usage per team or application on the same model.
Intelligent Prompt Routing and Model Cascading
Intelligent Prompt Routing puts two models from the same family behind one serverless endpoint and predicts which one will give the better response for each request. You can use a default router or configure the fallback model and the response quality difference yourself. It's optimized for English prompts and can't use your application's own performance data. For models from different families, non-English traffic, or business rules, build cascading in your application: a small model answers first, a validation or confidence check runs, and failures go to a larger model. Store the model ID in AWS AppConfig to switch models without redeploying code.
!Which Bedrock inference option fits?
Prompt Caching vs Semantic Caching
| Factor | Prompt caching (Bedrock feature) | Semantic caching (you build it) | |--------|----------------------------------|---------------------------------| | What is reused | Processing of the prompt prefix (tools, system prompt, documents) | The final answer to an earlier question | | Model call | Still made; only the rest is processed | Skipped on a cache hit | | Match rule | Exact prefix match | Similar meaning (embedding similarity) | | Implementation | , default TTL 5 minutes | Embeddings + vector store + TTL |
For prompt caching, checkpoints are processed in the order tools → system → messages, so put static content first and the user's question last. Check and in the response. Prompt caching isn't available for batch inference.
For semantic caching, embed the question, search MemoryDB or OpenSearch for a similar earlier question, and return the stored answer above a similarity threshold. A threshold that's too low returns wrong answers, and personalized answers must never go into a shared cache. For exact repeats, a hash of the normalized request in DynamoDB or ElastiCache is simpler.
Latency and Priority Options
Response streaming () lowers perceived latency. Latency-optimized inference () is in preview for selected models, and service tiers () offer Priority, Standard, Flex, and Reserved processing for supported models.
Exam Quick Reference
"Large volume, no real-time need" -- Bedrock batch inference (JSONL in S3) "Notify when a batch job finishes" -- EventBridge "Peak-hour throttling" -- Cross-Region inference (inference profile) "Process data only within one geography" -- Geographic inference profile "Separate model costs per team" -- Application inference profile + tags "Smooth out sudden request spikes" -- SQS buffer + Lambda reserved concurrency "Easy questions to a small model automatically" -- Intelligent Prompt Routing "Mixed model families or non-English routing" -- Application-level model cascading "Switch models without redeploying" -- AWS AppConfig "Long fixed prompt reused on every call" -- Prompt caching "Similar questions repeat, skip the model call" -- Semantic caching