Generative AI Core Concepts

Foundation models, LLMs, tokens, context windows, embeddings, vector DBs, chunking, RAG, prompt engineering, Amazon Bedrock — all generative AI core concepts explained for beginners.

AIF-C01 covers generative AI concepts at approximately 25% of the exam. Key terms: LLM, foundation models, tokens, embeddings, RAG, and prompt engineering.

---

 

What Is Generative AI?

Traditional AI classifies or predicts from existing data. Generative AI creates entirely new content — text, images, code, audio. ChatGPT, Claude, DALL-E, and GitHub Copilot are well-known examples.

---

 

Foundation Models

A foundation model is a large AI model pre-trained on massive datasets (internet text, books, code). A single model handles translation, summarization, code generation, and Q&A.

Think of a university education — broad foundational training across many subjects. Fine-tuning is like attending graduate school to specialize in a specific field.

Amazon Bedrock provides access to multiple foundation models (Claude, Llama, Titan) via a single API.

---

 

Core Concepts

Token The minimum unit of text the model processes. One word typically equals 1–2 tokens. Costs, speed, and context limits all depend on token count.

Context Window Maximum tokens the model can process in a single request. Like working memory — if the document exceeds the window, it must be chunked.

Embedding Converting text into a high-dimensional numeric vector. Similar meanings produce similar vectors. "Dog" and "puppy" are close in vector space; "dog" and "rocket" are far apart.

Vector Database Stores embeddings and performs similarity searches. Finds semantically similar documents even when exact words differ.

Chunking Splitting long documents into smaller pieces that fit within the context window. Too small = context lost; too large = irrelevant content mixed in.

---

 

RAG — Solving Hallucination

RAG retrieves relevant documents from a vector database and includes them as context before the FM generates an answer.

Documents embedded and stored in vector DB User question embedded Similar chunks retrieved from vector DB Chunks + question sent to FM FM generates grounded answer

---

 

Prompt Engineering

| Technique | Description | |-----------|-------------| | Zero-shot | Ask directly without examples | | Few-shot | Provide 2–5 examples before asking | | Chain-of-thought | Ask model to think step by step | | Role prompting | Assign a specific expert persona |

---

 

Exam Key Points

"Large pre-trained general-purpose AI model" — Foundation Model "Text understanding and generation specialized FM" — LLM "Minimum unit of text, affects cost" — Token "Max tokens per request" — Context Window "Convert text to numeric vector" — Embedding "Store and search embeddings" — Vector Database "Split long docs into smaller pieces" — Chunking "AI fabricates plausible false information" — Hallucination "Search + context injection to solve hallucination" — RAG "Image generation from noise" — Diffusion Model

Back to blog list