AI Fundamentals
Core concepts, definitions, and how AI works from the ground up.
52 questions
An LLM, or large language model, is an AI system trained on enormous amounts of text so it can understand and generate human langu…
An LLM works by predicting the next piece of text, one small step at a time, based on everything before it. When you type a questi…
A token is the small chunk of text an LLM actually reads and writes — usually a word, part of a word, or a piece of punctuation. L…
A context window is the maximum amount of text an LLM can consider at one time, measured in tokens. Think of it as the model's sho…
An embedding is a way of turning text into a list of numbers that captures its meaning, so a computer can compare ideas mathematic…
Temperature is a setting that controls how random or predictable an LLM's output is. At a low temperature (close to 0), the model…
Inference is what happens when an LLM actually runs to answer you — the moment it takes your prompt and generates a response. It's…
LLM reasoning is a model's ability to work through a problem in steps rather than blurting out an immediate answer. Instead of jum…
RAG, or retrieval-augmented generation, is a technique that lets an LLM answer using specific documents it fetches at the moment y…
An LLM agent is an AI system that doesn't just answer questions — it takes actions to accomplish a goal, using an LLM as its decis…
An LLM hallucination is when the model states something false as if it were true — confidently inventing facts, sources, names, or…
LLM training is the process of teaching a large language model to understand and generate language by having it learn patterns fro…
An LLM's architecture is the underlying design of the neural network that makes it work — and for virtually every modern LLM, that…
A foundation model is a large AI model trained on a broad sweep of data that can be adapted to many different tasks, rather than b…
RLHF, or reinforcement learning from human feedback, is a training technique that teaches an LLM to give answers people actually f…
Quantization is a technique that shrinks an LLM by storing its numbers at lower precision, making the model smaller and faster to…
Distillation is a technique for training a smaller, cheaper LLM to mimic a larger, more capable one — capturing much of the big mo…
A prompt is simply the input you give an LLM — the question, instruction, or text you type in to get a response. It's how you tell…
The four types of AI are reactive machines, limited memory, theory of mind, and self-aware AI — a classic framework describing how…
LLMs are used for almost anything involving language — writing, summarizing, answering questions, translating, coding, and increas…
Attention is the mechanism that lets an LLM decide which words in the input matter most when interpreting each word — and it's the…
Grounding is the practice of tying an LLM's answers to real, verifiable information — actual documents, data, or search results —…
MCP, the Model Context Protocol, is an open standard for connecting AI models to external tools and data sources in a consistent w…
Most LLMs are built primarily in Python, with the heavy number-crunching handled underneath by highly optimized code in C++ and CU…
A multi-agent system is an AI setup where several LLM-powered agents work together, each handling part of a task, instead of relyi…
RAG architecture is built as a two-phase pipeline: an offline step that indexes your documents into a searchable form, and a runti…
Chunking is the step where you cut a source document into smaller pieces before turning it into embeddings and storing it in your…
A RAG agent is a retrieval-augmented generation system wired into an agent loop, so it can search multiple times, choose between d…
Yes, substantially — but not to zero. RAG (Retrieval-Augmented Generation) meaningfully cuts down hallucination, and it's one of t…
There are three main variations on plain RAG — RAG Fusion, Hybrid Search, and HyDE — each patching a specific weakness in the basi…
GraphRAG is RAG that retrieves from a knowledge graph — a map of entities and how they relate to each other — instead of, or along…
A system prompt is the set of instructions a company or developer writes in advance, invisible to you, that tells the AI who to be…
Context rot is when an AI model's answers get less reliable the more you pack into its context window, even though everything you…
An AI agent is a software system that uses a large language model as its reasoning engine to pursue a goal on its own. Instead of…
Agentic AI is AI that pursues a goal on its own instead of waiting for the next prompt. You hand it an objective - "book the cheap…
Function calling - also called tool use - is how a language model reaches beyond text and gets real work done. Instead of guessing…
A vector database is a system built to store and search data by meaning rather than by exact matches. It turns text, images, or au…
Vector search is a way of finding information by meaning instead of exact wording. It turns text, images, or audio into lists of n…
A vector database is used to search and compare things by meaning rather than exact values. Because it stores embeddings and finds…
An AI coding agent is a coding tool that works in a loop: it reads your request, makes a plan, edits files across your project, ru…
Generative AI is a type of artificial intelligence that creates brand-new content — text, images, video, audio, and code — by lear…
AI image generation works by starting with a field of random visual noise and gradually cleaning it up into a picture, guided by t…
AI video generation works much like AI image generation, but stretched across time. Instead of producing one still picture, the mo…
A diffusion model is a type of AI that generates images by learning to reverse a process of adding noise. During training it repea…
Retrieval-augmented generation (RAG) is a technique that connects a large language model to an external knowledge source, so it ca…
Reranking is a second-stage pass that reorders the chunks your retriever pulled back, scoring each one against the question with a…
Hybrid search combines two retrieval methods that fail in opposite ways: dense vector search, which matches meaning, and lexical k…
Multimodal RAG is retrieval-augmented generation that retrieves and reasons over more than plain text — images, tables, charts, sc…
Semantic caching stores past answers keyed by the meaning of a query rather than its exact text, so a new question that's close en…
Zero-shot prompting means asking a model to do a task without giving it any examples — you just describe what you want and let the…
Prompt chaining means breaking a task into a sequence of smaller prompts, where the output of one becomes the input to the next. I…
Meta prompting is using an AI model to write or improve prompts instead of writing them all yourself. You describe the job you wan…