LLM glossary

169 large-language-model terms explained in plain English, from attention to zero data retention. Browse by topic for full definitions, or filter the A to Z list.

Updated October 10, 2026Guides

Browse by topic

Each topic page gives every term a short definition, a fuller explanation and links to related terms.

22 termsFundamentalsThe core ideas: what a large language model is, how text becomes tokens, what parameters and context windows are, and the words used to describe different kinds of models.16 termsModel architectureWhat sits inside a model: the transformer, attention and its faster variants, positional encodings, mixture-of-experts layers, and alternatives such as state space and diffusion models.24 termsTrainingHow models learn: pretraining, fine-tuning and LoRA, preference methods such as RLHF, DPO and GRPO, scaling laws, and the problems that show up along the way.33 termsInference and deploymentRunning models: sampling settings such as temperature and top-p, quantization formats, the KV cache, speed metrics, GPUs and VRAM, and the engines used to serve or self-host models.11 termsPromptingTechniques for instructing a model: system prompts, few-shot examples, chain-of-thought, prompt chaining and context engineering.23 termsBuilding LLM appsBuilding products on models: retrieval-augmented generation, search and reranking, tool calling, MCP, structured outputs and AI agents.14 termsEvaluation and benchmarksMeasuring models: benchmarks such as MMLU, GPQA and SWE-bench, arena Elo ratings, contamination, LLM-as-a-judge and long-context tests.16 termsSafety and governanceRisks, rules and safeguards: prompt injection, jailbreaks, guardrails, red teaming, alignment, privacy, copyright and the EU AI Act.10 termsBusiness and pricingWhat a model costs and how you buy it: per-token pricing, batch APIs, prompt caching, rate limits, dedicated endpoints and model licenses.

Every term, A to Z

169 terms

A

Agent memory memory, LLM memory

Mechanisms that let an AI system keep and recall information beyond a single context window.

Building LLM apps

Agentic workflow agentic AI, agentic system

A multi-step process in which LLMs plan, call tools and act, with control flow set by code or the model.

Building LLM apps

AI agent agent, LLM agent, autonomous agent

A system where an LLM decides which actions to take, uses tools, observes results and repeats until done.

Building LLM apps

AI text detector AI detector, AI content detector, GPT detector

A tool that estimates whether text was written by a person or generated by an AI model.

Safety and governance

Alignment AI alignment

Making AI systems pursue the goals their developers and users intend, and act within acceptable limits.

Safety and governance

API key secret key

A secret credential that authenticates your application to a model provider and ties usage to your account.

Business and pricing

Artificial general intelligence (AGI) AGI

A hypothetical AI that can match or exceed people across most cognitive tasks, not just narrow ones.

Fundamentals

Attention self-attention, multi-head attention

The mechanism that lets each token weigh and combine information from other tokens in the sequence.

Model architecture

Autoregressive model autoregressive generation, next-token prediction

A model that generates output one token at a time, each conditioned on everything before it.

Fundamentals

AWQ (activation-aware weight quantization) activation-aware weight quantization

A 4-bit quantization method that protects the weights most important to a model's activations.

Inference and deployment

B

Base model pretrained model

A model straight out of pretraining that continues text but hasn't been tuned to follow instructions.

Fundamentals

Batch API batch processing, batch inference, asynchronous batch

An asynchronous way to submit many requests at once, typically at half price with results within 24 hours.

Business and pricing

Benchmark LLM benchmark, benchmark suite

A standardized test set and scoring method used to compare models on a specific skill.

Evaluation and benchmarks

Benchmark contamination data contamination, test set leakage

When benchmark questions or answers appear in a model's training data, inflating its scores.

Evaluation and benchmarks

BM25 Okapi BM25, Best Matching 25

A classic keyword-ranking formula that scores documents by term frequency, rarity and length.

Building LLM apps

Byte-pair encoding (BPE) BPE

A subword tokenization method that builds a vocabulary by repeatedly merging the most frequent adjacent symbol pairs.

Fundamentals

C

Catastrophic forgetting catastrophic interference

When training on new data makes a model lose skills or knowledge it previously had.

Training

Chain-of-thought prompting (CoT) CoT, chain of thought, step-by-step reasoning

Getting a model to write out intermediate reasoning steps before giving its final answer.

Prompting

Chat model instruct model, instruction-tuned model, assistant model

A base model further trained to follow instructions and hold a conversation in turns.

Fundamentals

Checkpoint model checkpoint, weights file

A saved snapshot of a model's weights, and often its training state, at a point in training.

Training

Chunking text splitting, document chunking

Splitting documents into smaller passages before embedding them, so retrieval returns focused, relevant text.

Building LLM apps

Citations source attribution, source citations

References in a model's answer that point to the specific sources supporting each claim.

Building LLM apps

Computer use computer-using agent, GUI agent

Agents that operate a computer or browser through screenshots, mouse clicks and keystrokes, like a person.

Building LLM apps

Constitutional AI CAI

Anthropic's method of training models to follow a written set of principles using AI-generated feedback.

Safety and governance

Context engineering

Deciding what information goes into a model's context window, in what form and order, at each step.

Prompting

Context window context length, context size

The maximum number of tokens a model can consider at once, covering both input and output.

Fundamentals

Copyright and training data AI copyright, training data copyright

The unsettled legal questions over training AI models on copyrighted works without permission.

Safety and governance

Cosine similarity

A measure of how closely two vectors point in the same direction, widely used to compare embeddings.

Building LLM apps

Cost per token token pricing, price per million tokens, input and output pricing

How LLM APIs charge: a set price per million input tokens and a usually higher one for output.

Business and pricing

Cross-entropy loss loss, loss function, training loss, negative log-likelihood

The standard training objective for LLMs, measuring how much probability the model gave the correct next token.

Training

D

Data poisoning training data poisoning, poisoning attack

Corrupting a model's training data so it learns hidden behaviors, biases or backdoor triggers.

Safety and governance

Decoder-only model decoder-only transformer, causal language model, GPT-style model

A transformer made only of decoder layers that predicts the next token from the tokens before it.

Model architecture

Dedicated endpoint provisioned throughput, reserved capacity, dedicated deployment

Model capacity reserved for one customer, billed by time instead of per token.

Business and pricing

Diffusion language model diffusion LLM, dLLM, text diffusion model

A model that generates text by refining a whole draft in parallel over several steps, not token by token.

Model architecture

Direct preference optimization (DPO) DPO

A simpler alternative to RLHF that trains directly on preferred-versus-rejected answer pairs.

Training

Distillation knowledge distillation, model distillation

Training a smaller student model to imitate a larger teacher model's outputs.

Training

E

Elo rating (arena leaderboards) arena score, Elo score

A rating system from chess, adapted by crowdsourced leaderboards that rank models from head-to-head votes.

Evaluation and benchmarks

Embedding vector embedding, embedding vector

A list of numbers that represents the meaning of a token, sentence or document as a point in space.

Fundamentals

Embedding model text embedding model, embedder

A model that converts text into vectors so that similar meanings produce nearby vectors.

Building LLM apps

Emergent abilities emergence

Skills that seem to appear suddenly once models pass a certain scale, rather than improving gradually.

Fundamentals

Encoder encoder-only model, bidirectional encoder, BERT-style model

A transformer component that reads a whole input at once, in both directions, to build representations of it.

Model architecture

Epoch training epoch

One complete pass of a training process through the entire training dataset.

Training

EU AI Act Artificial Intelligence Act, Regulation (EU) 2024/1689

The European Union's risk-based law regulating AI systems and general-purpose AI models.

Safety and governance

Eval set golden dataset, evaluation dataset, test set

A curated set of test inputs, with expected outputs or grading criteria, used to measure an LLM application.

Evaluation and benchmarks

Evaluation (evals) evals, LLM evaluation, model evaluation

Systematically measuring how well a model or LLM application performs on the tasks you care about.

Evaluation and benchmarks

F

Feed-forward network (FFN) FFN, MLP layer, MLP block

The per-token neural network inside each transformer layer that transforms what attention gathered.

Model architecture

Few-shot prompting few-shot learning, multi-shot prompting, k-shot prompting

Including a few worked examples in the prompt so the model copies the pattern.

Prompting

Fine-tuning finetuning, model fine-tuning

Further training a pretrained model on a smaller, targeted dataset to change its behavior.

Training

FlashAttention flash attention

A fast, memory-efficient way to compute exact attention by working in small tiles inside fast GPU memory.

Model architecture

FLOPs floating-point operations, FLOP/s

Counts of floating-point arithmetic operations, the standard unit for training and inference compute.

Inference and deployment

Foundation model general-purpose AI model

A large model trained on broad data that can be adapted to many downstream tasks.

Fundamentals

Frontier model frontier AI

One of the most capable models available at a given time, usually from a well-funded lab.

Fundamentals

G

GGUF GGUF format

A single-file model format from the llama.cpp project, widely used to run quantized models locally.

Inference and deployment

GPQA GPQA Diamond, Graduate-Level Google-Proof Q&A

A hard multiple-choice science benchmark written by PhD-level experts and designed to resist web search.

Evaluation and benchmarks

GPTQ

A post-training method that compresses model weights to about 3-4 bits using a small calibration set.

Inference and deployment

GPU (graphics processing unit) graphics card, accelerator

The parallel processor that trains and runs most LLMs; memory size and bandwidth matter most.

Inference and deployment

Greedy decoding greedy search, argmax decoding

Always choosing the single most likely next token, with no randomness.

Inference and deployment

Grounding groundedness

Tying a model's answers to specific, verifiable sources such as retrieved documents, data or search results.

Building LLM apps

Group relative policy optimization (GRPO) GRPO

A reinforcement learning method that scores each answer against a group of answers to the same prompt.

Training

Grouped-query attention (GQA) GQA

An attention variant where groups of query heads share key and value heads to shrink the KV cache.

Model architecture

Guardrails AI guardrails, safety filters

Checks placed around a model to block, filter or correct unsafe, off-topic or malformed inputs and outputs.

Safety and governance

H

Hallucination confabulation, fabrication

When a model states something false or unsupported with the same confidence as a fact.

Fundamentals

HumanEval

An OpenAI coding benchmark of 164 Python problems, each checked by running unit tests.

Evaluation and benchmarks

Hybrid search hybrid retrieval

Combining keyword search and semantic vector search so each covers the other's blind spots.

Building LLM apps

I

In-context learning (ICL) ICL

A model's ability to pick up a new task from instructions or examples in the prompt, without retraining.

Prompting

Inference model inference, LLM inference

Running a trained model to produce output, as opposed to training it.

Inference and deployment

Inference engine inference server, serving engine, serving framework

Software that loads a model and serves it efficiently, handling batching, memory and scheduling.

Inference and deployment

Inference provider model API provider, LLM API provider, model host

A company that runs models on its own hardware and sells access through an API, usually priced per token.

Business and pricing

Instruction tuning instruction fine-tuning, supervised fine-tuning, SFT

Supervised training on instruction-and-response examples so a model learns to follow requests.

Training

J

Jailbreak jailbreaking

A prompt crafted to trick a model into producing content its safety training is meant to refuse.

Safety and governance

K

Knowledge cutoff training cutoff, training data cutoff

The date after which a model's training data stops, so it knows little about later events.

Fundamentals

KV cache key-value cache

Memory that stores attention keys and values for earlier tokens so they aren't recomputed each step.

Inference and deployment

L

Large language model (LLM) LLM, language model

A neural network trained on huge amounts of text to predict and generate language.

Fundamentals

Latency response time, end-to-end latency

How long you wait for a response, from sending a request to receiving the full answer.

Inference and deployment

Layer normalization LayerNorm, layer norm

Rescaling each token's vector to a consistent range inside every layer, which keeps training numerically stable.

Model architecture

Learning rate LR, step size

The setting that controls how big a step the model's weights take on each training update.

Training

llama.cpp

An open-source C/C++ engine for running LLMs efficiently on CPUs, Macs and consumer GPUs.

Inference and deployment

LLM-as-a-judge LLM judge, model-graded evaluation, AI judge

Using a language model to grade or compare other models' outputs against a rubric.

Evaluation and benchmarks

Logits raw scores, unnormalized scores

The raw, unnormalized scores a model assigns to every token in its vocabulary at each step.

Inference and deployment

Logprobs log probabilities, token probabilities

The log probabilities a model assigned to the tokens it generated, exposed by some APIs.

Inference and deployment

Long context long-context model, long-context window

The ability to process very long inputs, from hundreds of thousands to millions of tokens.

Fundamentals

LoRA (low-rank adaptation) low-rank adaptation, LoRA adapter

A cheap fine-tuning method that trains small add-on matrices instead of the whole model.

Training

Lost in the middle

The tendency of models to use information at the start or end of a long input better than the middle.

Evaluation and benchmarks

M

Max tokens max_tokens, max output tokens, output limit

The cap you set on how many tokens a model may generate in one response.

Inference and deployment

Mixture of experts (MoE) MoE, sparse mixture of experts, sparse MoE

An architecture that routes each token to a few specialist sub-networks, so only part of the model runs per token.

Model architecture

MLX Apple MLX

Apple's open-source machine learning framework, optimized for running and fine-tuning models on Apple silicon.

Inference and deployment

MMLU Massive Multitask Language Understanding

A widely cited multiple-choice benchmark covering 57 subjects, from math and law to medicine.

Evaluation and benchmarks

Model card system card

A document describing a model's intended uses, training, evaluation results, limitations and risks.

Safety and governance

Model collapse

Quality decay when models are trained repeatedly on data generated by earlier models.

Training

Model Context Protocol (MCP) MCP

An open standard for connecting AI applications to tools and data sources through a common interface.

Building LLM apps

Model license license, open-weights license, weights license

The legal terms that govern how you may use, modify and distribute a model's weights and outputs.

Business and pricing

Model routing and fallback LLM router, model cascade

Sending each request to the most suitable model, often a cheap one first, with fallbacks for failures.

Building LLM apps

Multi-agent system multi-agent, multi-agent architecture

An application in which several LLM agents with different roles or tools collaborate on a task.

Building LLM apps

Multimodal multimodal model, multimodal LLM, vision-language model

Able to take in or produce more than one kind of data, such as text, images and audio.

Fundamentals

N

Needle in a haystack NIAH, needle test

A long-context test that hides one fact in a long document and asks the model to find it.

Evaluation and benchmarks

Numeric precision (FP16, BF16, FP8, INT8) data type, dtype, bit width

How many bits are used to store each number in a model, trading accuracy for memory and speed.

Inference and deployment

O

Ollama

A free tool that downloads and runs open-weights models locally with simple commands and an API.

Inference and deployment

On-device AI edge AI, local AI, local LLM

Running models directly on a phone, laptop or other local hardware instead of in the cloud.

Inference and deployment

Open Source AI Definition (OSAID) OSAID

The Open Source Initiative's standard for when an AI system can properly be called open source.

Safety and governance

Open weights open-weight model, open model

A model whose trained parameters are published for anyone to download and run.

Fundamentals

Overfitting

When a model learns its training examples too closely and performs worse on new data.

Training

P

Parallelism (data, tensor, pipeline, expert) distributed training, model parallelism, multi-GPU training

Ways of splitting a model's training or serving across many GPUs so it fits in memory and runs faster.

Training

Parameter-efficient fine-tuning (PEFT) PEFT, parameter-efficient tuning

A family of fine-tuning methods that train a small number of new or selected parameters instead of the whole model.

Training

Parameters weights, parameter count

The learned numbers inside a model; their count is the usual measure of model size.

Fundamentals

pass@k pass at k, pass@1

The probability that at least one of k generated attempts at a problem is correct.

Evaluation and benchmarks

Perplexity PPL

A measure of how well a model predicts text; lower means the model is less surprised.

Evaluation and benchmarks

PII (personally identifiable information) personal data, personally identifiable information

Information that can identify a specific person, such as names, emails, phone numbers or ID numbers.

Safety and governance

Positional encoding position embedding, positional embedding

Information added to a transformer so it knows the order of tokens, since attention alone ignores order.

Model architecture

Post-training

Everything done to a pretrained model to turn it into a useful, safer assistant.

Training

Pretraining pre-training

The first, most expensive training stage, where a model learns language by predicting the next token.

Training

Prompt caching context caching, prefix caching

Reusing the processed form of a repeated prompt prefix to cut cost and latency on later requests.

Business and pricing

Prompt chaining chained prompts, LLM chain

Breaking a task into a sequence of prompts where each step's output feeds the next.

Prompting

Prompt engineering prompt design, prompting

Designing and testing the instructions and context you give a model to get reliable results.

Prompting

Prompt injection

An attack that hides instructions in content a model reads, hijacking it to ignore or override its real instructions.

Safety and governance

Prompt template prompt templating, templated prompt

A reusable prompt with placeholders, or the model-specific format that wraps chat messages.

Prompting

Proximal policy optimization (PPO) PPO

A reinforcement learning algorithm that improves a model in small, controlled steps; the classic RLHF optimizer.

Training

Q

QLoRA quantized LoRA

LoRA fine-tuning on top of a 4-bit quantized base model, cutting memory needs dramatically.

Training

Quantization model quantization, weight quantization

Storing a model's numbers at lower precision to cut memory use and speed up inference.

Inference and deployment

Query rewriting query reformulation, query transformation, query expansion

Using a model to reformulate a user's question into better search queries before retrieval.

Building LLM apps

R

Rate limit API rate limit, quota, throttling

A provider-imposed cap on how many requests or tokens you can send in a given time window.

Business and pricing

ReAct (reason + act) ReAct, reason and act

An agent pattern in which a model alternates between reasoning about a task and taking tool actions.

Building LLM apps

Reasoning model thinking model, large reasoning model

An LLM trained to work through a problem in intermediate steps before giving its final answer.

Fundamentals

Red teaming AI red teaming, adversarial testing

Deliberately attacking a model or AI application to find harmful behaviors and security weaknesses before others do.

Safety and governance

Reinforcement learning from human feedback (RLHF) RLHF

Training a model to produce answers people prefer, using a reward model learned from human rankings.

Training

Reinforcement learning with verifiable rewards (RLVR) RLVR, RL with verifiable rewards

Reinforcement learning where rewards come from automatic checks, such as a correct answer or passing tests.

Training

Repetition penalty frequency penalty, presence penalty

A setting that lowers the scores of tokens already used, to discourage loops and repeated phrases.

Inference and deployment

Reranker reranking model, re-ranker

A model that re-scores an initial list of search results to put the most relevant ones first.

Building LLM apps

Residual connection skip connection

A shortcut that adds a layer's input to its output, keeping signals and gradients flowing through deep networks.

Model architecture

Retrieval information retrieval, IR

Finding and returning the most relevant pieces of information from a collection for a given query.

Building LLM apps

Retrieval-augmented generation (RAG) RAG

Fetching relevant documents at query time and adding them to the prompt so the model answers from them.

Building LLM apps

Reward model preference model, RM

A model trained to score responses by how much people, or a rubric, would prefer them.

Training

Role prompting persona prompting, role-play prompting

Telling a model to adopt a role or persona, such as an experienced editor, to shape its responses.

Prompting

Rotary position embedding (RoPE) RoPE, rotary embedding

A widely used way of encoding token positions by rotating query and key vectors by position-dependent angles.

Model architecture

S

Scaling laws neural scaling laws, Chinchilla scaling

Empirical rules that predict how model quality improves as you add parameters, data and compute.

Training

Self-consistency majority voting

Sampling several independent answers and picking the most common one to improve accuracy.

Prompting

Self-hosting self-hosted LLM, on-premises deployment

Running models on hardware you control, in your data center or cloud account, instead of calling an API.

Inference and deployment

Semantic search vector search, dense retrieval, neural search

Search that matches by meaning rather than exact keywords, usually by comparing embeddings.

Building LLM apps

Serverless inference pay-per-token inference, shared endpoint

Calling models on shared infrastructure and paying only for what you use, with no servers to manage.

Business and pricing

Sliding window attention local attention, windowed attention, SWA

Attention limited to a fixed number of recent tokens, keeping cost and memory from growing with length.

Model architecture

Small language model (SLM) SLM, compact model

A language model small enough to run cheaply or on a single device, often under about 10B parameters.

Fundamentals

Softmax softmax function

A function that turns a list of raw scores into probabilities that are positive and sum to one.

Model architecture

Special tokens control tokens, reserved tokens

Reserved tokens that mark structure, such as the start of a message or the end of output.

Fundamentals

Speculative decoding speculative sampling, assisted generation

A speed-up where a small model drafts several tokens and the large model checks them all at once.

Inference and deployment

State space model (SSM) SSM, state-space model

A sequence architecture that carries a fixed-size running state instead of attending over every past token.

Model architecture

Stop sequence stop token, stop string

A string that tells the model to end generation as soon as it produces it.

Inference and deployment

Streaming token streaming, streamed responses

Sending a model's output to the client piece by piece as it is generated, rather than all at once.

Inference and deployment

Structured outputs structured generation, constrained generation

Forcing a model's output to follow a defined format, typically valid JSON matching a schema you supply.

Building LLM apps

SWE-bench SWE-bench Verified

A benchmark that tests whether a model or agent can resolve real GitHub issues in Python projects.

Evaluation and benchmarks

Sycophancy AI sycophancy

A model's tendency to tell users what they want to hear rather than what is accurate.

Safety and governance

Synthetic data model-generated data

Training or test data generated by a model or program rather than collected from people.

Training

System prompt system message, developer message, system instructions

Standing instructions, set by the developer, that shape a model's behavior for an entire conversation.

Prompting

T

Temperature sampling temperature

A sampling setting that controls how random or predictable a model's word choices are.

Inference and deployment

Test-time compute inference-time compute, inference-time scaling, test-time scaling

Extra computation spent while answering, such as longer reasoning, to get better results.

Inference and deployment

Throughput aggregate throughput, system throughput

How much work a system completes per unit of time, such as total tokens per second across all users.

Inference and deployment

Time to first token (TTFT) TTFT

How long a model takes to start responding after receiving a request.

Inference and deployment

Token

The basic unit of text a language model reads and writes, often a word or word fragment.

Fundamentals

Token budget context budget, token allowance

A planned limit on how many tokens a request, user, feature or task may consume.

Business and pricing

Tokenization tokenizer

Splitting text into the tokens a model uses, and mapping each one to an ID number.

Fundamentals

Tokens per second output speed, TPS, tok/s

How fast a model generates output, the standard measure of LLM generation speed.

Inference and deployment

Tool calling function calling, tool use

Letting a model request that your code run a defined function, then using the result in its answer.

Building LLM apps

Top-k sampling top_k

A sampling setting that restricts each choice to the k most likely next tokens.

Inference and deployment

Top-p (nucleus sampling) nucleus sampling, top_p

A sampling setting that limits choices to the smallest set of tokens covering a set share of probability.

Inference and deployment

Transformer transformer architecture

The neural network design behind nearly all modern LLMs, built around the attention mechanism.

Model architecture

Tree of thoughts (ToT) ToT

A technique where a model explores and evaluates multiple branching lines of reasoning, backtracking as needed.

Prompting

V

Vector database vector store, vector index

A database built to store embeddings and quickly find the ones most similar to a query.

Building LLM apps

vLLM

A popular open-source engine for serving LLMs at high throughput on GPUs.

Inference and deployment

VRAM (GPU memory) video RAM, GPU memory

The memory on a GPU, which must hold the model's weights plus working memory to run it.

Inference and deployment

W

Watermarking AI watermarking, AI watermark

Embedding a hidden, detectable signal in AI-generated content so it can later be identified as machine-made.

Safety and governance

Z

Zero data retention (ZDR) ZDR, no data retention

An arrangement in which an AI provider does not store your prompts and outputs after processing them.

Safety and governance

Building with LLMs? Get a vendor-neutral plan.

Tell us what you're building. Within 2 business days you get a model shortlist, an architecture outline and a cost estimate, plus optional introductions to vetted implementation partners.

Free and vendor-neutral: no model provider pays for placement.
Your details go only to partners you agree to.