The vocabulary of modern AI systems, defined precisely

Pull-quote: “Most disagreements about an AI system are two people using the same word for different things and neither noticing.”

Twenty-seven terms carry almost all the weight in a conversation about an AI system, and most teams use at least six of them loosely. That looseness is an engineering problem rather than a vocabulary one: a team that has not agreed what “agent” means cannot agree on what it is building, and a team that thinks fine-tuning adds knowledge will spend three months solving a retrieval problem with the wrong tool.

This is a reference. Each term gets one precise sentence of definition, then one or two sentences on the misunderstanding it causes. The terms are grouped by layer rather than alphabetically, because the layers are how the system is built: text becomes numbers, numbers become a request, a request gets context, context enables action, and action has to be trusted. Read it in order once and the dependencies make sense. After that, jump to whichever layer is currently causing an argument.

Every claim about an external standard or technique cites its primary source. Where a definition is contested in ordinary usage, the version here is the one that makes the engineering decision clearer, and the ambiguity is named rather than hidden.

What you will be able to do

• Define twenty-seven core terms precisely enough to settle a design argument.
• Name the specific misunderstanding attached to each term, and the cost it produces.
• Distinguish the five pairs of terms that get confused most often, including fine-tuning against retrieval.
• Explain to a non-specialist why a larger context window does not remove the need for retrieval.
• Read a model card or an API reference without guessing at what a parameter does.

What do the words for text and numbers mean?

This layer describes how text becomes something a model can compute on. Every cost and limit in the system is denominated here.

Token. A token is the atomic unit a language model reads and generates, usually a fragment of a word rather than a whole word or a single character. What people get wrong: estimating tokens from word counts. Counts vary by language, formatting, and how much code or punctuation is present, so use the model’s own tokeniser rather than a multiplier. Billing, context limits, and rate limits are all denominated in tokens, which makes this the unit that matters commercially.

Tokenisation. Tokenisation is the deterministic process of splitting text into tokens using a fixed vocabulary learned during training, most commonly by a byte pair encoding scheme that merges frequent character sequences into single units (Sennrich, Haddow and Birch, 2016). What people get wrong: assuming it is the same across models. Two models given identical text produce different token counts, so a prompt that fits one model’s window may not fit another’s, and a cost estimate does not transfer between providers.

Embedding. An embedding is a fixed-length list of numbers produced by a model such that inputs with similar meaning land near each other under a distance measure. What people get wrong: two things. Embeddings from different models are not comparable, so re-embedding your corpus is mandatory when you change the embedding model. And proximity means similarity of a general kind, not relevance to a specific question, which is why pure vector search returns plausible documents that answer nothing.

Vector. A vector, here, is the array of floating point numbers an embedding consists of, stored in a database that finds nearest neighbours quickly. What people get wrong: treating “vector database” as a category of intelligence rather than an index. It is an index. Choosing one is an infrastructure decision about scale, filtering, and operations, not a decision about answer quality.

Context window. The context window is the maximum number of tokens a model can consider in one call, counting input and generated output together. What people get wrong: reading it as free space. It is a shared budget, so a long input reduces room for the answer, and retrieval quality typically degrades well before the limit because relevant material gets buried among irrelevant material. A larger window changes what is possible, not what is advisable, which is the subject of the context budget.

What do the words for asking a model something mean?

This layer covers the request and its performance characteristics. These terms are most often used as if they were settings on a machine rather than statistical controls.

Prompt. A prompt is the complete input sent to the model for one call, including instructions, retrieved documents, conversation history, tool schemas, and the user’s question. What people get wrong: equating the prompt with the user’s message. In a production system the user’s text is often under five percent of the prompt, and most quality problems come from the other ninety-five percent.

System prompt. A system prompt is the instruction block placed in a privileged position at the start of a conversation to set role, constraints, and output expectations for every turn. What people get wrong: treating it as a security boundary. It is a strong steer, not an enforcement mechanism, and content arriving later in the context can override it, which is the basis of prompt injection.

Inference. Inference is a single forward run of a trained model to produce output, with no change to the model’s weights. What people get wrong: assuming it is one uniform operation. It has two phases with different cost profiles: prefill, which processes the whole input at once, and decode, which generates one token at a time. That split is why a long prompt with a short answer and a short prompt with a long answer behave very differently.

Sampling. Sampling is the procedure that selects the next token from the probability distribution the model produces, controlled by parameters such as top-k, top-p, and temperature. What people get wrong: believing the model “chooses” a word. The model outputs a distribution over the whole vocabulary and the sampler picks from it, so changing sampling parameters changes behaviour without changing the model.

Temperature. Temperature is a scaling factor applied to the model’s output scores before they become probabilities, where lower values concentrate probability on the most likely tokens and higher values flatten the distribution. What people get wrong: two things. Temperature zero is not a determinism guarantee, because batching and floating point arithmetic on parallel hardware can still produce different outputs for identical inputs. And low temperature does not improve factual accuracy; it makes the model more consistently itself, including consistently wrong.

Latency versus throughput. Latency is how long one request takes, usually split into time to first token and the time between subsequent tokens; throughput is how many tokens the system produces per second across all concurrent requests. What people get wrong: optimising one and reporting the other. Batching raises throughput and worsens per-request latency, so a system tuned for cost efficiency feels slower to every individual user, which is the trade examined in cost and latency engineering.

What do the words for giving a model your data mean?

This layer covers everything between your content and the model’s context. These terms are frequently used as synonyms and are not.

Retrieval. Retrieval is selecting a small set of relevant items from a large corpus in response to a query, by any method: keyword search, vector similarity, metadata filters, SQL, or a graph traversal. What people get wrong: equating retrieval with vector search. Hybrid approaches combining lexical matching with vector similarity usually beat either alone, particularly for identifiers, part numbers, and exact phrases that embeddings handle badly.

RAG. Retrieval-augmented generation is the pattern of retrieving relevant material at query time and placing it in the model’s context, so the answer comes from that material rather than from training. What people get wrong: calling it a feature rather than a pipeline with independently measurable stages. Chunking, retrieval, reranking, and generation each fail in their own way, and a single “RAG accuracy” number tells you nothing about what to fix, as retrieval layer failures sets out.

Reranking. Reranking is a second retrieval stage in which a more expensive model scores each candidate against the query directly, reordering a broad first-stage result set into a precise short list. What people get wrong: skipping it. First-stage retrieval compares precomputed representations of query and document separately, which is fast and approximate; a reranker reads both together, so retrieving twenty candidates and reranking to four usually beats retrieving four.

Grounding. Grounding is the constraint that every factual claim in an answer must be traceable to supplied source material, enforced by instruction, by citation requirements, and by verification after generation. What people get wrong: assuming that supplying documents grounds the answer. Placing sources in the context makes grounding possible; checking that the claims match the sources makes it real.

What do the words for making a model act mean?

This layer is where the vocabulary is newest and the confusion is most expensive, because these terms determine what software gets built.

Agent. An agent is a program in which a model decides which action to take next, executes it through tools, observes the result, and repeats until a stopping condition is met. What people get wrong: applying the word to any system that calls a model. A fixed sequence of model calls is a workflow, however sophisticated; the distinguishing property of an agent is that control flow is chosen at runtime by the model, which what actually makes something an agent examines in detail.

Tool call. A tool call is a single structured request emitted by a model, naming a function and supplying arguments that conform to a schema you declared. What people get wrong: thinking the model executes anything. It emits a name and arguments as text; your code decides whether to run it, runs it, and returns the result. Every security boundary in an agent lives in that gap.

Function calling. Function calling is the model API capability that makes tool calls possible: you send declarations of available functions with your request, and the model can respond with a conforming call instead of prose. What people get wrong: treating “function calling” and “tool call” as competing terms. Function calling is the capability, a tool call is one instance of using it.

Structured output. Structured output is a constrained generation mode in which the response is forced to conform to a supplied schema, typically JSON Schema. What people get wrong: reading conformance as correctness. The guarantee is that the shape is valid, not that the values are true. The schema subset is also restrictive in surprising ways: OpenAI’s implementation requires additionalProperties: false, requires every property to be listed in required, and expresses optionality as a union with null rather than by omission (OpenAI Structured Outputs guide). Getting it reliably right is the subject of structured output from language models.

MCP. The Model Context Protocol is an open standard for connecting AI applications to external systems, so a tool server written once can be used by any compatible client (Model Context Protocol introduction). Its versions are dated strings marking the last backwards-incompatible change, and the current protocol version is 2025-11-25 (MCP versioning). Servers expose three kinds of thing: tools the client may call, resources it may read, and prompt templates it may use. What people get wrong: expecting MCP to solve authorisation and tool design. It standardises the connection, not which tools an agent should have or what it may do with them.

What do the words for changing a model mean?

This layer covers modifying the model itself. These four are the most misapplied in budget conversations, because all four sound like the answer to “the model does not know our business”.

Fine-tuning. Fine-tuning is continued training of an existing model on your own examples, updating weights so behaviour shifts towards the patterns in that data. What people get wrong: expecting it to install knowledge. It is effective for format, tone, structure, and task-specific behaviour, and it is a poor and expensive way to make facts available, because facts change and weights do not. If the requirement is “know our current inventory”, the answer is retrieval.

LoRA. Low-Rank Adaptation freezes the pretrained weights and trains a small pair of injected low-rank matrices instead, so adaptation produces a compact adapter rather than a full copy of the model (Hu et al., 2021). What people get wrong: assuming it is a lesser form of fine-tuning. For most adaptation tasks it reaches comparable quality at a fraction of the training and storage cost, and several adapters can be swapped over one base model.

Distillation. Distillation is training a smaller student model to reproduce the outputs of a larger teacher, transferring behaviour into a cheaper form (Hinton, Vinyals and Dean, 2015). What people get wrong: expecting the student to match the teacher generally. Distillation transfers competence on the distribution it was trained against, so the model performs well on that task and degrades outside it.

Quantisation. Quantisation stores a model’s weights, and sometimes its activations, at lower numeric precision, cutting memory footprint and bandwidth so it runs on smaller hardware. What people get wrong: treating it as free. Quality loss is real and uneven, hitting long reasoning chains and rare cases harder than short common ones, so the only reliable way to size the trade is to run your own evaluation set at each precision.

What do the words for trusting the output mean?

This layer separates a demonstration from a system. All three terms routinely describe activities that do not actually happen.

Hallucination. A hallucination is fluent, confident output that is not supported by the provided sources or by fact. What people get wrong: describing it as a bug to be fixed. Generation is prediction, so unsupported output is a property of the mechanism rather than a defect in one model, and the engineering response is constraint and verification rather than waiting for a model that does not do it.

Evaluation. Evaluation is measuring output quality against a fixed dataset with a defined scoring method, repeated identically whenever anything changes. What people get wrong: calling manual spot-checking evaluation. If the inputs are not fixed and the scorer is not defined, you cannot tell an improvement from noise, and every prompt change becomes an argument about impressions. The harness matters more than the metric, which is the argument in the agent evaluation harness.

Guardrails. Guardrails are deterministic checks running outside the model on its inputs and outputs, blocking, modifying, or escalating when a rule is violated. What people get wrong: writing guardrails as prompt instructions. An instruction is a request to the model; a guardrail is code that runs whether the model complies or not. If clever input can override it, it was never a guardrail.

Where this goes wrong

“We need to fine-tune it on our data.” The symptom is a training budget approved to solve a knowledge problem. The cause is treating fine-tuning and retrieval as interchangeable answers to “the model does not know our business”. The fix is to ask whether the missing thing changes: if it changes, retrieve it; if it is a stable pattern of behaviour or format, fine-tune.

“The context window is huge, so we can put everything in.” The symptom is a slow, expensive system whose answers got worse after the corpus was expanded. The cause is treating window size as a substitute for selection. The fix is to keep retrieving and reranking, and to measure answer quality against the number of documents supplied, which usually peaks well below the limit.

“Set temperature to zero so it is deterministic.” The symptom is a test suite that fails intermittently and a team that does not trust it. The cause is expecting a hardware and batching-dependent computation to be bit-identical. The fix is to assert on properties, schemas, and semantic equivalence rather than on exact strings.

“The output is valid JSON, so the pipeline is correct.” The symptom is well-formed records containing wrong values. The cause is reading schema conformance as a correctness guarantee. The fix is validation after parsing: arithmetic consistency, reference lookups, and cross-checks against records you already hold.

“We have multiple agents.” The symptom is agents passing summaries to each other, detail lost at every hop, and a token bill several times the estimate. The cause is describing one agent with several tools as a multi-agent system, then building the architecture that description implies. The fix is to reserve the term for cases where context isolation between agents is the actual requirement.

Common questions

What is the difference between a token and a word?

A token is a fragment of text from a fixed vocabulary, usually shorter than a word: common words are often a single token, while rare words, names, numbers, and code split into several. Word count is therefore an unreliable proxy for cost or for fitting inside a context window. Count with the tokeniser belonging to the model you are calling, because counts differ between models.

Is RAG the same thing as fine-tuning?

No, and they solve different problems. RAG supplies information at query time by placing retrieved material in the context, so the answer can reflect data that changed this morning. Fine-tuning changes the model’s weights so it behaves differently, which suits format, tone, and task structure rather than facts. Most production systems need retrieval, and only some need fine-tuning.

Does a bigger context window remove the need for retrieval?

No. A larger window raises the ceiling on how much you can supply, but supplying more irrelevant material reliably makes answers worse and always makes them slower and dearer. Retrieval is a selection problem, and selection remains necessary regardless of window size. The practical test is to plot answer quality against the number of documents supplied; it usually peaks well before the limit.

What is the difference between a tool call and MCP?

A tool call is the model emitting a structured request to run a named function with arguments. MCP is an open protocol standardising how a client discovers and connects to servers offering those functions, so a tool implemented once works with any compatible client. You can make tool calls with no MCP anywhere, and MCP does not decide which tools an agent should be given.

What does temperature actually control?

Temperature scales the model’s output scores before they are converted to probabilities, which changes how sharply the sampler favours the most likely next token. Lower values make output more repetitive and predictable; higher values make it more varied. It does not control accuracy, effort, or how carefully the model reasons, and zero does not guarantee identical output across runs.

Next steps

Take this list into your next design review and ask each person to define “agent”, “grounding”, and “evaluation” in one sentence before the discussion starts. The disagreements that surface in those three answers are usually the disagreements the meeting was going to have anyway, in a form that can be settled in five minutes.

Then read what actually makes something an agent if the first definition caused an argument, and the context budget if the window discussion did. When a team is choosing between building the loop and adopting a library, choosing an agent framework applies this vocabulary to that decision. More engineering writing sits in the Signals index.

Zorost keeps a single definition of each of these terms across every platform we operate, which is an operational choice rather than a stylistic one: one term per concept is what lets an evaluation number mean the same thing in two teams. We publish what we test in the open in AI Fieldwork.