AI and AI agent glossary

The repeating cycle your agent runs through: look at the current state, figure out what to do, call a tool, check the result. It loops until the task is done.

A full record of everything your agent did in one run: every LLM call, tool invocation, reasoning step, and state change. Essential for debugging and observability.

Reusable ways to structure agent systems. Covers routing, parallelisation, orchestration, and evaluation-driven development.

A system that uses a language model to decide what to do next on its own. It perceives, reasons, acts, then observes the result, and keeps going until the job is done.

The connection that lets one piece of software ask another for something, without a person in the middle. When a vendor says their agent "integrates with" your systems, an API is usually how.

The password one tool uses to prove to another that it is allowed in. It is issued to a person rather than to an organisation, and billed by usage rather than as a subscription, so someone has to own cancelling it later.

A standard test the industry uses to compare models. Benchmarks run on clean, curated data, so a score is a ceiling rather than a forecast of how a model will do on your work.

A prompting technique where you get the model to show its working step by step before giving an answer. It noticeably improves accuracy on harder tasks.

A pattern that stops sending requests to a failing service after repeated errors, giving it time to recover. Stops one failure from taking down everything else.

The maximum number of tokens an LLM can handle in one request, covering both your input and its output. Current models range from a few thousand tokens to 1M+ (some open-weight models reach 10M).

Directed Acyclic Graph. A graph where edges have direction and there are no cycles. In agent planning, nodes are steps and edges mean "this depends on that".

A queue where messages land after all retry attempts fail. You can inspect them later and replay manually once you've fixed the issue.

Numbers that represent what text means. Similar text gets similar numbers, which lets you do semantic search and clustering instead of just keyword matching.

Your agent's ability to spot when something fails, work out why, and try a different approach. You need this for anything running in production.

Measuring how well your agent performs using metrics like task completion rate, accuracy, cost, and latency. You can't improve what you don't measure.

You include a handful of input-output examples in your prompt so the model learns the pattern you want. Great for controlling format and behaviour.

Training a pre-trained model further on your own examples so it behaves the way you need - its tone, its format, the way it handles your kind of task. It changes behaviour rather than knowledge: for facts that change, give the model your documents instead.

One of the large, general-purpose models that everything else in AI is built on top of - trained once, at enormous cost, by a handful of organisations, then adapted by everyone else. When a vendor says they "use AI", this is almost always the layer they are renting rather than building.

Industry shorthand for whichever handful of models currently sit at the cutting edge of capability. It is not a fixed list - the membership changes every few months.

A feature where LLMs output structured JSON that matches a function schema you define. This is how you reliably connect models to your APIs and tools.

Tying LLM outputs back to real sources of truth like documents, databases, or tool results. This is how you reduce hallucinations.

Safety checks that keep your agent within acceptable bounds. Think input validation, output filtering, approval gates before risky actions, and resource limits.

When an LLM confidently makes something up. The output sounds right but is factually wrong. You need grounding and verification to catch this in production.

A pattern where your agent pauses at critical decision points and asks a human for approval before continuing. Gives you a safety net without killing autonomy.

How long the AI takes to answer. It matters most wherever somebody is waiting - a customer mid-chat, a fraud check at the checkout - and barely at all for work that runs overnight.

Model Context Protocol. An open standard that gives AI models a universal way to connect to external data sources and tools.

How agents hold onto information between interactions. Short-term memory is the conversation context, working memory is the current task state, and long-term memory is persistent storage.

A cost-saving strategy where you start with the cheapest model and only escalate to a pricier one if the quality isn't good enough.

Multiple specialised agents working together on a problem, each handling a different domain or capability. You need orchestration patterns to wire them up.

The coordinating agent that hands out tasks to specialised sub-agents, collects their outputs, and stitches the results into one coherent response.

Caching the static parts of your prompt (system prompts, tool definitions) so the API doesn't reprocess them every call. Can cut costs by up to 90% for repeated prefixes.

Designing and refining prompts to get the behaviour you want from an LLM. Covers few-shot examples, system prompts, structured templates, and more.

An attack where malicious input tricks the LLM into ignoring its instructions and doing what the attacker wants instead. A serious security risk for any agent.

Retrieval-Augmented Generation. You fetch relevant documents from a knowledge base and stuff them into the prompt so the LLM has real facts to work with.

A prompting pattern where the model thinks out loud before each action. It alternates between reasoning steps and tool calls, which helps it make better decisions.

Service level agreement. The uptime, response times and accuracy a vendor commits to in writing, and what happens when they are missed. An accuracy number is only worth as much as the test behind it - ask what task it was measured on, and whose data.

One unit of work in a trace. Each span has a name, start time, duration, and metadata. In agent tracing, spans cover LLM calls, tool executions, and loop iterations, nested in a hierarchy.

Getting LLM output token by token as it's generated instead of waiting for the full response. Makes your UI feel faster and lets you show progress in real time.

Getting the LLM to return output in a specific format, usually JSON matching a schema. This makes parsing reliable so your downstream code doesn't break.

The instruction you give the LLM to set its role, behaviour, constraints, and available tools. It persists across the conversation and shapes every response.

Controls how random the LLM's output is. Low values (0 to 0.3) give you focused, deterministic answers. Higher values (0.7 to 1.0) add more creativity and variation.

A cap on the total tokens your agent can use in one run, counting both input and output across all loop iterations. Keeps your costs from blowing out.

The chunks LLMs break text into. One token is roughly 3/4 of an English word. Your costs and context limits are both measured in tokens.

How an LLM calls external functions. The model outputs a structured request with a tool name and arguments, your system runs it, and the result goes back to the model.

A JSON object describing a tool's name, purpose, and parameters. The LLM reads this to know when to call the tool and what arguments to pass.

A database built for storing and searching embeddings. You use it to find similar documents fast, even across millions of records.

Prompting without any examples. You rely on a clear instruction and the model's pre-trained knowledge to get the output you need.