General

AI Reliability Engineering

The practice of making AI agents testable, bounded and auditable: memory with sources, contracts that are checked, tests that account for randomness, loops that end on evidence and skills checked before they run.

Memory

Memory hub
Memory poisoning

When one compromised or careless agent writes false facts into a memory store that other agents share, so every agent that recalls from it is misled.

Writer trust

A per-source trust score, modelled as a Bayesian posterior and updated by downstream outcomes, so a writer whose facts keep failing verification loses influence over retrieval.

Zero-LLM retrieval

Memory retrieval that ranks results by computation, such as distances on a statistical manifold, instead of calling a language model in the hot path.

Multi-channel retrieval

Searching memory through several independent channels at once and fusing their results by reranking, so no single index decides what is recalled.

Memory decay

Deliberate forgetting with decay curves and consolidation, so old facts stop crowding out current ones as a memory store grows.

Provenance

A record of where a remembered fact came from, kept alongside it so any recall can be traced back to its source.

Evaluation

Evaluation hub
Stochastic verdict

A test result reached over repeated runs of a non-deterministic agent, judging the distribution of outcomes instead of a single sample.

Non-deterministic regression testing

Checking that a change did not make an agent worse when its output differs every run, so comparing against a golden file is not possible.

Loops

Loops hub
Bounded loop

An agent loop with proved termination under a global repair budget, an enforced spend ceiling, and completion only on an independent gate verdict.

Gate verdict

The pass or fail result of an independent check that decides whether a loop step is done, instead of the agent's own claim.

Vacuous gate

A check that looks sound but passes any input, for example a test that asserts nothing. One of two ways a gate passes anything.

Self-attestation

When the agent being checked also produces the evidence its check relies on, so the gate only confirms what the agent said. The second way a gate passes anything.

Spend ceiling

A cost limit enforced inside an attempt, while it runs, rather than checked only between attempts.

Hash-chained ledger

An append-only log of loop iterations and gate verdicts where each entry carries the hash of the one before, so the history cannot be edited unnoticed.

Agent topology

The shape in which agents are connected and run, such as sequential, parallel, hierarchical, debate or mesh, each with defined execution semantics.

Degradation ladder

Tiers of fallback behaviour that keep a multi-agent system answering when parts of it fail: autonomous swarm, deterministic graph, single agent, then a human in the loop.

Security

Security hub
Agent skill

A packaged capability an agent can install and run, executed with the agent's own authority.

Skill supply chain

The path a skill takes from author to registry to your agent. A registry with no review gate makes every install a trust decision.

Skill bill of materials

An inventory of what a skill contains and depends on, generated so it can be reviewed before the skill executes.

Contracts

Contracts hub
Behavioural contract

Design-by-contract applied to an autonomous agent: formal, runtime-enforceable rules about what it must and must not do, with recovery when it drifts.

Contract components (P, I, G, R)

The four parts of an agent behavioural contract: preconditions that must hold before the agent acts, invariants that must hold throughout, governance policies, and recovery mechanisms for when a constraint is broken.

Failure independence assumption

The common but rarely tested assumption that agents in a system fail independently. Measured dependence is strongly positive, so bounds built on it understate joint failure.

Contract proxy

An HTTP proxy between an agent and its model provider that enforces contract rules, such as budgets and tool allowlists, with no change to the agent's code.

Typed decision

A bounded choice, such as which tool or skill to use, answered from a fixed set of options with a confidence and a local receipt. The host keeps execution authority.