Independent research · AI Reliability Engineering

Trust,
and verify.

Open-source tools and arXiv preprints that make AI agents show their work: memory with a source, decisions with a receipt, contracts that hold, tests that account for randomness, and skills checked before they run.

10 arXiv preprints13 open-source toolsLocal-first · runs on your machine

Papers: Bounded Loops, Agent Behavioral Contracts II, SuperLocalMemory 4.0, Qualixar OS, SuperLocalMemory V3.3 — The Living Brain, SuperLocalMemory V3, AgentAssay, Supply Chain Security for Agentic AI Skills, Agent Behavioral Contracts, SuperLocalMemory — Bayesian Trust Defense.

Fig. 1

A loop stops when the gate says so, not when the agent does.

One bounded loop, replayed. The agent writes a patch, an independent gate runs the tests, and only a passing verdict ends the run.

retry · budget 5 iterationsagentwrites the patchgateruns the test suitedoneterminal=DONE
Bounded Loops · arXiv:2609.27871iteration 0 / 5

$ bl run refactor-auth --gate tests

iter 1 · agent patch written

iter 1 · gate FAIL · 41/44 tests

iter 2 · agent patch revised

iter 2 · gate FAIL · 43/44 tests

iter 3 · agent patch revised

iter 3 · gate PASS · 44/44 tests

terminal=DONE · ledger written

Illustrative replay. The live figure will play back a recorded run with a link to its ledger.

7 flagship tools

Each one catches a different way agents fail.

Every chapter states what the tool claims and links to the paper and code that back it. Where a tool has published limits, they sit next to the claim.

Memory · SuperLocalMemory

Agents forget. Shared memory gets poisoned.

A governed, local-first memory system for agents. Every recall points back to where it came from, and a writer whose facts keep failing verification loses influence over what gets recalled.

  • 70.4%on the LoCoMo benchmark with no LLM in the loop (V3.3, Mode A) arXiv:2604.04514
  • 7retrieval channels, including semantic, keyword, entity graph, temporal and spreading activation arXiv:2604.04514
  • 72%trust degradation for sleeper attacks, from Bayesian trust scoring arXiv:2603.02240
  • 2,199/2,200scoped component properties upheld across eleven fault-injection scenarios, each run 200 times arXiv:2608.08253

Verify it

LicenceAGPL-3.0
Local: memory stays on this machineLocal memoryOn this machineRollback: DNS flipStaging: Postgres 17Skip tests to deployAgent AAgent Bby Agent Aproject scopereceipt issuedsemantickeywordentity graphtemporalassociativehow do we roll back?Fused andrerankedSupported answer, with its receipthow to roll back?Rollback: DNS flipfrom Agent Aproject scopeNothing relevant storedzxqv plorth wimbleThat isn't in SuperLocalMemory.Memory hooks only inject supported answers.Writer trusttrust holdstrust dropsLow trust:not recalledClaude CodeCodexCursorCloudMode A:no model call
Illustration with example data: write, recall with a receipt, trust decay, local-only mode

Orchestration · Qualixar OS

One runtime for every agent framework.

Qualixar OS designs a team for each task, routes every step to a model, has judges check the output, and steps down a tier when parts fail instead of going silent.

  • 12multi-agent topologies with formal execution semantics, including grid, forest, mesh and maker arXiv:2604.06392
  • 2,821test cases across 217 event types and 8 quality modules arXiv:2604.06392
  • 4degradation tiers: autonomous swarm, deterministic graph, single agent, human in the loop (README)

Verify it

LicenceFSL-1.1-ALv2
Degrade a tier, keep answeringReview PRs forsecurity issuesMemoryForgeEst. cost $0.05AnalystReviewerWriterLeadTask decompositioncostqualitylatencyRouterlearns from resultsLocal modelFast modelStrong modelTeamoutputJudgesreach consensusRejectedForge redesignsmax 5 roundsApprovedFiles on diskAutonomous swarmDeterministic graphSingle agentHuman in the loopfailedfailedStill answeringFailures move work down one tier at a time.
Illustration of one task through Forge, routing, judges and the degradation ladder

Coordination · SLM Mesh

Parallel coding sessions that don't collide.

SLM Mesh lets AI coding sessions on one machine find each other, message, lock files and share state through a broker on localhost. Nothing leaves the machine.

  • 8MCP tools for discovery, messaging, file locks and shared state (README)
  • 10 mindefault lock timeout, so a crashed session never holds a file forever (README)
  • 127.0.0.1the only address the broker binds to, with a random 32-byte token per session (README)

Verify it

LicenceElastic-2.0
Shared state, visible to every peerLocal broker127.0.0.1Claude CodeCursorCodex3 peers foundfixing theDB raceDB schemachangeddb.tslockwaitsShared statedb_fix: doneAll peers read the same state
Illustration of three sessions coordinating through the local broker

Decisions · Jev Decision Layer

Give your agent a bounded choice. Get a typed answer and a local receipt.

An open-source decision layer for agents: typed task, tool, skill and review routing, local policy gates, and an auditable receipt for every decision.

Published limits

  • Confidence and probabilities are advisory; thresholds are uncalibrated defaults.
  • Synthetic self-tests check local gates, not decision-model accuracy or end-to-end savings.
  • Host adapters have different verification evidence. Review the host guide before adopting.

Verify it

LicenceMIT
Host agentLocal layerDecision modelNext tool?greprevieweditclosed listHost decidesand asks youQuestionpick 1 of 3Policy gateIn scopeNo secretsValid optionReceipt · localchoice: reviewgate: passedhost: decidesTyped answerreviewconfidence 0.82Type: optionAdvice onlyExample values
Illustration of one decision; values are examples

Contracts · AgentAssert

Prompts express intent. Contracts enforce it.

Design-by-contract for autonomous agents. Preconditions, invariants, governance policies and recovery are stated formally and checked at runtime, so a violation is caught at the step that caused it.

  • 1,980sessions across 7 models from 6 vendors arXiv:2602.22302
  • 88–100%hard-constraint compliance, with drift bounded and under 10 ms per action arXiv:2602.22302
  • 90%of missions where one instance failed, a second instance of the same model failed too, over 18,000 code-scored missions arXiv:2608.12895
Contract (example)Pre: input validHard: no PII outSoft: tone okallowed by the contractStep 1Step 2Step 3Step 4Step 5Drift risingHard breach at step 4Two agents, same modelillustrative missionsAgent AAgent BThey fail on the same missionsJoint reliability floorno independence assumedPaper II · arXiv:2608.12895
Illustration of a runtime contract check

Testing · AgentAssay

You can't diff an agent's output against a golden file.

The output legitimately changes every run, so AgentAssay tests the distribution instead of one sample: stochastic verdicts over repeated runs, reached in a fraction of the tokens a naive repeat-and-compare would spend.

  • 78%fewer trials with sequential testing (SPRT), and no extra tokens when testing traces you already have arXiv:2603.02601
  • 86%detection power from behavioural fingerprinting, where binary testing has 0% arXiv:2603.02601
  • 3verdicts: pass, fail, or not enough evidence yet

Verify it

LicenceAGPL-3.0
Repeated runs (simulated)6 runs · 5 passed14 runs · 12 passed24 runs · 22 passedRegression flagged: v2 vs v1Required: 0.7000.51pass ratev1v2PASSINCONCLUSIVEFAILRuns used6 of 100 planned14 of 100 planned24 of 100 planned76% fewer runs than the fixed plan
Illustration of a stochastic verdict; counts are simulated

Security · SkillFortify

A skill runs with your agent's full privilege. Check it first.

Static analysis, provenance verification and a bill of materials for agent skills, so a skill can be judged before it executes.

  • 96.15%F1 on its detection experiments, re-measured on the released v0.6.0 arXiv:2603.00195
  • 5theorems with full proofs arXiv:2603.00195
  • v2The revision publishes its own negative result: information-flow analysis added no detections over pattern matching on this corpus

Verify it

LicenceElastic-2.0
Claude skillMCP serverLangChain22 in alldeploy-helperExample skillParsed to one modelsame checks for every formatNetwork call · any hostReads secret filesProvenance: unsignedBOM: 3 deps, 1 unsignedVerdict: UNSAFEreported before it runs
Illustration with sample findings

The rest of the toolkit

6 more tools, same rules.

Each one is open source, installable today and linked to its code. Bounded Loops has its own paper.

Loops

Bounded Loops

Agent loops and graphs that end on an independent gate, with an append-only ledger of every iteration.

On a 69-loop catalogue its instrument found 47 vacuous gates in shipped, reviewed code.

Gateway

SLM MCP Hub

One hub process in front of every MCP server and every AI client, with federated tool discovery, a shared cache and cost telemetry.

CodeAGPL-3.0

Runtime

Agent Amplifier

Deterministic hooks for coding agents: effort routing, goal anchoring, convergence detection and token budgeting.

CodeAGPL-3.0

Contracts

AgentAssert Type-C

Behavioural contracts with no agent code changes: an HTTP proxy that enforces budgets, latency, tool allowlists, topic bounds and PII filters.

CodeMIT

Workbench

ModStage

Run Codex, Grok, Gemini, Kimi, Qwen Code, OpenCode and Claude subagents side by side inside Claude Code: watch, steer and approve every step.

CodeMIT

Interfaces

touchless-mac

Control your Mac with your hand through the built-in camera: point, pinch to click, thumb and ring finger to scroll. Fully local, written in Rust.

CodeMIT

Papers · 10 preprints

The research behind the tools.

All are preprints on arXiv, and the titles here match arXiv exactly. Open the full research page.

arXiv:2609.278712026-08-20cs.SEpreprintBounded Loops

Bounded Loops: Pre-Run Spend Bounds, Proved Termination, and Verified Completion for Agent Harnesses

The instrument found 47 vacuous gates — checks satisfied by the absence of the thing they check — in shipped, reviewed code

Summary

In mainstream agent frameworks a step ends when the agent's own output says it has finished. This paper states what a harness must instead guarantee — that it finishes, does not drift, and does not overspend — and proves each one: termination in closed form under a global repair budget, no DONE without a gate verdict in an append-only hash-chained ledger, and a spend ceiling enforced inside an attempt rather than between attempts. It then builds the instrument that measures whether a harness delivers it, characterising two ways a sound-looking check passes anything: vacuity and self-attestation. Engine, catalogue and corpus are Apache-2.0.

Varun Pratap Bhardwaj, Garima Singh, Arun Pratap Bhardwaj

arXiv:2608.128952026-08-13cs.AIpreprintAgentAssert

Agent Behavioral Contracts II: Certifying Compositional Reliability Without Assuming Independence

Two instances of the same model failed together on 90% of missions where either failed — independence is not a safe assumption

Summary

Multi-agent reliability analysis routinely assumes component failures are conditionally independent, and almost never tests it. This paper tests it and finds strong positive dependence, which inflates joint failure rates well beyond what independence-based bounds predict. It replaces the assumption with a certificate that assumes no dependence structure at all — a linear program over the joint distribution, bounded by a Bonferroni–Clopper–Pearson box — and proves that fitting a dependence model instead degrades coverage as sample size grows.

Varun Pratap Bhardwaj, Garima Singh, Arun Pratap Bhardwaj

arXiv:2608.082532026-08-08cs.AIpreprintSuperLocalMemory

SuperLocalMemory 4.0: The Governed Memory Operating System for AI Agents

Governed memory with scoped component evidence and disclosed negative results

Summary

The revised preprint studies governed local-first memory and distinguishes implementation from verified effectiveness. Component evidence is protocol-scoped; current package behavior requires separate checks.

Varun Pratap Bhardwaj, Garima Singh, Arun Pratap Bhardwaj

arXiv:2604.063922026-04-07cs.AIpreprintQualixar OS

Qualixar OS: A Universal Operating System for AI Agent Orchestration

One substrate for agent orchestration: 12 topologies, Forge team design and three-layer model routing

Summary

Every agent framework reinvents orchestration, and none of them interoperate. Qualixar OS defines a universal substrate: execution semantics for 12 multi-agent topologies, Forge for designing agent teams, three-layer model routing with a Bayesian POMDP, and a consensus-based judge pipeline.

Varun Pratap Bhardwaj

arXiv:2604.045142026-04-06cs.AIpreprintSuperLocalMemory

SuperLocalMemory V3.3: The Living Brain -- Biologically-Inspired Forgetting, Cognitive Quantization, and Multi-Channel Retrieval for Zero-LLM Agent Memory Systems

Seven retrieval channels and zero LLM calls in the hot path; 70.4% on LoCoMo in zero-LLM mode

Summary

A memory system that never forgets degrades: old facts crowd out current ones and retrieval quality falls. V3.3 borrows the brain's answer — decay curves, consolidation, and quantized cognitive states — and retrieves across six independent channels fused by reranking, without a single LLM call in the hot path.

Varun Pratap Bhardwaj

arXiv:2603.145882026-03-15cs.AIpreprintSuperLocalMemory

SuperLocalMemory V3: Information-Geometric Foundations for Zero-LLM Enterprise Agent Memory

75% LoCoMo accuracy without cloud calls, from a four-channel retrieval architecture

Summary

Retrieval quality is usually bought with LLM calls. V3 buys it with geometry instead: memories live on a statistical manifold where relevance is a distance, so ranking is computed rather than generated. The result is a benchmark score competitive with cloud systems, produced entirely on-device.

Varun Pratap Bhardwaj

arXiv:2603.026012026-03-03cs.AIpreprintAgentAssay

AgentAssay: Token-Efficient Regression Testing for Non-Deterministic AI Agent Workflows

Sequential testing needs 78% fewer trials; trace-first analysis of existing traces adds no token cost

Summary

You cannot diff an agent's output against a golden file, because the output legitimately differs every run. AgentAssay tests the distribution instead of the sample: stochastic verdicts over repeated runs, with a sampling strategy that reaches a decision in a fraction of the tokens a naive repeat-and-compare would burn.

Varun Pratap Bhardwaj

arXiv:2603.001952026-02-27cs.CRpreprintSkillFortify

Formal Analysis and Supply Chain Security for Agentic AI Skills

96.15% F1, five theorems with full proofs, and a published negative result on information-flow analysis

Summary

An agent skill is arbitrary code that runs with the agent's own authority, installed from a registry with no review gate. This paper gives the threat model a formal treatment, then builds static analysis, provenance verification and SBOM generation across 22 agent frameworks so a skill can be judged before it executes.

Varun Pratap Bhardwaj

arXiv:2602.223022026-02-25cs.AIpreprintAgentAssert

Agent Behavioral Contracts: Formal Specification and Runtime Enforcement for Reliable Autonomous AI Agents

Design-by-contract for agents — invariants that hold on every execution path. 1,980 sessions across 7 models

Summary

Prompts express intent; they do not enforce it. This paper adapts design-by-contract to autonomous agents: preconditions, invariants, governance policies and recovery mechanisms stated formally, then checked at runtime, so a violation is caught at the step that caused it rather than three tool calls downstream.

Varun Pratap Bhardwaj

arXiv:2603.022402026-02-17cs.AIpreprintSuperLocalMemory

SuperLocalMemory: Privacy-Preserving Multi-Agent Memory with Bayesian Trust Defense Against Memory Poisoning

Shared memory is an attack surface — Bayesian trust scoring that degrades a poisoned writer instead of trusting it

Summary

When several agents write to one memory store, a single compromised agent can poison what every other agent later recalls. This paper models writer trust as a Bayesian posterior updated by downstream outcomes, so a source that keeps producing facts which fail verification loses influence over retrieval — without any of the data leaving the machine.

Varun Pratap Bhardwaj

§05 / Dispatch

Join the AI Reliability Engineering movement.

New papers, new releases and field notes from building agents. No fluff.

No spam. Unsubscribe in one click.