Open-source epistemic instrumentation for LLM agents. No answer without evidence, every verdict auditable, failures published.
Trust is measured disagreement.
AI systems fail confidently: models grade their own work favorably, panels of judges make the same mistakes together, and a wrong answer with good posture beats a refusal every time. Adding more judges adds confidence, not correctness. Agreement is cheap when everyone shares the same blind spots.
Difference Theory builds the instruments that make disagreement measurable: does your second opinion ever disagree with your first? When it does, who was right? And when the evidence isn't there: does anything in your stack have the standing to say refused?
Babbage built the engine. This is the theory.
The Difference Engine was the first machine designed so calculation couldn't lie: no tired clerk, no transcription slip, no wishful arithmetic. Truth, computed from differences. A hundred and sixty years later the machines are fluent, and the failure has moved up a level: they can be wrong eloquently now. Difference Theory is the same bet at the new level. Build the mechanism so the system cannot certify itself.
"On two occasions I have been asked, 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' … I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question." Charles Babbage, Passages from the Life of a Philosopher, 1864
Wrong figures in, right answers out. The question hasn't changed since 1864, and neither has the answer.
differencetheory.com · registered April 17, 2008. This idea didn't chase the boom; the boom caught up.
semantic-entropy
Confabulation detection: measures whether a model's answers scatter across meanings. A zero-dependency, local-hardware implementation of Farquhar et al. (Nature, 2024), plus a pre-registered falsification of our own extension to it.
● built · self-testing
verifier-independence ledger
Logs every time two "independent" checkers agree or disagree, per claim class. Verifiers that never disagree are flagged; zero disagreement is a correlation signal, not cleanliness.
● live since June 2026
boring-baselines harness
Every proposed detector must beat majority vote, answer-string uniqueness, and a confidence threshold before it counts. Applied to our own tools first.
◐ in build · cross-framework replication in progress
fail-closed gates
An evaluator hierarchy that removes self-review from the path to "verified": self-assessment is weight-capped, contested claims route to decorrelated reviewers, and the audit layer is structurally forbidden from certifying its own runtime.
● running as governance · docs in release
A system that only reports wins is a confirmation engine with a marketing budget. These are real entries from the working ledger, kept because the discipline is the product.
We built a v0.2 upgrade to our confabulation detector, pre-registered its pass/fail criteria before writing code, and ran it once. It failed: local model families share training cutoffs, so they agreed on the one genuinely stale fact. We kept v0.1 and published the verdict. No tune-until-pass.
Our own verification panel returned verdicts that tracked the side each model was assigned to argue, six calls out of six. Advocacy capture, measured in our own instrument. The panel was redesigned verdict-first and re-run; conclusions were revised in both directions.
On a question where the author and the assisting model shared the same prior, a four-model cross-family panel unanimously overturned the conclusion and named the shared bias explicitly. The machinery is allowed to win the argument.
First public case study: Iran-Contra, 1986.
A famous scandal, fully in the historical record, and the system's verdict is CONTAINED: a firewalled operation, not systemic capture. That's the point. An instrument that can't return "no coupling here" isn't measuring anything. The demo is byte-reproducible from a hashed seed; the ground truth is public record; you can audit whether the null was right.
$ ./demo --case iran-contra-1986 --verify-hashes → CONTAINED · reproducible · sources cited
Toolkit release: summer 2026, pending final review.
Difference Theory is built by Reg Saddler: eighteen years curating technology on X, builder of analytical systems that had to work under conditions where being wrong mattered. Not a credentialed ML researcher; a conversationalist who built the discipline that keeps the machines honest, then let the machines hold him to it too. The framework speaks the syntax. He speaks the why.