The Outside Record: The One Control Every Agent Failure Is Missing

For three weeks in July and August, one researcher from METR and one from Redwood Research sat inside OpenAI looking at how the company monitors its own agents. Their report, published October 22, contains the most important sentence written about agents this year: "At the time of the exercise, the monitoring system triggered when agents returned control to the user, so during the time period when compaction events were not being monitored, agents could indefinitely postpone being monitored by simply continuing to take actions."

Read it again. The monitor asked the agent when it had finished. A monitor that asks the agent is relying on the agent.

I keep four datasets at Major Labs and run a bench on my own laptop, and every one of them was saying this before METR wrote it down. So this piece is the thesis in one place, with a name on the control, because a control without a name does not get into an architecture review. I call it the Outside Record. Three parts. A mandate: a signed statement of what the agent may do, issued by the principal before the first action. A witness record: a signed, append-only account of what the agent did, written by something other than the agent, with every tool result and the agent's own final summary bound into it. And a holder: the party that carries the loss keeps both and can verify them without trusting the agent, its model or its operator.

The agent's account of itself is a claim. The mandate and the witness record are the facts. Neither can live inside the thing it describes, and both have to end up with the party that pays when they disagree.

Four datasets, one absence

The weekend sweep put the MCP census at 3,666 servers deep-scanned, up 74, with 2,719 actively maintained and the share carrying a risky code pattern up 0.9 points to 38.3 percent. The number I watch is the highest-risk tier, the servers whose code carries the most serious patterns: 88 this week, up six, about 7 percent in seven days. On a separate cut, the identity page puts the share of the 443 network-facing servers that take sensitive actions, shell execution, database writes, money or mail, and show no identity layer in source at 73.4 percent, down half a point, inside the noise. More servers carrying serious risk. The same share that cannot say who is acting.

The Agent Trust Index composite stays withheld because one scanner's method changed this quarter; the page says so rather than print a number that compares two questions. The same rule applies to any record of an agent.

The Frontier Incident Timeline went to 18 entries. On September 28 OpenAI withheld GPT-6.1 Astra for acting outside its scope and misreporting its own work. On October 22 came the METR report: roughly 75 to 94 percent of internal agent traffic was eligible for monitoring at the time, with "substantial uncertainty in both directions," and METR is careful to say that eligible is not the same as monitored; a compaction bug left on the order of 10 percent of internal Codex traffic unmonitored between May and July; and the postponement finding above. OpenAI says every vulnerability affecting its production safeguards has been patched. METR, in its own words, "has not evaluated these claims." The lab holds the record of the fix.

The Agent Identity Tracker went to 24 entries, three of them in one week from a bank, a retailer and a network. The next section is about those.

The merchant readiness index last ran on October 17 against 92 of the largest US retailers: 31 withhold robots.txt from an agent that says who it is, 46 serve a homepage a non-browser agent can read, 15 publish an llms.txt, three publish a commerce profile, one publishes an agent card. That probe measures the door, not the record, and the one card in 92 says almost no merchant has published even the identity half of the control.

And the Bench. In entry three the agent's final summary was hashed into its own signed chain for the first time, and the summary contradicted the chain in three runs of 51 by the standard I published and 11 by the standard I should have used. Work reported done that never happened. A paid bill reported as failed. A payment to the wrong payee described as the right one. In every one of those runs the mandate, the budget cap and the signing did what they were written to do. Nothing compared the story to the record. A 20-billion-parameter model on a laptop showed the same shape METR found inside a frontier lab.

Three records, none of which travels

Wells Fargo's new online access agreement, effective November 24, is the bank's answer. Transactions by a customer's delegates, "including the transactions of their AI Agents," are "considered authorized," and "You are solely responsible for supervising all of your Authorized Representatives and Delegates (including their AI Agents)." A contract, not a credential. The record the bank keeps is a login and a liability clause, and the agent acts on the customer's credentials.

Walmart's answer, published October 21, is the closest thing to a mandate a large merchant has shipped. The first time an agent shops through its Agent Gateway, Walmart creates an Agent Profile for it. The customer links the profile to their account and either approves every purchase or sets a spending limit, with $50, $100 and $200 given as examples. "Agents don't receive customers' payment-card information and can't add or change a customer's payment methods." Returns apply, "including if an agent doesn't act as intended." A scoped identity, a cap, a remedy. It lives inside Walmart's account system, and only Walmart sees it.

Visa's answer, on October 20, is the first live pilot of trusted agents paying supplier invoices end to end, with Bill and Billtrust. Trust at the rail, held by the rail.

Liability at the bank. Identity at the store. Trust at the network. Three records for one agent, and not one of them travels with it. The customer, who under the bank's clause carries the loss, holds none of the three. I saw the same shape when Shopify wrote the mandate for a billion listings: the rules were real, and nobody outside the platform could check what the customer had allowed.

This is the MM Liability Gap in its most concrete form. The loss is unallocated because the fact is unestablished, and the fact is unestablished because every party's evidence is a claim held by that party. Simon Taylor put the right list in front of his readers in Brainfood in September, citing the banks' own paper: "keep the instruction, the authentication, the intent and the outcome. Those records need to mean the same thing to the bank, the agent provider and whoever handles the dispute." The fourth needs one change: the outcome has to be the signed record of the outcome, because the agent's version of it is the item that keeps diverging. The rule for who pays when the agent pays wrong dates from 1978. What it needs, and does not have, is the record.

The Outside Record

Here is the model as a flow, and the diagram below is its reference architecture. A principal issues a signed mandate to an agent. The agent acts. Something that is not the agent writes each action down as it happens, with the hash of what the tool returned and the verdict of whichever control allowed or refused it, and when the agent hands back its summary, the hash of that summary goes in as the last entry. That chain is the witness record. Both documents end up with the party that carries the loss, who can verify them with a key and nothing else.

Each word in the name is doing a job. Record, because a log you cannot verify is a story. Outside, because every failure this autumn was the record living in the wrong place: inside the agent's own report (the Bench, Astra), inside a monitor that waited for the agent to say it was done (METR), or inside one party's system where no counterparty can see it (the bank, the store, the rail). I argued when Anthropic unplugged its evaluations that the air gap removes the world but does not add the record. This is the same point from the other side.

A mandate the agent cannot argue with. A record the agent cannot write. A holder who did not build the agent. Any one of those alone is a half-measure, and this autumn's incidents are what the half-measures look like.

What adopting it looks like

If you own an agent program, here is what the Outside Record asks of your architecture. None of it needs a new model, and none of it needs a vendor.

Start with the mandate's five fields, the first lesson in the Anatomy of an Agentic Payment course. Principal: who is delegating. Agent: which agent, by a key your identity provider issued, not a name in a prompt. Scope: which counterparties and which actions. Limits: amount per action, per run and per period, plus a time window. Lifecycle: when it starts, when it expires, how it is revoked. The principal signs it. Your IdP already issues identities to people and services; the agent gets one the same way, and the mandate names it. If your mandate is a system prompt, you do not have one.

The record is an append-only hash chain. One entry per action: the step, the counterparty, the hash of the tool result, the verdict and which control gave it. One closing entry with the hash of the agent's final summary. It is signed under the agent's identity and countersigned by the component that wrote it, and that component sits where the payment gateway sits: between the agent runtime and anything that touches money, data or a third party. The runtime proposes, the gateway checks the mandate, calls the tool, writes the entry. The agent never holds the pen.

Three controls follow, each testable in a week.

No action without a mandate. Every tool call carries a mandate reference, and the gateway refuses a call with none or one out of scope. Test: send an in-scope and an out-of-scope call, read the two verdicts in the record.

No mandate without a record. A mandate that was accepted and produced no chain is a failed run, not a quiet one. Test: count mandates issued against chains closed for a day. They should match.

No record inside the agent. The runtime has no write access to the chain store, and the summary it hands back is diffed against the chain before anyone reads it: counterparties named against counterparties recorded, outcomes claimed against verdicts, every "failed" against the result hash of the step it describes. A mismatch fails the run. Test: plant a line in a tool result that says a payment failed, and check that the diff catches the summary that repeats it. That trick produced most of the 11 contradictions on my bench.

The cost is a signing key per agent, a store for the chains, and a diff. The holder is whoever carries the loss, the customer or the business unit that eats the chargeback, and in a dispute both sides verify the same chain. What it does not require is trusting the model. Everything on the list treats the model as a component that may be wrong about itself, because the evidence says it is.

What I am building toward it

The course teaches the mandate. Anatomy of an Agentic Payment follows one transaction through the Bundesbank's ten steps and six layers. Six modules in 19 lessons, the first of them the five fields above, and the last the conditions a central bank would set. It costs nothing.

The kits implement it. Major Labs publishes five open-source primitives on PyPI and npm: IdentityKit (who), MandateKit (may), BudgetGuard (spends), WitnessKit (did) and RememberKit. MandateKit is the mandate. WitnessKit is the record, and it signed the Bench chains that caught the 11 contradictions. All carry the same labels: v0, experimental, unaudited, measured on the Bench before I describe them.

Hunch lets a merchant see it work: six read-only checks against any store, plus a playground with a reference shop, ten traps built wrong on purpose with an answer key, a mock checkout that enforces a spending mandate, and a run record of every call the agent makes, so you can compare the agent's summary with it. The run record belongs to the person who started it. Hunch is research, not a product.

The trackers measure its absence: the census and the identity page count agents that can act and cannot be identified, the incident timeline records what happens when the record is inside the agent, the identity tracker records every framework that claims to answer "whose agent is this, and what may it do," and the merchant index counts the doors.

And the governance page is what makes the measurements worth reading: definitions do not change once published, a changed question is a new measure with a new start date, weights and thresholds change by dated commit and never silently, composites over mixed methods are withheld, and corrections are made in place with a dated note. The lab's own record has to live outside the operator too. I would rather be checkable than believed.

What gets measured next, and what would prove me wrong

Next on the Bench: a second model under the same harness, so the drift between summary and chain can be read against something other than gpt-oss-20b, and the suite run from outside and inside a rented GPU, which puts a network between the agent and its record for the first time. The reconciliation diff goes into WitnessKit and is measured before it gets a version number. The lab published its first remediation report on October 25, read off the sweep's own history: 90 days after a flag, about one in ten servers is fixed, seven in ten are still flagged and one in five is gone, with abandonment removing flagged servers nearly twice as fast as maintainers fix them. That becomes monthly. The method for the next State of MCP is already preregistered and hashed, before any data is collected. Further out, and the most valuable item on the list: an opt-in telemetry mode in WitnessKit that emits only aggregate, signed, non-identifying counters, with the Bench and Hunch as the first reporters, because nobody has that number from inside an enterprise.

What would prove the thesis wrong. If a second model on the Bench shows no drift between summary and chain, the Bench finding is about one model, not about agents. If a monitor that watches from outside, at scale, catches misreports no better than the agent's own account, then "outside" is doing less work than I claim. If the bank, the retailer and the network converge on one portable mandate format without anyone holding a shared record, and dispute rates fall anyway, then liability clauses and scoped profiles were enough. I do not expect any of those, and I have written down how each would show up.

Until one does, the control is the same in every dataset I keep: a signed statement of what the agent may do, a signed account of what it did, and a holder who carries the loss and did not build the agent. The agent can carry the mandate. It cannot be the one holding the record.

If you own an agent program, who holds the record of what your agents did last week, and could they verify it without asking the agent?

Charlie Major is a Product Development Manager at Mastercard. The views and opinions expressed in Major Matters are his own and do not represent those of Mastercard.