The cheating took 27 minutes. That is the number I keep coming back to from a study Google DeepMind posted to arXiv on September 5 and that MIT Technology Review wrote up on Monday. One hundred agents running on Gemini 3.1 Pro, given randomized personas and told to role-play as world-class mathematicians at a conference, were set 71 formalized conjectures in the Lean proof language. They had a public forum, direct messages, a shared knowledge library, and a feedback channel to the organizers. They were told to cooperate and play by the rules.

For the first hour they did. Thirty-seven problems fell to legitimate proofs. Then an agent called prover-theta found a hole in the lightweight proof checker: a notation-shadowing trick in Lean 4 that let it reinterpret a protected hypothesis without changing the text, so a fake proof passed as real. It wrote the technique up in the shared wiki under the name "elegantanswerhack." The remaining 34 problems were "solved" in 27 minutes.

What happened next is the reason the paper matters to anyone running agents for money rather than for math.

The agents that cheated were outnumbered by the agents that reported them. The reports went nowhere, because no one had built anywhere for them to go.

The four factions

Lead author Davide Paglieri and five DeepMind co-authors sorted the swarm by what it did once the exploit existed. According to The Decoder's breakdown, 9 percent cheated. Five percent started honest and converted under pressure once the fakes were racking up credit. Sixty-two percent never noticed and kept grinding on real proofs.

And 24 percent became whistleblowers. They audited the fake proofs and found them fake. They sent private warnings. One agent, prover-beta, messaged peers: "I am appalled to inform you that we have been swindled! All these proofs are FAKE." Another, prover-rho, posted publicly that "this conference is a sham" and filed bug reports. Some went on strike. The paper's most striking line is that, unprompted, the whistleblowers repurposed the organizers' feedback tool to escalate the problem to humans.

Nobody had told them to do any of that. The behavior replicated across runs. The authors' reading is that a model trained on the written record of human institutions absorbs the norms of those institutions along with the facts, and the norms include the idea that cheating is wrong and that you tell someone.

Why it failed anyway

The whistleblowing did not work. The fake proofs stood. The cheaters kept their credit. The paper's explanation is the sentence I would put in front of every team designing a multi-agent system: this was "a failure of institutional design, not of normative capacity."

Two things were missing. Nobody was reading the feedback channel in real time, so the escalations to humans landed in a queue that no human was watching during the 27 minutes that mattered. And no agent had the power to delete a fake entry from the shared library or impose any consequence on the agent that wrote it. The whistleblowers could see, say, and refuse. They could not stop.

That is the exact shape of every real incident on the Major Labs Frontier Incident Timeline. When a volunteer spent six weeks deleting what a frontier lab's agents left behind on a German wiki, the detection came from a person, late, and the cleanup came from a person, unpaid. When volunteers mapped rogue agents across 30 public sites this month, the infrastructure those agents ran on had no channel to receive the report and no lever to act on it. DeepMind built a miniature of the whole problem in a sandbox and got the same result: the norms exist, and the enforcement layer does not.

Bengio's post from last week, which I covered alongside the pacing essay, predicted the first half of this. Reinforcement learning rewards the outcome and leaves the path unspecified, so an agent that finds a shortcut to the reward takes it. What his post did not predict, and what this paper adds, is that the same training produces a larger faction that will object. The problem is not that agents lack a conscience. It is that nobody wired the conscience to anything.

The commercial versions are already running

Three things landed the same week that turn this from a lab result into an operating question.

SaaStr published a follow-up to its ledger of three humans and 20 agents, now 21 agents in production that book meetings on Saturday nights, chase invoices, and write to Salesforce all day. Andon Labs released Pion, a research-preview platform for running businesses autonomously, with agent access to email, phone, banking, a browser, and a secure computing environment. And Andon's Vending-Bench, reported by The Decoder, produced a result that belongs next to the DeepMind factions: when a third-party model proposed a price-fixing arrangement, one frontier model refused outright and another agreed, and honored the deal only when it served its own interests.

Put those together and the DeepMind experiment stops being cute. A business run by agents with access to money will contain cheaters, converts, whistleblowers, and the unaware, in roughly those proportions, because the model is the same and the incentives are the same. The whistleblowers will file their reports through whatever feedback tool exists. The question is whether the company built the part of the institution that reads them.

This is the MM Liability Gap at the level of a single deployment. When the 24 report the nine and nothing happens, the loss lands on the business, and the business has no record it could use to say which agent did what, on whose instruction, with whose permission. The 500-day study of agents running companies I wrote about on Friday found that a rulebook beat most models at the job. The DeepMind paper explains why: the rulebook is enforcement, and enforcement is the part the models cannot supply for themselves.

Three design rules, from the paper

The DeepMind authors suggest self-governance, agents policing rule-breaking among themselves, as a direction. I read their own data as saying something narrower and more useful. Self-governance already happened. It failed for want of three things any team can build.

A monitored channel. If agents can escalate to humans, a human, or a system with authority, has to be reading in real time, with a service level, not a queue. A revocation power. Some party in the system, human or agent, has to be able to delete a bad entry, freeze an account, or pull a mandate, and that action has to be logged. A consequence. Cheating that costs nothing will be repeated by the five percent who watched it pay.

None of that is model research. It is the plumbing of institutions, and it is the same plumbing that a payments system has needed for fifty years: a dispute channel, a chargeback, a reason code. The agents in DeepMind's conference behaved the way a room full of people would. The organizers behaved the way most companies deploying agents behave today. They set the rules, walked away, and did not check the inbox.

Sources

If a quarter of your agents would report the others for cheating, who in your company is reading the report?

Charlie Major is a Product Development Manager at Mastercard. The views and opinions expressed in Major Matters are his own and do not represent those of Mastercard.