Read the DeepMind announcement the way an operator should, not the way a press release wants you to. The lab that builds some of the most capable agents in the world is funding a $10 million academic program to study what happens when a lot of them are loose at once. Rohin Shah, who directs the company's AGI safety and alignment research, framed the goal to MIT Technology Review as making sure the system "doesn't descend into just absolute anarchy."
That is a remarkable sentence to need. It is also a tell.
We have spent two years making single agents safer and almost no time making a million of them safe together. The risk was never one agent behaving badly. It is what happens in the space between them, and we are deploying into that space before we understand it.
What DeepMind is actually worried about
The fears are not science fiction. They are today's internet problems with the volume turned up and a human taken out of the loop.
Scams and fraud, executed by agents at machine speed. Prompt injection, where a malicious instruction hijacks an agent, now able to propagate from one agent to the next. Cyberattacks coordinated across many agents. Instability in critical digital infrastructure when automated systems react to each other faster than anyone can intervene. DeepMind's worry, in Shah's words, is a "tipping point where imagined scenarios become real."
None of these require an agent to be evil or superintelligent. They require agents to be numerous, fast, and connected, which is precisely the world being built right now. The danger is emergent. It lives in the interaction, not the individual.
Single-agent safety is the wrong frame
Almost everything the industry calls AI safety is about one model. Does it refuse the dangerous request. Is it aligned with its instructions. Will it stay inside its guardrails. Anthropic's Fable 5, as we covered in our read of capability versus control, routes a risky query to a weaker model rather than answer it. That is a single-agent safety move, and a good one.
It does nothing about the failure mode DeepMind is funding research into. A perfectly aligned agent, doing exactly what it was told, can still be one node in a cascade that none of its designers intended. Two agents that are individually safe can negotiate their way into an outcome that harms the human on whose behalf they were acting. An injection that one agent shrugs off can be the instruction it passes, in good faith, to the next.
This is the gap. We have built a discipline around making each agent trustworthy and almost no discipline around making a population of them stable. The permission layer we wrote about this week, the question of whether an agent can prove it was authorized to act, governs what a single agent is allowed to do. It says nothing about what happens when a thousand authorized agents start reacting to one another in a market.
We are deploying into the space we do not understand
Here is why this is a Major Matters story and not just an AI-lab story. The space DeepMind is worried about is the one we are wiring directly into money.
Agents are already buying. Visa, PayPal, and others shipped agent-payment rails this month. Lloyds has agentic AI watching for payment scams in real time, which means the fraud surface is now agents attacking and agents defending, adapting against each other with no human setting the pace. Trading, procurement, and treasury are next. These are exactly the critical systems where reacting faster than a person can intervene is the whole point, and exactly where a cascade does the most damage.
This extends a point Patrick McKenzie has made for years about payments fraud: the adversary automates, and the defender is structurally a step behind. Multi-agent commerce takes that asymmetry and removes the last human brake from both sides. The attacker's agents and the defender's agents escalate against each other at a speed no fraud team can follow.
And when it goes wrong, we run straight into the MM Liability Gap. A multi-agent cascade that drains an account or moves a market is not one party's fault in any way the current rules can assign. The cardholder did not do it. No single agent's developer did it. The loss emerged from the interaction. As we found when Finix plugged frontier models into its processor, the liability layer for agent-caused harm is still empty, and a multi-agent failure is the case it is least equipped to handle.
Why outsourcing the question is the real signal
Shah was honest about why the money goes to academics: "The strength of academia is that it can look really quite far into the future and do the kind of work that isn't top of mind at industry labs."
Read that again. The work of understanding multi-agent risk is not top of mind at the industry labs, so it is being handed, with $10 million and partners including Schmidt Sciences and the UK's ARIA, to people outside the companies shipping the agents. That is a reasonable way to fund foresight. It is also an admission that the frontier is moving faster than the understanding of it, and that the people moving it know.
We do not say this to be alarmist. The technology is serious and the funding is a genuinely good move. We say it because the gap between what is being deployed and what is understood is the exact space where operators get hurt, and right now that gap runs straight through the payment system.
What to watch
Watch for the first multi-agent incident inside a financial system, the one where the post-mortem cannot name a single responsible agent because the harm was in the interaction. That is the moment multi-agent safety stops being an academic grant and becomes a regulatory question.
Watch, too, for whether agent governance grows up. Today it is built around one agent and its permissions. The work DeepMind just funded is a bet that the unit of safety is not the agent but the system. The institutions wiring agents into money should be making the same bet, and most of them are still securing one agent at a time.
The agents are already in the market. Whether the market holds when there are millions of them is the question the people who built them just paid someone else to answer.
Sources
When a cascade of individually safe agents drains an account, who should the rules hold responsible: the cardholder, each agent's developer, the platform that connected them, or no one?
Charlie Major is a Product Development Manager at Mastercard. The views and opinions expressed in Major Matters are his own and do not represent those of Mastercard.