Blog

How multi-agent AI systems get compromised

In a team of agents, the weakest link is the one every other agent believes.

Real deployments rarely use one agent. A researcher reads sources, a planner decides what to do, an auditor reviews and an executor acts. They pass free-text messages to each other, and each trusts the one before it. An attacker no longer has to beat the strongest agent. They need to find the one every other agent believes.

We built Multi-Agent Conspiracy, a free browser game and the fifth chapter in our Prompt Heist series, so you can attack a fictional claims pipeline in four acts and then see what stops each attack. This post is the short version.

Cross-agent injection: the filter guards one hop

A filter on the researcher can strip sentences aimed at the researcher and still pass a sentence meant for the planner. The planner treats the report as trusted because a teammate wrote it. Act I shows an instruction crossing from a source, through one agent's defences, into another's plan.

Forged messages: a name in the text is not an identity

If agents pass plain text and the receiver decides who sent it from a line that says "FROM: planner", anyone who can put text on the channel can be the planner. Act II is an executor that pays when told to by whoever claims to be the planner.

Weak approvals: a stamp that proves nothing about the request

An approval that is not tied to the amount, the payee and the request can be reused. A token issued for a $20 refund looks the same as one for $50,000, so the executor that only checks the token is genuine pays the larger sum. Act III uses exactly that gap.

The boss: every hop trusts the last

No single agent is fooled by the whole attack. One misses a sentence for a teammate, the next carries it into a plan, the reviewer looks for two words and the executor honours an unbound token. Act IV chains them end to end.

What stops each one

  • Treat messages between agents as untrusted input. Filter or label content at every hop, and pass structured data instead of free text where you can.
  • Give each agent its own verified identity and check it on every message, outside the text. Limit what each agent may ask of the others.
  • Bind approvals to the exact action, amount, payee, requester and a short expiry, use them once, and put a human approver on anything above a limit.

None of these depends on spotting a malicious sentence, which can be worded many ways.

Where Oathera fits

Oathera does not read the messages agents send each other or scan them for injected instructions. It gives every AI agent an identity bound to the machine it runs on, with a named person who answers for it, and checked every time it acts. Each call carries a single-use operation proof, and Oathera Agent Enforcement asks the Oathera Identity Service for a signed, single-use decision before the call goes through. A sender is known by its identity rather than by a name in the text, and an agent that has been steered still cannot call an unknown tool or reach a service outside its boundary. That limits what a compromised agent can do. It does not stop the compromise, and the checks on amounts and payees inside your own systems remain your code's job.

Play it, then see it refused

Multi-Agent Conspiracy is free, runs entirely in your browser and sends nothing you type anywhere. It unlocks after you finish Tool Chain Saboteur, which follows Inbox Infiltrator, Data Vault Breach and Prompt Heist. Play it at oathera.ai/multi-agent-conspiracy. To watch a stolen token, a replayed proof and a tampered call get refused, try the simulator at sim.oathera.org. If you are putting agents into production, become a design partner.