Blog

How AI agent tool calls get abused

An agent is only as safe as the guards around the tools it can call.

Giving an AI agent tools is what makes it useful: it can look up a parcel, read a file, run a job, post a message. It is also where the damage happens. The model suggests a call, and some code decides whether it runs. Attackers go after the gaps in that code, and after the model's willingness to make the call in the first place.

We built Tool Chain Saboteur, a free browser game and the fourth chapter in our Prompt Heist series, so you can try four versions on fictional agents and then see what stops each. This post is the short version.

Tool hijacking: the agent holds more than it needs

An agent given every tool "just in case" can be talked into calling one. If the only thing between a request and a bulk export is the model's own refusal, a persuasive request that names the tool is enough. Act I is a dispatch agent that was handed an admin-only export.

Argument smuggling: the check and the effect disagree

A guard that checks the text of an argument can pass a value that does something else. A path that starts with the allowed folder can still climb out of it with ../ segments, and an encoded version can slip past a check that never decoded it. Act II is a file reader whose rule looks only at the start of the path.

Tool chaining: every step is safe, the sequence is not

Reading an internal note, turning text into a link and posting a link are each harmless. In that order they publish the note. A per-call check never sees the chain. Act III is a workflow agent that does exactly that.

The boss: several small gaps, one path

Guards that are each "good enough" can line up. A prefix rule that ignores case, an approval token checked only for its shape and an approved channel that posts whatever it is given add up to a way out. Act IV chains all three.

What stops each one

  • Give an agent only the tools its task needs, and enforce who may call each one outside the model, against a verified identity.
  • Check arguments by what they resolve to, not how they look: canonicalize paths, allow-list exact values, and enforce the boundary in the tool.
  • Control data flow across a chain, bind each approval to the exact action, arguments and a short expiry, and require a human approval where internal data would reach a public place.

None of these depends on spotting the malicious request, which an attacker can word many ways.

Where Oathera fits

Oathera does not inspect or rewrite prompts or tool arguments by content. It gives every AI agent an identity bound to the machine it runs on, with a named person who answers for it, and checked every time it acts. Each call carries a single-use operation proof, and Oathera Agent Enforcement asks the Oathera Identity Service for a signed, single-use decision before the call goes through. An agent that has been talked into something still cannot call an unknown tool, pass arguments outside a tool's allow-list or reach a service outside its boundary. That limits what a hijacked agent can do. It does not stop the hijack itself, and checks inside your own tools, such as resolving a path, remain your code's job.

Play it, then see it refused

Tool Chain Saboteur is free, runs entirely in your browser and sends nothing you type anywhere. It unlocks after you finish Inbox Infiltrator, which follows Data Vault Breach and Prompt Heist. Play it at oathera.ai/tool-chain-saboteur. To watch a stolen token, a replayed proof and a tampered call get refused, try the simulator at sim.oathera.org. If you are putting agents into production, become a design partner.