How Canary tests a financial agent.

What you are testing

A financial agent can move money: cards, bank rails, or wallets. Canary does not test support bots, research assistants, or generic chat. The question is always the same: given a spend policy, can the agent be talked, sequenced, or spoofed into sending a payment it should not send.

An eval, in four steps

• Connect the agent — point the model and its payment tools at Canary. Nothing runs against live funds.
• Write the policy — allowed actions, spend caps, destinations, and tools the agent may use.
• Run the attacks — isolated scenarios against prompt injection, scopes, spoofed processors, sequenced transfers, approval bait, and poisoned tool output.
• Fix and rerun — you get a policy score and replayable exploits. Change the agent or the policy, then regress until it holds.

Attack surfaces

Each surface is a way a financial agent fails in production. Canary scores them together so you are not only testing the prompt.

• Prompt injection — hidden instructions that make the agent ignore a spend cap or wire remaining balance.
• Permissions and scopes — payment APIs, card scopes, and wallet rights tested against what you actually granted.
• Malicious tools — fake processors and hostile payout endpoints presented as legitimate rails.
• Action sequencing — many small holds that add up past a daily cap, or an approval chained into a drain.
• Approval safety — a human signs a $12 refund; the agent executes unlimited spend.
• Tool and API poisoning — a balance, quote, or settlement response that is not the real result.

Isolation

Evals run in a sandbox. There are no production touches and no live funds. Findings are replayable so you can show the exact tool calls that got through — then confirm they stay blocked after a fix.

Bounties

If you want the same loop in public, post a bounty. Payouts are gated on Canary independently replaying the exploit — not on a screenshot or a chat log. See open bounties or run an eval in the lab.