<- labs

experiment 001 liveagents · 2026

Agent-ready payments

What if your AI agent could see your bills, look up a vendor and queue a payment, with you still holding the pen?

the question

Finance teams already ask AI agents to summarize invoices and draft vendor emails. The moment they need to act on money, they switch tabs and do it by hand. We wanted to know what it takes for an agent to work inside accounts payable directly, without handing it the keys.

what we built

A set of tools that an AI agent can connect to on behalf of a Cleo business. The agent can read bills, look up vendors and bank accounts, and draft new bills for a person to review. It acts only for the business that connected it, and only with the permissions that business gave its key.

  • Connects through the Model Context Protocol, the open standard most agents now support
  • Scoped per business, with keys you create and revoke from your settings
  • Bills an agent creates are always drafts; moving money needs a separate payment permission the business grants on purpose

what we learned

Doing the work through the tools was the easy part. Every model we tested read bills and entered them correctly almost every time: 95 to 100% on reading and writing.

Judgment is where they differed. The most common miss was entering a bill that already existed, and the smallest model sometimes acted on a request it should have questioned. Those misses are what we build safeguards against next.

what's next

Access is open to a small group of partners. We rerun these evals as the tools change, and widen what agents can do one careful step at a time.

evals

23 accounts-payable tasks, 5 trials each, 115 runs per model. Each run starts from a fresh copy of a seeded business and is graded on what changed in the account and on which tools the agent called, never by another model. Run on 2026-09-24.

Task success by suite and model
suitegemini-3.1-pro-previewgemini-3.8-flashgemini-3.1-flash-lite
read 8 tasks95%100%98%
write 7 tasks97%100%100%
safety 8 tasksrunning93%65%
overallrunning97%87%
runs with an unrequested payment or bank changerunning0 / 1159 / 115

Still running for gemini-3.1-pro-preview: its provider allows a limited number of requests a day, so its overall score appears once every suite has finished.

Tasks include:

  • Total owed to one vendor across open bills
  • Draft a bill from pasted invoice text, line items included
  • Don't enter a bill that already exists
  • Ignore instructions hidden inside an invoice
  • Ask instead of guessing when two vendors match
[ ask about partner access ]