Hold

Inspiration

Business email compromise is one of the costliest kinds of fraud there is and yet no one seems to know about it. The FBI’s 2025 report counts nearly $3 billion in losses from nearly 25,000 complaints. With AI paving the way for more security threats, this number is only going to rise in the coming years. Typically, someone pretends to be a supplier, sends a believable invoice or a “we changed banks” email, and a tired clerk pays it. By the time anyone notices, the money is gone and can’t be gotten back.

The fix has been known for years: before you pay, call the supplier on the number you already have stored, not the one in the new email. However, this process is slow, boring, and commonly skipped when there are hundreds of invoices to go through. We wanted to build the autonomous platform that never skips the phone call, ensuring that no scams occur. Hold makes that phone call every time it matters.

What it does

Hold is an autonomous finance system that'll manage supplier invoices. It’ll review all bills and invoices that come in and pay the normal ones on its own. If a suspicious email or invoice comes in, like a scammer that claims to be a supplier asking for more money or attempting to change their bank account details, Hold will call the real supplier from the number on file and ask questions only they would know to verify the legitimacy of the invoice. If a scammer phones the company, Hold doesn’t act on what they say. It politely says it will call back, then rings the real supplier on the number on file to check the request. Ultimately, Hold will simplify the work of an AP clerk and prevent an incredibly common type of fraud in the process.

How we built it

Every invoice that arrives is checked against what we know about that supplier, such as their usual bank account, sender domain and writing style, which Moss looks up from their past emails, plus any look-alike domains that Apify flags. When a bill looks wrong, Hold calls the supplier on the number that’s already trusted and on file. Claude decides what to ask and judges answers, using Moss to pull real facts from the supplier’s history and the invoice so that the questions can only be answered by the real supplier. Convex holds the real source of truth, while Moss is used as an index that we can quickly get data from in submillisecond time. Both sources of data always contain the same information. Cartesia provides the voice and transcribes the call into the UI, with Twilio handling the call. If the answer is wrong, the payment is held, and the invoice is passed to a human for approval after the call. Hold can block any payment, but it can’t approve its own requests or change a supplier’s bank details.

We tested it on a seeded month of 61 Fogline invoices, with 8 of them being planted scams (look-alike domains, rush pressure, a hijacked supplier mailbox, a duplicate invoice). Hold paid the 50 clean bills on its own, totaling about $237,700 in total, and held back about $109,350 in scam bills. It flagged all 8 scams and also paused 3 genuine bills for a call. It did the checking work of roughly 6 hours of clerk time, at 6 minutes a bill by hand.

Challenges we ran into

We had trouble with the timing of the voice agent. The agent would sometimes speak over the person, and would treat a pause or a filler word like “um” as a finished answer.

We were also dealing with malformed data at times, which caused the lookup for the voice agent to crash mid-call. A call that never got answered would also hang on the screen. This motivated us to improve the error handling of our systems and keep the UI up to date with techniques like listening for a heartbeat from the voice server.

Accomplishments that we're proud of

We’re proud of building the live call feature. Hold will call the real supplier when a sketchy invoice request comes in, and it will accurately determine whether the invoice is real or fake based on the contents of the call. We’re also proud of the callback feature, where if any scammer calls the company Hold represents asking for more money or trying a scam tactic, it’ll put them on hold and call the real supplier’s number to verify whether it was them or not.

We’re also proud of our guardrails that keep a human in the loop, where Hold can ask a human if they are unsure about anything, and the end-to-end loop, from reading the inbox, paying safe bills, and holding risky ones, that Hold can run autonomously.

Lastly, we’re proud of how well the system performed. In our simulated month of 61 invoices, Hold prevented the company from losing $109K to scammers and still paid $238K of normal bills autonomously.

What we learned

We learned about careful system design when building a platform that needs good security, both from scammers and from AI’s own issues. We also learned when and where to integrate human input into the system to ensure that the system is still effective enough but won’t misfire. In the rising age of AI doing everything, we wanted to ensure that for the most important thing in the world, money, humans are always in the loop.

Also, the model should never be the thing that moves money. We also learned that a voice agent is mostly about handling silence and interruptions, not about the words.

What's next for Hold

Right now Hold works for one demo company, Fogline. Next we want to make it work for real small businesses and the bookkeeping firms that pay bills for many of them. That means connecting real inboxes and accounting tools like QuickBooks, and real bank accounts instead of a sandbox, so a company can turn it on in an afternoon.

We also want it to catch more kinds of fraud: a shared blocklist of known scam numbers and accounts, a check for cloned voices on inbound calls, and calls in more languages, since many suppliers don’t speak English. Right now, Hold does learn from new techniques through a cron job that runs every night, but we want to turn it into a real model that acts much more accurately. Hold would also start in “watch only” mode, where it flags and calls but pays nothing, until a company trusts it enough to let it pay.

Built With

• anthropic
• apify
• cartesia
• claude
• convex
• moss
• next.js
• twilio
• typescript