Inspiration
In 2025, the FBI reported that businesses lost over $3 billion to business email compromise. That's when scammers send emails pretending to be someone a business already trusts, like a vendor they pay every month.
The fix is just as simple. Call the vendor and ask. But small business owners don't have that type of time, especially if they are bombarded with multiple different invoices. Plus if the email is fake, the phone number in it probably is too. So then the owner would have to spend more time fact checking the invoice information before calling them.
This is where Vouch comes in, its an agent that the owner can use that will asses the invoices, fact check the information, and call the supplier.
What it does
When an invoice comes in, our platform reads it and checks whether anything looks off. Did the bank details change? Is the email coming from a slightly wrong domain? Is it pushing you to pay right now? If something seems wrong, it freezes the payment.
Then, using the Bright Data, it looks up the vendor's real phone number online, not the one in the email, and calls them with an AI voice agent. It asks whether they really changed banks, and asks something only the real vendor would know, like how much last month's invoice was.
If they say yes, the payment goes through. If they say "we never sent that," the payment gets blocked and the owner gets an alert.
How we built it
Gemini reads each invoice and pulls out the important stuff: vendor, amount, bank details and who sent it. We then developed an algorithm that incorporates Gemni's input along with other factors to give a risk score.
Bright Data does the fact-checking. We use it to find the vendor's real website and phone number, and to check whether the sender's domain actually matches the company. ElevenLabs handle the phone call. When the call ends, the result and transcript get sent back to our server.
Neon acts like the agent's memory. It remembers every vendor, their usual invoice amounts, their bank details, past calls and every action the agent takes. That history is what lets the agent ask questions only the real vendor can answer.
The dashboard is built with Next.js, and you can watch a payment go from frozen to blocked in real time.
Challenges we ran into
One of the main challenges we ran into was wiring up all the different APIs in time. To make the agent workflow, we had to connect a couple of different technologies together in order, so we struggled with standardizing how the data looked whenever it entered or left a different API. Along with that, a more specific issue we faced was whenever we would look to fact check fake information through the Bright Data Web Scrapper, it would often autocorrect the url name or the company leading to many false negatives. To fix this, we had to edit the settings of the Web Scrapper, and also how we handled the data to let us know if it was autocorrecting.
What we learned
The AI model was the easy part. The hard part was everything around it: getting the data into the same shape between every tool, keeping track of where each invoice was in the process, and logging every step so we could see why the agent made a decision.
We also learned that tools trying to be helpful can work against you. A search that autocorrects typos is great for normal users, but in fraud detection the typo is the whole point. Splitting the work between the AI and our own code made the agent much more reliable. Letting Gemini read the invoice and having our code make the final risk call meant every decision had a clear reason we could explain.
Finally, we learned why a phone call beats an email reply. If a scammer controls the email thread, replying just goes straight back to them. Calling a number you found yourself doesn't.
What's next
We want to build a shared blocklist, so if one business catches a scam account, everyone using the agent is protected. We'd also like to connect it to tools like QuickBooks and real email inboxes, and screen incoming calls from scammers pretending to be vendors.
Demo Password: Password123
Built With
- bright-data
- cursor
- elevenlabs
- gemni
- neon
- next.js
- node.js
- postgresql
- react
- typescript
- vercel