-
-
A pressured payment is held on PayPal for 10 minutes. One tap cancels it, and nothing moves.
-
Verbo: send money on PayPal by speaking your own language. Live demo at verbo-371g.onrender.com.
-
The payment read back in Hindi from a fixed template, with English underneath. The amount stays $25.00.
-
The problem: money sent to a number one digit off, and the two people share no language to sort it out.
-
The words change between languages. The name and the amount don't.
-
The person who got the money by mistake sees a page in Spanish and returns it in one tap through PayPal.
-
Two contacts sound alike (Lopez and Lopes), so Verbo asks which one, out loud, in the sender's language.
-
Case Desk: every recovery case by sender and recipient language, with deadlines and evidence. Simulated data.
-
A number one digit off a contact's: Verbo stops and shows who it really reaches and the language they read.
Inspiration
My grandfather speaks mostly only Telugu. He's older and not comfortable with technology, so he relies on his voice far more than on typing.
One day he sent money to a phone number that was one digit off. The person who received it spoke only Hindi. He couldn't explain the mistake, and they couldn't understand him.
We got the money back only because I happened to be there and could translate between them. Most people in his position don't have someone standing next to them.
Payment apps are built around English and typing. People like my grandfather would rather just say the payment, in their own language. Voice makes that possible, but it also brings miscommunication:
- a name that sounds like another name
- forty heard as fourteen
- a phone number one digit off
When the money reaches someone who speaks a different language, there's often nobody to translate.
What it does
Verbo is built to be that translator. It checks the payment before it is sent, reads it back in the sender's own language, and if money still goes to the wrong person, helps both people talk to each other so it can come back.
Before the money moves
Speak in your own language. Hindi, Telugu, Spanish, Portuguese, Arabic (right to left) or English. Typing works too.
Hear it back in your language. A fixed template states who gets how much, with English underneath. The amount is always written $40.00, even inside Arabic, and it is read aloud.
Names that sound the same. Manuel Lopez or Manuel Lopes? Verbo asks which one, out loud, in your language.
A number one digit off. Verbo stops, shows who the number really reaches and the language they read, and offers the contact you probably meant.
Amounts said in words must be typed in digits, because forty and fourteen are easy to mishear.
A 10-minute hold. Pressure wording, a first payment or an unusual amount raises the risk. Risky payments are authorized on PayPal and held. One tap cancels, and nothing moves.
After a mistake
Ask for it back. AI drafts a polite request; the sender edits and sends it.
A page in their language. The person who received the money gets a private page in their language, a translated chat, and a one-tap return through PayPal.
Case Desk. Operators see every case, language pair, deadline and evidence packet.
How I built it
The AI never decides money. It only helps with the words. Code checks the name and the amount, a fixed template reads the payment back, and the person confirms.
- Speech to text: Groq Whisper (
whisper-large-v3-turbo) turns what the sender says into text. - Extraction: Google Gemini fills a strict zod schema: recipient, amount, currency. If the model fails, an offline rules parser takes over.
- Code checks: the amount must appear in what was actually said (Telugu, Devanagari and Arabic digits are normalised). The recipient is matched, sound-alike names and near-miss numbers are found, and a risk score decides whether to hold.
- Read-back: a fixed template in the sender's language, never generated text. ElevenLabs reads it aloud.
- Payment: PayPal Orders v2 authorizes first and captures only after any hold. Payments v2 voids a cancelled payment, and Payouts v1 sends the money back. Every money-changing call carries an idempotency key. Orders go through PayPal's Server SDK, generated by APIMatic.
The checks, in code
Sound-alike names use a phonetic key and Jaro–Winkler similarity:
$$ \text{sound-alike}(a, b) \iff \text{JW}(a, b) \ge 0.9 $$
A number one digit off differs in exactly one position, or has two neighbouring digits swapped:
$$ d_H(a, b) = 1 $$
The risk score is plain code over facts Verbo already has:
$$ s = 3\,x_{\text{pressure}} + 3\,x_{\text{near number}} + 3\,x_{\text{look-alike}} + 2\,x_{\text{unusual}} + x_{\text{first}} + x_{\text{language}} $$
$$ \text{hold} \iff s \ge 4 $$
Built with: Next.js and TypeScript, Drizzle and Postgres. AG Grid and AG Studio power the Case Desk and the people picker, Bryntum Scheduler Pro shows recovery deadlines, and Render hosts the live demo.
Challenges I ran into
- Free AI quotas. Gemini's free tier allowed as few as 20 requests a day per model. Verbo walks a list of models, skips any at quota until its retry time, and reuses answers.
- Speech recognition. Gemini's audio models returned errors and inconsistent transcripts in my tests, so I moved speech-to-text to Groq Whisper.
- Numbers inside right-to-left text.
$40.00inside an Arabic sentence can flip or split apart. Every value is isolated so it reads the same in every language. - Native-script digits. "౪౦" in Telugu and "٤٠" in Arabic both have to become 40 before code can compare them.
- Holding money safely. A held payment must be cancellable without ever being captured, so Verbo authorizes first and void the authorization on cancel.
- Sound-alike names across scripts. "Lopez" and "Lopes", or the same name in Telugu and in English, have to be recognised as look-alikes.
Accomplishments that I'm proud of
- The full PayPal sandbox money loop works end to end: send, authorize, capture and payout; a risk hold that is voided; and a return payment.
- Six languages, including right-to-left Arabic, with the amount written identically in every one.
- 57 / 57 scripted scenarios passed with both the offline parser and Gemini, and code caught 11 / 11 deliberately wrong model outputs. (My own test cases, not real users.)
- 232 automated tests.
- A live demo anyone can try, with no PayPal login and no keys.
- Accessible: large type, high contrast, keyboard and screen-reader support, right-to-left layout.
What I learned
- Let the AI handle the words and code handle the money. It made the system far easier to trust and to test.
- Fixed templates beat generated text for money. They are the only way to guarantee that the amount someone hears is the amount that gets paid.
- Voice is the easiest way in, and where most mistakes start. Every voice feature needs a check behind it.
- Getting money back is about understanding. The person who received money by mistake usually wants to help, if they can understand what is being asked.
What's next for Verbo
- Native-speaker review of the Hindi, Telugu, Spanish, Portuguese and Arabic templates (machine-translated today).
- Testing with real voices from older and less tech-comfortable users, starting with family members like my grandfather.
- Recovery links by SMS instead of on screen.
- More languages from the speech provider's list of 60.
Try it: verbo-371g.onrender.com · Code: github.com/Vrinda072/verbo
Built With
- ag-grid
- ag-studio
- apimatic
- bryntum
- drizzle
- elevenlabs
- ffmpeg
- gemini
- groq
- next.js
- node.js
- paypal
- paypal-orders-api
- paypal-payouts
- pglite
- postgresql
- react
- remotion
- render
- tailwindcss
- typescript
- vitest
- web-speech-api
- whisper
- zod