KAPYIŊI

Inspiration

Paying someone should not depend on being able to see a screen. Many payment flows rely on small buttons, forms and visual checks, which are hard to use with a screen reader and impossible for people who cannot read. We wanted to show what PayPal could feel like with an assistant built in: you talk, it listens, and you stay in control of your money.

Kapyiŋi means tree in Senoufo. It is the name of our agent.

What it does

Kapyiŋi is a voice-first payment assistant built on the PayPal Sandbox. It is designed for people who are blind, have low vision, or have trouble reading or navigating, and for anyone who would rather talk than type.

You say things like:

"Send thirty euros to my brother"
"I want to pay my bill"
"What happened with my last payment?"

Kapyiŋi finds the contact, reads the payment back, asks for your fingerprint or Face ID (or, on a device without one, asks you to say the amount back), hands you over to PayPal to approve, and announces the result only once PayPal confirms it.

Two modes, chosen by the user. Voice mode, or a classic mode with the usual screens where Kapyiŋi stays silent. Kapyiŋi never guesses who is blind.
Hands-free listening. Kapyiŋi stops speaking as soon as the user starts talking, so there is no icon to touch.
Your own space. Contacts, history, passkeys and settings belong to each user. No e-mail and no password: a passkey brings the user back to their space on any device.
Works with screen readers. If the user has VoiceOver or TalkBack on, Kapyiŋi speaks only through them, so you never hear two voices.
Also usable by touch and keyboard. Every spoken answer is also a large numbered button.
Contacts by voice. Add, change or remove a contact, with a read-back and a confirmation before anything is saved.
Honest answers. It says plainly that PayPal does not share the balance with outside apps, and that it cannot reach PayPal staff.

How we built it

PayPal: Orders v2 (create with payee, capture, show) to send money, and Invoicing v2 (search, show, status) to read and pay bills. Every money call carries a PayPal-Request-Id.
AI: Claude Haiku 5.5 turns one transcribed sentence into a structured intent (action, contact, amount, currency). It sees only the sentence and the contacts' names and nicknames, has no tools, and plays no part in confirming a payment.
Stack: React, Vite and Tailwind CSS (Atkinson Hyperlegible typeface, French and English), a Node/Express server, Upstash Redis, deployed on Vercel. Speech uses the browser's Web Speech API.

Three locks keep the money safe:

Lock	Who decides	What it stops
1. Rules	Our code, no AI	A tricked AI, odd amounts, duplicates: known contact, per-payment limit, a prepared request lasts 5 minutes and works once
2. Intent and owner	The user, on their own device	Misheard words, a reflex "yes": read-back first, then a WebAuthn passkey (fingerprint or Face ID) bound to that one payment, or the amount said back
3. Authorisation	PayPal	Someone else holding the phone: the user approves on PayPal's own page, and Kapyiŋi never handles PayPal passwords

A bare "yes" never sends money, and a different amount said at the confirmation stops and asks. We chose a passkey rather than a voiceprint because browsers offer no speaker-verification API and a voice can be recorded.

Challenges we ran into

Trusting an AI with money. We kept the AI to one job, understanding a sentence, and put every decision about money in plain code that we can test, including a test where a compromised AI tries to send 9,999 €.
Confirming by voice safely. A spoken "yes" is not proof of intent. Read-back, the amount said back and the passkey each answer a different risk.
Listening while speaking. Letting the user interrupt the assistant without a button meant making sure the agent stops talking and listens at the right moment.
Sandbox details. Captures can come back as pending when the recipient does not hold the currency, and an invoice page has no automatic return, so Kapyiŋi checks the status when you come back.
Building as a team in a few weeks, with the work split by file to keep merges clean.

Accomplishments that we're proud of

A full journey, from spoken sentence to verified PayPal result, that works for someone who never looks at the screen.
50 automated tests covering the safety rules, spoken amounts in French and English, the whole conversation, the fingerprint step, user spaces and both storage back-ends. They run against a fake PayPal server, so we also checked the real Sandbox by hand.

What we learned

Accessibility is a design constraint from the first line, not a feature added later. And with AI and money, the useful question is not what the model can do, but what it must never be allowed to decide.

Try it

Source code and run instructions: https://github.com/kapyini-hackathon-2026/kapyini-voice-payments (MIT license). Everything runs on the PayPal Sandbox with test accounts and play money.