Inspiration
Small shopkeepers often manage customer payments using notebooks, spreadsheets, or WhatsApp messages. Creating payment requests, remembering due dates, and following up on pending payments can become time-consuming.
We wanted to simplify this everyday problem with something people already know how to do naturally: speaking.
What if a shopkeeper could simply say, "Ravi-ku 450 rupees Friday-kku", and an AI could understand the request, prepare a payment link, and help track whether the customer has paid?
That idea inspired Khata Voice — a voice-first AI payment assistant designed for small merchants, with support for Tamil and English.
What We Built
Khata Voice transforms natural speech into actionable PayPal payment requests.
Instead of manually filling out payment forms, a shopkeeper can:
- Speak naturally: Enter a payment request in Tamil, English, or Tanglish using the microphone. 2.Let AI understand:Google Gemini extracts the customer name, amount, due date, and payment note from the recognized text. 3.Review and confirm: The shopkeeper can edit the extracted details before creating the payment. 4.Generate a payment link:PayPal Sandbox creates a Checkout Order and provides a customer approval link. 5.Track payments: Payment request details are stored in Redis, while the latest transaction status is retrieved from PayPal.
The goal is to make payment management feel as simple as having a conversation.
How We Built It
We built Khata Voice using Next.js, React, TypeScript, Tailwind CSS, and Shadcn UI.
Google Gemini — Intelligence Layer
Gemini powers the natural-language understanding. Browser speech recognition first converts the shopkeeper's spoken words into text. Gemini then interprets that text and extracts structured payment information, including customer details, amount, due date, and notes.
PayPal — Payment Layer
We integrated the PayPal Orders REST API to create payment orders, generate customer approval links, and capture approved payments. PayPal handles the payment processing, while the application manages the request experience.
Upstash Redis — Storage Layer
Redis stores payment request information, including customer details, amounts, due dates, PayPal order IDs, and payment links. This allows the shopkeeper to access payment records through a centralized ledger.
Frontend and Backend
Next.js provides the application interface and server-side API routes. We used reusable Shadcn UI components, Tailwind CSS, validation, authentication, rate limiting, and structured error handling to build a reliable demonstration.
Challenges We Faced
One of the biggest challenges was handling natural-language payment requests across Tamil, English, and Tanglish. The same information can be expressed in many different ways, so extracting consistent and accurate payment details required careful prompt design and structured output validation.
Another challenge was connecting AI-generated information to a real payment workflow. We needed to ensure that the shopkeeper could review and correct the extracted information before any payment order was created.
Integrating PayPal also required understanding the order lifecycle, including order creation, customer approval, payment capture, and status verification.
We also considered regional payment limitations. PayPal India has restrictions on domestic payments between Indian PayPal users, so our hackathon implementation uses the PayPal Sandbox to demonstrate the payment-request workflow rather than claiming to be a production-ready domestic payment solution.
What We Learned
This project helped us understand how AI can go beyond answering questions and become part of an actual business workflow.
We learned how to:
- Use Gemini for multilingual natural-language understanding and structured data extraction.
- Integrate an AI service with a real payment provider's API.
- Manage payment order lifecycles and transaction statuses.
- Use Redis for persistent payment request storage.
- Handle authentication, validation, rate limiting, and API errors.
- Design a voice-first experience that reduces manual interaction.
Most importantly, we learned that the real value of AI is not just understanding what someone says, but turning that understanding into something useful.
What's Next?
Khata Voice is a starting point toward making digital payment management more accessible to small merchants.
Our future plans include merchant-specific accounts, verified PayPal webhooks, automated payment reminders, improved Tamil speech recognition, enhanced payment analytics, and integrations with payment providers that support domestic transactions in India.
Khata Voice — Speak. Pay. Track.
Less typing. Less manual tracking. More time for business.
Built With
- artificial-intelligence
- digital-payments
- fintech
- gemini-api
- generative-ai
- google-gemini
- multilingual-ai
- natural-language-processing
- next.js
- payment-automation
- paypal
- paypal-sandbox
- react
- redis
- rest-api
- shadcn-ui
- speech-to-text
- tailwind-css
- tamil
- typescript
- upstash-redis
- vercel
- voice-recognition
- web-application
