UPI Sahayak

Turning UPI confusion into clarity.

Comment

Inspiration

This project started with a very simple, real-life moment.

At a small kirana shop, we saw a customer show a green “Payment Successful” screen and say, “Ho gaya.”
But the soundbox didn’t beep.

The shopkeeper paused.

For a few seconds, he didn’t know what to do.

If he stops the customer, it feels rude.
If he lets them go, he might lose money.

That hesitation — that confusion — is something millions of small merchants face every day.

We realized the problem isn’t technology.
UPI is fast. Payments work.

The problem is trust and clarity.

And that’s where we decided to help.

What it does

We built UPI Sahayak, an AI-powered assistant for small merchants.

Not another payment app.
Not another complicated dashboard.

Just a simple helper.

Something that:

  • checks if a payment screenshot is real, processing, or fake
  • listens when a merchant speaks in their own language
  • explains what to do next
  • automatically drafts a bank complaint
  • keeps dispute history and customer khata records

If they can use WhatsApp, they can use UPI Sahayak.

That was our design rule.

How we built it

We used tools that are reliable, scalable, and familiar.

  • Gemini 2.5 Flash Native Audio
  • gemini-3-flash-preview
  • React + TypeScript for the web app
  • FastAPI (Python) backend
  • Firebase for authentication
  • Firestore for storing history and khata
  • Docker for deployment
  • And Google Gemini AI for multimodal intelligence

Gemini became the “brain” of the system.

It looks at screenshots, understands voice inputs, extracts transaction details, detects fake proofs, and even writes dispute emails — almost like a real assistant sitting beside the merchant.

Challenges we ran into

The hardest part wasn’t coding.

It was empathy.

We had to constantly ask: “Will a non-technical shopkeeper understand this?”

We removed forms.
Reduced typing.
Added voice.
Added regional languages.

We learned that simple is actually harder than complex.

Teaching AI to understand messy screenshots and real-world language was also challenging.
Network delays, unclear screenshots, and ambiguous payment states made us design carefully so the app gives clear, actionable guidance instead of confusing results.

Accomplishments that we're proud of

We’re proud that we built:

  • A fully working multimodal AI assistant using both screenshot and voice
  • Support for 23+ Indian languages
  • Instant fake screenshot detection
  • Automatic bank complaint generation
  • A clean, Google-style interface with almost zero learning curve

Most importantly, we built something that solves a real, everyday problem, not just a technical demo.

What we learned

This project taught us something important:

Technology is powerful, but only when it feels human.

For a merchant, even losing ₹500 matters.

If our app can reduce even a little stress at the counter,
help them trust a payment,
or save their time…

then it truly makes a difference.

We also learned how multimodal AI — combining vision and voice — can make technology more natural and accessible for everyone.

UPI made payments digital.
With UPI Sahayak, we hope to make them understandable and stress-free.

What's next for UPI Sahayak

This is just the beginning.

Next, we plan to:

  • improve fake detection accuracy
  • integrate directly with bank systems
  • add WhatsApp and SMS reconciliation
  • enhance the khata feature
  • and optimize for low-network and offline environments

Our goal is simple:
to make UPI Sahayak the trusted companion for every small merchant in India.

Built With

  • docker
  • fastapi
  • firebase-authentication-(google-sign-in)
  • firebaseauthentication-googlesignin
  • flutter-(dart)
  • flutter-dart
  • gemini-2.5-flash-native-audio-preview
  • gemini-3-flash-preview
  • gemini-3-flash-preview-(gemini-3-flash-preview)
  • google-firestore
  • google-generative-ai-sdk
  • googlefirestore
  • googlegenerativeai-sdk
  • python
  • react19
  • render-(backend-hosting)
  • render-backendhosting
  • tailwind-css-(material-3)
  • tailwindcss-material3
  • typescript
  • uvicorn
  • vercel-(frontend-hosting)
  • vercel-frontendhosting
  • vite
Share this project:

Updates