Vernacular AI

An AI storefront for empowering rural artisans to digitize handlooms using native voice featuring local zero-commission UPI sales and PayPal global export mode.

Comment

Inspiration

India's rural handloom weavers create some of the most beautiful textiles in the world, yet they remain trapped in poverty by middlemen. The barrier to entry for global e-commerce—requiring perfect English, professional photography, and complex payment gateways—is incredibly steep for a village artisan.

Inspired by the PayPal AI Hackathon and the Seva First Innovation Challenge, we asked ourselves: What if an artisan could build a global e-commerce storefront just by speaking in their native tongue? We built Vernacular AI to bridge the gap between offline rural heritage and global digital commerce.

What it does

Vernacular AI is a dual-commerce, two-sided marketplace connecting rural artisans directly with buyers.

For the Artisan (The Studio): An artisan takes a photo of their saree on their budget Android phone and records a short voice note describing it in Bengali or Hindi. Our platform uses Google Gemini's multimodal AI to instantly generate an SEO-optimized product catalog in English, complete with a calculated "Fair Market" local price (INR) and an export-markup international price (USD).

For the Buyer (The Storefront): Buyers browse a premium, responsive marketplace featuring a randomized "Masterpiece Carousel" and a "Voice of the Weaver" audio player that streams the artisan's original voice recording, building immense trust and emotional connection. The commerce engine is split into two modes:

  • Local Mode (Seva First): Buyers in India can check out via a zero-commission generated UPI QR code and a pre-filled WhatsApp order flow, ensuring 100% of the money goes directly to the weaver's bank account.
  • Global Export Mode (PayPal): International buyers can toggle the storefront to USD and securely check out using the integrated PayPal React SDK, instantly making the village weaver a global exporter.

How we built it

We architected a modern, highly scalable full-stack application:

  • Frontend: Built with Next.js, TypeScript, and Tailwind CSS. We implemented a lightweight Client Context Dictionary for instant UI localization (English, বাংলা, हिन्दी) without heavy routing.
  • Backend: Powered by a Python FastAPI server hosted on Render.
  • AI Engine: Google Gemini Multimodal API processes both the raw image data and the vernacular audio file to generate the structured JSON catalog and pricing.
  • Database & Auth: Supabase handles our PostgreSQL database, Row Level Security (RLS), and Storage buckets (for the audio files). We utilized Google 1-Tap OAuth to provide frictionless, zero-cost login for Android-heavy rural users.
  • Payments: Integrated the PayPal JavaScript SDK for the global sandbox checkout, and qrcode.react for dynamic zero-commission UPI generation.

Challenges we ran into

Building a true two-sided marketplace is notoriously difficult. Our biggest challenges included:

  1. Auth Race Conditions: Synchronizing Google 1-Tap logins with Supabase database triggers caused routing loops where artisans were temporarily identified as shoppers. We solved this with targeted .update() and .upsert() state-syncing.
  2. Complex State Management: Getting the CSS flexbox carousels to seamlessly slide while handling dynamic AI-generated text, images, and audio players required strict aspect-ratio locking and container boundary enforcement.
  3. Database Security: Securing the platform required complex Row Level Security (RLS) policies in PostgreSQL to ensure artisans could only mutate their own inventory while keeping the storefront public.

Accomplishments that we're proud of

  • Successfully wiring the PayPal Developer Sandbox to dynamically read AI-generated USD pricing.
  • Engineering the "Voice of the Weaver" audio pipeline—saving raw microphone data to a Supabase bucket and successfully streaming it back to the Next.js frontend.
  • Eliminating SMS/Twilio onboarding costs completely by engineering a frictionless Google OAuth flow that perfectly matches the rural Android demographic.

What we learned

We learned how to deeply integrate Backend-as-a-Service (BaaS) tools like Supabase with custom Python AI microservices. We also gained immense experience in building accessible, mobile-first UIs that don't compromise on a premium e-commerce aesthetic.

What's next for Vernacular AI

We plan to expand the vernacular dictionary to include Marathi, Tamil, and Telugu. Next, we want to integrate shipping and logistics APIs so that when a PayPal transaction clears globally, a local courier is automatically dispatched to the weaver's village to pick up the parcel.

Built With

Share this project:

Updates

Submission history