GlassPay

Home page
Contacts
Activity page
Profile configuration
Assistant
Statistics by week
Profile configuration
Statistics by month

Inspiration

Splitting a dinner bill or charging a friend still means pulling out your phone, opening an app, typing amounts and copying links. Smart glasses like the Oakley Meta / Ray-Ban Meta are becoming everyday devices, but payments haven't caught up. We wanted to collect money the way you'd ask for it in real life: by looking at someone and saying it out loud, while keeping PayPal-level safety.

What it does

GlassPay is a hands-free payments agent for whoever is collecting the money:

Identify the payer: the payer shows a temporary QR (glasspay:<token>) from the payer web app. It carries no personal data, just an opaque, short-lived token. It can be scanned with the phone camera or with Oakley Meta glasses (iOS).
Say what to charge: "Charge 350 pesos for dinner". An AI agent (Gemini) turns the phrase into a structured proposal: amount, currency, concept and payer.
Confirm explicitly: the app reads the preview aloud ("350 MXN for dinner. Confirm?") and nothing is created until the collector says "yes" or taps Confirm.
Payer approves with PayPal: the request appears on the payer's phone, they approve it in PayPal, and the order is captured.
Get paid: "Payment received!" is shown (with voice and haptics) only after PayPal confirms through a verified webhook.

It also includes movements history, saved contacts (charge again without scanning), dashboard stats and charts, biometric confirmation for sending money, scheduled charges from the assistant, and a Spanish and English interface, voice and assistant.

How we built it

Android app (Kotlin): collector wallet with Home, Movements, Charge and Contacts tabs, camera QR scanning, speech recognition, text-to-speech, biometric checks and the session secret encrypted with the Android Keystore.
iOS app (SwiftUI): the same flow, with Oakley Meta glasses integration for QR reading and voice output through the glasses, plus English support.
Payer web app (Vite): creates the payer profile, shows the QR, polls pending requests and lets the payer approve or reject with PayPal.
Server (Node.js + Express): PayPal Orders API in Sandbox (create order, capture, return/cancel pages), verified and idempotent PayPal webhooks, collector and payer identity, contacts and movements, deployed on Render.
AI agent (Gemini): converts transcribed speech into a structured proposal with a 5 s timeout, a retry on 429/503 and a deterministic fallback parser, so the flow still works if the model is slow or down. An evaluation suite of 25 phrases (including ambiguous and adversarial ones) checks the agent.

Challenges we ran into

Letting AI near money safely. We made the AI propose-only: it can't create charges or touch the database. Amount and currency must be backed by the spoken text, and every charge requires explicit human confirmation.
Truthful payment status. The return page never marks a payment as paid and PayPal's PayerID is ignored; only a verified webhook moves a request to "paid".
Identity without exposing data. Single-use, five-minute QR resolutions bound to the collector, hashed status tokens sent only in headers, and secrets that are shown once and never placed in URLs.
Speech in the real world: recognizers mishearing words, noisy rooms, and voice output routed through the glasses.
Free-tier hosting: Render's free plan sleeps and wipes SQLite, so we built a keep-alive and a demo checklist around it.

Accomplishments that we're proud of

A complete end-to-end flow (voice → AI → confirmation → PayPal → verified webhook) running on real devices.
Native apps on both Android and iOS, with real smart-glasses support on iOS.
An AI agent with guardrails and a fallback, so a model outage never blocks a payment.
A security model with double confirmation, opaque QR identity and server-only secrets.

What we learned

How the PayPal Orders API, capture and webhook verification fit together, and why the webhook, not the redirect, is the source of truth.
How to design AI agents for payments: structured output, validation against the user's own words, and human-in-the-loop confirmation.
Voice UX: read back, confirm, and give clear feedback with sound, haptics and animation when the screen is on your face.

What's next for GlassPay

A fully glasses-native experience on Android as the SDKs allow, so you never take out your phone.
Moving from PayPal Sandbox to production, with Venmo and more currencies.
Splitting a bill among several people in a single sentence.
Persistent storage and production hosting, plus payment reminders and recurring charges.