Inspiration: Paying someone in person often means asking for their payment handle, searching through contacts, or scanning a QR code. We wanted to make recipient discovery feel as natural as looking at the person—while keeping the recipient’s app closed and their phone offline. What it does NearPay lets a payer scan an opted-in recipient’s face to find their payment profile. The payer reviews the recipient and amount, then approves checkout with PayPal. Recipients enroll once and don’t need to keep the app open or run background services to be found.
Face matching is used to find a recipient; it is not payment authorization. The payer must still review and confirm the payment.
How we built it: We built the mobile app with Expo, React Native, and TypeScript. During enrollment and scanning, ML Kit detects a face on the phone, and a bundled FaceNet TensorFlow Lite model creates a 128-dimensional face embedding. The camera photos are processed locally and discarded; the app sends the embedding to the NearPay server for matching.
The Fastify API stores opted-in templates in PostgreSQL, encrypted with AES-256-GCM, and compares them during a scan. It returns the recipient’s display name, a masked PayPal reference, and a short-lived match session.
For checkout, the server creates a PayPal Orders API order. The payer approves through PayPal, and the server verifies the order details and payment status before capture.
Challenges we ran into: Getting camera capture, face detection, cropping, and model inference to work consistently across Android devices took iteration. We also found an integration mismatch where the Android face detector’s result didn’t include a success field that the app expected; valid captures were therefore rejected. Correcting that check let enrollment progress.
We had to balance recipient convenience with privacy: keeping the recipient offline means matching happens on a server, so face embeddings need careful protection. PayPal seller setup and payment status verification also required us to keep checkout server-controlled rather than trusting a mobile success callback.
Accomplishments that we're proud of: We brought together account setup, explicit face discovery opt-in, on-device face processing, server-side matching, payment review, and a PayPal Sandbox checkout path in one mobile prototype. We’re especially proud that recipients can be offline at scan time, while the payer still gets a clear review and confirmation step.
What we learned: A face embedding is still sensitive biometric data, even when it isn’t a photo. Face matching can help locate a recipient, but it does not prove liveness or authorize a transaction. We also learned that payment approval and capture must be verified with the payment provider on the server.
What's next for NearPay: We want to evaluate the matcher across more devices, lighting conditions, and consenting participants; improve liveness and anti-spoofing; and strengthen account recovery, retention, and key-management practices. We also need to complete the PayPal seller onboarding and partner requirements for a production deployment.
Built With
- aes-256-gcm
- android
- api
- expo.io
- facenet
- fastify
- ml
- native
- node.js
- orders
- paypal
- postgresql
- react
- rest
- tensorflow
- typescript
