ReactionPay

ReactionPay: Turning global micro-transactions into unforgettable shared moments through real-time WebRTC video and live AI emotion capture.

Comment

Inspiration

This project is born directly out of my own life. I am in a long-distance relationship with my girlfriend abroad, and anyone separated by thousands of miles understands the emotional gap: you want to celebrate their day, treat them to a coffee, or surprise them with a gift. Yet, every modern payment app treats money transfers like a silent, cold database update. You tap a screen, numbers drop from your account, and minutes later you get a plain text message saying "thanks." The most meaningful part of giving a gift is completely lost: seeing the face of the person you love light up the exact moment they receive it. I searched everywhere for an app that lets you look into someone's eyes over a video call and transfer funds directly face-to-face in that exact second. Nothing existed that connected emotional presence with financial transactions. I decided to build ReactionPay to bridge that gap—turning cold cross-border transfers into a shared, face-to-face celebration.

What it does

ReactionPay is a mobile-first web application that merges real-time encrypted video calling with instant digital gifting and automated emotion capture: Peer-to-Peer Video Calling: Two users enter a simple room name to connect instantly via high-quality, low-latency video and audio without needing third-party user accounts. In-Call PayPal Sandbox Gifting: Senders can choose preset gift tiers (like $5 Coffee, $20 Bouquet, or $50 Date Night) or input custom amounts. The PayPal payment sheet slides up directly over the live call, allowing checkout to happen without pausing or closing the video feed. Instant P2P Synchronization: Using WebRTC DataChannels, the moment a transaction is completed, a synchronized celebration event fires on both phones—bursting confetti, triggering audio chimes, and displaying a live gift notification banner on the recipient's screen. Multimodal AI Emotion Radar & Polaroid Keepsake: At the exact millisecond the gift lands, the application captures a high-resolution frame of the recipient's live video reaction. Google Gemini 2.5 Flash analyzes their facial expression, scores their happiness index, and writes a customized, poetic one-line Polaroid caption. Memory Vault: Captured moments are stamped into physical-style Polaroids, added to an in-session Memory Vault, and can be downloaded directly to the device as a keepsake.

How we built it

The entire platform was built with an honest, resourceful mobile-first workflow: ​100% Smartphone Engineering: The entire project was written, debugged, and tested directly on an Android smartphone using the Acode mobile code editor, with version control managed through Git and hosted globally on Cloudflare Pages. ​AI Collaborative Co-Engineering: I drove the product vision, interaction design, fintech UI architecture, and feature pipeline, while collaborating with Gemini to co-engineer the JavaScript, WebRTC state machines, and multimodal API integration. ​Fintech UI/UX Design: Built with semantic HTML5 and modern CSS3, adopting an authentic PayPal fintech palette (#001c64 Deep Navy, #0070ba Blue, and #ffc439 Gold) paired with lightweight glassmorphism overlays to keep the video stream visible beneath interactive controls. ​Edge Hosting on Cloudflare: Deployed via Cloudflare Pages to ensure fast, low-latency static delivery and high-availability SSL across global edge networks, critical for international cross-border connections. ​Open-Source WebRTC (PeerJS): Integrated PeerJS backed by public Google STUN servers (stun.l.google.com:19302) to ensure zero-barrier P2P camera streaming and instantaneous cross-device event broadcasting without requiring backend servers. ​Payment Engine: Implemented the universal PayPal JavaScript SDK (client-id=sb), handling dynamic order creation, sandbox authorization, capture callbacks, and live balance reconciliation. ​Multimodal Vision Intelligence: Utilized an off-screen HTML5 to capture raw video frames, encoding them to Base64 JPEG buffers and dispatching them to the Gemini 2.5 Flash API with strict JSON schema instructions for emotion detection and caption generation.

Challenges we ran into

Developing Exclusively on a Mobile Screen: Writing nested JavaScript callbacks, debugging styling, and organizing multi-file projects on a touch screen in Acode required patience, discipline, and constant testing. Simultaneous Video Playback and Checkout Sheets: Mobile browsers frequently throttle background media when modals or third-party iframes appear. Crafting a floating drawer architecture that allowed the PayPal SDK to open while maintaining a smooth, hardware-accelerated 30 FPS video feed beneath it required fine-tuning CSS layer properties and pointer-events. Zero-Latency Cross-Device Synchronization: Ensuring the recipient's screen triggered confetti, sound, and snapshot capture the instant the sender approved the PayPal payment required managing bidirectional WebRTC DataChannel states alongside sandbox payment callbacks. Dynamic Video Aspect-Ratio Extraction: Accounting for different smartphone camera orientations and dynamic WebRTC stream resolutions required engineering an adaptive canvas resizer to prevent stretched or distorted Polaroid snapshots.

Accomplishments that we're proud off

Building a Full-Stack Concept on a Phone: Proving that an ambitious, working hackathon application integrating WebRTC, payments, and generative AI can be developed entirely on an Android smartphone. ​Overcoming Infrastructure Hurdles via Cloudflare: Rapidly pivoting deployment to Cloudflare Pages directly from a mobile browser when infrastructure challenges struck, successfully keeping our edge site fast, responsive, and global. ​Frictionless Open Testing: Eliminating login gates, complicated tokens, and paid server setups so judges and users can open a link, enter any room name, and start a video call immediately. ​Preserving Human Connection in Payments: Successfully taking a sterile transaction—moving money digitally—and turning it into an emotional, joyful memory that lives on in a downloadable Polaroid.

What we learned

The Power of Resourcefulness: Advanced developer machines are convenient, but clear vision, strong architectural fundamentals, and AI pair-programming are all you need to build functional software anywhere. WebRTC DataChannels for Micro-Interactions: Gained deep practical insight into leveraging WebRTC data streams as an ultra-fast event bus for synchronizing multi-user UI states without a database. Context-Aware Multimodal AI: Discovered how effective Gemini 2.5 Flash is at interpreting subtle human emotions from video frames and generating warm, creative copy that resonates on a personal level.

What's next for ReactionPay

What's next for ReactionPay 3-Second Motion Clips: Evolving the single-frame Polaroid into a short animated GIF or mini MP4 recording that captures the full reaction arc from surprise to smile. Real-Time Cross-Border Currency Conversion: Adding live currency conversion APIs so senders can pay in their domestic currency while recipients see the gift reflected in their local currency. Scheduled & Milestone Surprise Gifting: Allowing partners to lock a gift inside a video room with a countdown timer that unlocks only during a special anniversary or birthday video call.

Built With

Share this project:

Updates

Submission history