EyeConnect

EyeConnect Home Screen
App Icon on Home Screen
Vercel Deployment
Supabase Backend database

Inspiration

Blind and visually impaired individuals often struggle with everyday tasks that rely heavily on sight reading labels, identifying objects, navigating spaces, or checking expiry dates. While AI tools exist, they frequently fail in real-world, imperfect environments.

During HackNYU, we wanted to build something human-centered. Something fast, accessible, and reliable. That became EyeConnect - a platform that instantly links blind users with sighted volunteers through real-time video calls for visual assistance.

What it does

EyeConnect connects blind users with sighted volunteers for real-time visual help. Here’s what the platform offers:

Peer-to-peer video calls using WebRTC Instant volunteer matching via Supabase Realtime Accessible UI designed for blind users - large buttons, minimal navigation, screen-reader-friendly Volunteer dashboard to accept incoming help requests AI fallback system: If no volunteer responds within 30 seconds, the user is redirected to an AI visual assistant powered by the OpenRouter Vision API

The goal is to ensure blind users always get visual assistance, either human or AI.

How we built it

We used a modern, lightweight stack focused on speed and reliability:

Frontend: React + TypeScript + Vite UI: shadcn-ui + Tailwind CSS Realtime signaling: Supabase Realtime Channels Video calls: WebRTC (SDP exchange + ICE candidates) Database: Supabase table to track call states and connect users across devices AI vision: OpenRouter API for fallback image-to-text assistance

Flow:

Blind user requests help. Supabase broadcasts the request to volunteers. A volunteer accepts -> WebRTC handshake begins. Signaling happens through Supabase. A peer-to-peer video stream is established. If no volunteer accepts -> AI fallback activates.

Challenges we ran into

WebRTC connection issues (SDP negotiation, ICE candidate syncing) Users getting stuck on “connecting…” without proper signaling Preventing multiple volunteers from accepting the same request Creating a screen-reader-friendly UI under time pressure Integrating a 30-second AI fallback without breaking the main flow Syncing real-time state changes across different clients

Accomplishments that we're proud of

Built a fully working real-time video assistance platform in under 24 hours Successfully integrated Supabase Realtime + WebRTC Designed an interface that blind users can navigate easily Added a smart AI vision fallback to guarantee help 24/7 Created a clean, modern, accessible UI using shadcn + Tailwind Implemented reliable multi-device call matching

What we learned

Deep understanding of WebRTC and real-time communication How to use Supabase Realtime for cross-client event syncing How to design digital products for accessibility State management under real-time conditions Integrating AI as a fallback solution Team collaboration under tight hackathon deadlines

What's next for EyeConnect

Stronger global video reliability with TURN servers Voice navigation for blind users Better volunteer matching (language, region, availability) PWA/mobile app version Session logs for safety + transparency More advanced AI scene interpretation Optional audio transcription paired with video assistance