RemindAR

• 

  Landing Page

• 

  Try demo section

• 

  Demo

• 

  Dashoard view with add later notification

• 

  Our vision

• 

  How it works

Inspiration

RemindAR was inspired by how deeply memory affects human connection. For people living with dementia or cognitive decline, not recognizing familiar faces can be confusing, distressing, and isolating. These moments affect memory, confidence, dignity, and relationships. We wanted to explore how technology could support memory at the moment it’s needed, rather than testing or correcting the user afterwards. The idea of subtle AR overlays - similar to future smart glasses

• felt like a natural, non-intrusive way to assist recognition while preserving independence.

What it does

RemindAR is a real-time assistive AR prototype that helps people recognise faces and recall relationships during live interactions. Using a webcam, it detects faces, recognises known individuals, and displays contextual information such as names, relationships, and shared memories as floating AR-style labels. Users can register new people and query their memory using natural voice input in English, Hindi, & other languages (powered by Gemini).

How we built it

RemindAR is built as a real-time system with a modern frontend and an AI-powered backend.

1. Face detection runs in-browser using MediaPipe for fast, low-latency performance.
2. Face recognition uses InsightFace embeddings on the backend, matched using cosine similarity:
3. Gemini 3.0 Flash is used for voice transcription, structured data extraction, and contextual memory queries in a single API call.
4. WebSockets enable real-time communication between the frontend and backend.
5. SQLite provides fast local storage, with Firebase Firestore used for cloud sync and persistence.
6. The frontend is built with React, TypeScript, and Vite, using subtle GSAP animations to reinforce an AR-like experience.

Challenges we ran into

1. Keeping face recognition fast in real time without reducing the quality and accuracy of face embeddings.
2. Making sure updates to a person’s details are reflected instantly during live face recognition.
3. Supporting multiple languages reliably, which was difficult with local models and was solved using Gemini.
4. Allowing instant face re-scans without causing UI lag or interrupting the user experience.
5. Bringing together live camera, AR overlays, voice input, and dashboards into a smooth and cohesive frontend.

Accomplishments that we're proud of

1. Building a fully real-time face recognition pipeline with live AR overlays.
2. Implementing voice-first person registration and memory queries using Gemini 3.0 Flash.
3. Supporting multilingual input (English, Hindi, Hinglish) without separate pipelines.
4. Designing an assistive, low–cognitive load UI instead of a purely technical demo.
5. Creating a reliable hybrid storage system using SQLite and Firebase Firestore.

What we learned

1. Building for memory challenges requires an accessibility-first approach, where reducing cognitive load matters as much as model accuracy.
2. Real-time system design benefits from splitting responsibilities: MediaPipe for in-browser detection and InsightFace for backend recognition.
3. Using Gemini 3.0 Flash for transcription, structured extraction, and memory queries in a single pipeline simplifies multimodal AI workflows.
4. Combining SQLite (local) and Firebase Firestore (cloud) improves reliability and graceful degradation in assistive applications.

Changelog

Over the past few days RemindAR has received significant updates!! Here's whats changed:

1. Updated frontend to be more informative with a "splash screen" to guide you through the process when trying the demo.

2. Context history per person, updates as you update the details and get saved in the dashboard. Can be accessed by clicking on the person's card in dashboard.

3. Region selector for more language support and more accurate Speech-to-text.

4. Add later feature. RemindAR notifies you if you want to add a person whom you've been talking to for more than 3 minutes. You can click 'later' and their temporary entry will be saved in dashboard.

5. Native language support. RemindAR will now infer from the spoken content which language you mostly talk in and show the entries in that language only for a more inclusive experience.

What's next for RemindAR

Future work includes optimising RemindAR for wearable AR devices, improving on-device processing for privacy, and expanding accessibility features such as gesture-based interaction and personalised memory cues to make users' lives easier. I also plan to explore collaborations with caregivers to evaluate real-world usability and refine the system based on real user needs.

Built With

• firebase
• gemini
• gsap
• mediapie
• python
• react
• sqlite
• tailwind
• typescript