ReCall

Poster

Inspiration

Our ability to remember details is far more limited than we realize. Research on the Ebbinghaus Forgetting Curve shows we forget up to 70-80% of new information within just 24 hours (Murre & Dros 2015). We realized that in a busy world filled with fast-paced classes, networking events, and spontaneous conversations, people often walk away from meaningful interactions only to forget important details.

This “retro” limitation of human memory inspired us to build ReCall, a modern AI-powered system that helps us remember the people we meet and the conversations we share with them.

What it does

ReCall works side by side with camera recording glasses. Users have the capability to record up to their entire day with any smart glasses and once complete, upload their captured footage to our mobile. ReCall then identifies individual conversations identifying the people involved, transcribing speech, summarizing the conversation, and storing everything into a searchable “memory timeline.” ReCall helps users recall names, shared interests, and key moments from past interactions, something human memory often loses within hours. If you meet someone at a conference, it will help you remember important details about them and automatically find their LinkedIn to stay in touch! Throughout using this app the user will build the applications "memory" and then ask questions to an AI chat application. Forgot who you talked to about an upcoming birthday? No problem! With the ReCall AI answering system it will search your conversation memory and point you to the answer. Finally, it will automatically pull highlights from your memories. If you spoke about an event or connecting with someone soon, it will be extracted and recorded on a highlights page!

How we built it

We built ReCall as a full end-to-end ML pipeline for Smart Camera Glasses:

Capture Layer - Users record a video on their own smart glasses uploads to ReCall. We utilized a pair of our own to record conversation with friends and use an example test cases.
Backend Processing (Flask) - The backend system takes in the video and begins by extracting the audio, converting it to text via AWS Transcribe / Google Chirp v3. After this, it recognizes the face of the person you are speaking with using Amazon Rekognition. The program runs a face recognition algorithm to compare faces already in the users "memory" to the current face. Once we identify the correct location to store the information, we use Google Gemini to create summaries, generate embeds and search for other information about the user.
Memory Layer - We store summaries, identified people, timestamps, and conversation metadata as structured JSON. This information includes a caption of the persons face and name. The user has the ability to update the users name if there is a typo.
Mobile App (React Native + Expo) - The frontend provides a clean interface for browsing conversations, searching names or keywords, and reviewing insights. Expo allowed for us to make a mobile app in React, easily connecting it to our own mobile device to test and run locally.

Challenges we ran into

Hardware limitations: Consumer smart glasses rarely provide direct API access, so we built ReCall around manual upload instead of live streaming. The glasses with API capabilities were outside of our budget. Winning 1st or 2nd will lets us upgrade to the more advanced glasses for continued development.
Synchronizing multimodal data: Aligning audio transcripts with face-recognition timestamps required careful pipeline design.
Multi-API latency: Had to introduce threading to coordinate Google Chirp, Rekognition, and Gemini API to minimize delays and optimize run time.
Legal and ethical constraints: We spent significant time understanding one-party consent laws and designing ReCall to follow all regulations. We wrote up a report as to why this project is legal and within the boundaries of the law. Report found here.

Accomplishments that we're proud of

We built a functioning prototype in hours connecting AWS, Google Chirp, NLP, and mobile development. Coming into this hackathon we had never used many of the AWS services or other API calls. We spent a lot of time reading documentation, watching tutorials and understanding their inner workings. We learned a lot throughout our development time and we had fun doing it! We are proud of our efforts and how well we worked as a team.

What we learned

We learned how to combine computer vision, transcription, NLP, embeddings, and mobile development into one system. We gained experience designing ethical AI tools, navigating privacy constraints, and building user-centric applications. Most importantly, we learned how to turn a raw idea into a real, working prototype under hackathon pressure.

What's next for ReCall

Integrating with more advanced camera glasses to support near real-time memory assistance
Building a more personalized “memory graph” linking related interactions
Local processing to reduce cloud usage and improve privacy
Potential RCOS project next semester!