MemoryLens

Inspiration

I have seen a lot of people in my time, specifically grandparents, go through dementia and it can be incredibly saddening to see them forget their loved ones, and to see how their loved ones react, which is why I built MemoryLens

What it does

MemoryLens is a face detector and memorizer built for people who suffer with dementia. This is ideally integrated into AR glasses such as Meta glasses but as for right now, it is within an app. This will scan people's faces and store them into a Supabase database, in which whenever that face is brought up again, even when the app is closed and reopened, the face will be remembered. It also records conversation history and brings up captions for whenever people are talking. You must manually enter the person's name and relationship but everything else will be summarized with an LLM we have integrated into our app to summarize the conversation that was occurring.

How we built it

We built MemoryLens with reactjs and supabase. Originally created the base of the app using lovable but later fixes some edges with claude code.

Challenges we ran into

One major challenge we ran into was face detection. We originally used regular face-api.js and tried to use the github page for that but that was having an incredibly hard time with detecting faces and detecting faces is the number one priority for this app. We fixes this challenge by using vladmandic's fork which worked a lot better and is recording and detecting faces with no issue now.

Accomplishments that we're proud of

Accomplishments that I am proud of is being able to use Supabase as this is my first time working with a major open source backend service and using it properly. Another thing that I am proud of is actually incorporating an LLM to go with the app. Although it is a simple gpt-wrapper, I am proud that I was able to manage to squish it into the app.

What we learned

What I learned in this project is the use of Supabase and LLM's

What's next for MemoryLens

Next things for MemoryLens include, making the incorporating people not manual and can be incorporated just by listening in on conversations, exporting the conversation logs into something else so that it could be used by caregivers, notifications when people leave, clearing data button, stats display saying like "You talked to X amount of people today". Also the face recognization is a bit iffy. It does work but you might need to move ur face to different angles and locations for it to remember.

Built With

• face-api.js
• react
• supabase