FamiliarFaces

Inspiration:

My grandmother lives with a severe form of dementia called Alzheimer’s. Every day, she faces the heartbreaking challenge of forgetting not only her own identity but the faces and stories of the people who love her most. Through intensive brainstorming we came up with a solution for this prominent problem: FamiliarFaces.

What it does

An app for dementia patients that runs on a tablet/monitor in their room. When a visitor walks into the camera frame, it recognizes their face and instantly displays reassuring context about who they are allowing patients to recall an individual from past memories. It also records and summarizes conversations for future reference.

How we built it

We built our app using TypeScript with Next.js and Tailwind CSS for the frontend. For the backend and database, we used Prisma ORM with SQLite. We implemented facial recognition using face-api.js and handled AI conversation summarization locally using Ollama with the model Llama 3.2. The app runs fully locally and integrates with the browser and device’s camera and speech capabilities.

Challenges we ran into

We ran into the big hurdle of ambition and not knowing how to integrate. We wanted to add many features, deploying it for real, and had to cut back and in the process, learn new tools and other solutions to achieve our final result.

Accomplishments that we're proud of

We are extremely proud of our concept and execution of our Familiar Faces web - app. This is a prominent issue that we’re all extremely passionate about fixing!

What we learned

We learned a lot about scope and making a feasible project. In the process of thinking of an idea that we struggled to implement, we learned many new tools such as face-api.js that really tested our skills adapting in real time and learning on the spot.

What's next for FamiliarFaces

What’s next is deployed on the web and scaling up to a production-ready database backend with properly implemented security. There are plenty of aspects to refine such as making the recognition more reliable and cleaning up the little details in the UI. Some other additions and interfaces this could be applied to could be AR glasses or VR headsets such as the apple vision pro.

Built With

• face-api.js
• llama3.2
• next.js
• prismaorm
• sqlite
• tailwindcss