Anchor

Web App
Camera, Arduino, and Raspberry PI wiring

Inspiration

Watching my friend and his family struggle to care for his grandfather with Alzheimer's, who would ask "who are you" to those around him multiple times a day, revealed a gap in current Alzheimer's technologies. Not only does this constant forgetting often put extra work on the caregiver, it also erodes the patients confidence and social dignity. Addressing these problems is what we aimed to do with Anchor.

What it does

Anchor is a privacy-conscious assistant for both the caregiver and the Alzheimer's patient. The rotating camera module is placed at the front door of the patient's house and uses facial recognition to identify visitors. It finds the information of the identified visitor to remind the patient who is in their house without them having to ask, using their relationship to that person and a memory anchor. The caregiver uses the webapp, where they can access a live video feed and enroll new people to the database.

How we built it

Software: Beginning with the basic idea of the project, we built the features we knew we needed in the backend, starting with the facial recognition software using DeepFace and connecting that to the SQLite database. From there, we implemented the Web App functionality using JavaScript, HTML, and css. Once these were done, we then integrated the Gemini and ElevenLabs API's to allow for the device to speak to the user.

Hardware: We built our system using a Raspberry Pi 5, Arduino UNO, USB IR camera module, and stepper motor with controller. The camera module connects to the Pi and runs firmware code to capture and stream the live video feed to our software interface. The stepper motor, which rotates the camera for a greater view range, connects to the Arduino through a motor controller. We started by wiring the hardware components together, then developed the software for motor rotation and video capture. Then, we integrated everything using a local server to enable real-time communication between the hardware and software interface.

Challenges we ran into

The first challenge we came across was before we even started the project; we had to research and learn about what exactly the main problems are for those with Alzheimer's and their families. We needed to understand not just the medical factors of the disease, but also the social impact it had on them and their families. This required a significant amount of research before we were finally able to think of a non-intrusive solution to the problems.

The main challenge of actually building the device was integrating the hardware and software elements. Although we were able to get both parts working separately, it was very difficult to have them communicate a live video feed seamlessly. Not only was the integration itself challenging, but we also had to ensure there was relatively low latency between them. To ensure low latency, we had to greatly optimize the facial recognition software along with streamlining the WebSocket communication.

We also ran into challenges connecting the Raspberry Pi to WiFi that required multiple configuration attempts and network troubleshooting. The SSH configuration added another level of complexity in authentication and remote access setup. Establishing reliable communication between the Raspberry Pi and Arduino was also difficult, since the Pi only has digital GPIO pins, we had to use the Arduino as a bridge to provide PWM control for our DC stepper motor.

Accomplishments that we're proud of

The facial recognition and database are the features we are most proud of. This was the main challenge of developing the backend, as finding a format to store the face data in then uploading that to a database required balancing storage efficiency, retrieval speed, and face-matching speed.This was a crucial problem to solve as all of these factors were required to minimize latency.

What we learned

We gained experience on integrating an AI-Powered back end to frontend hardware. We learned how to use WebSockets to send the necessary data back and forth while processing information on both ends. We also gained a deeper understanding on both using facial recognition and optimizing its efficiency. Outside of building the device, we gained an understanding of the challenges faced by those with Alzheimer's and their loved ones.

What's next for Anchor

One of the biggest features that we would like to implement is a physical shell for the device. This could mean an encasing for a front door camera or even making it wearable for the patient so it works wherever they go. Additionally, we would like to do more research into the social side of Alzheimer's so that the device can speak to the patient more empathetically and accurately.