BigBrother

Recording of Dorm Room
Site
Timeline Event

Inspiration

Having personally forgotten minor things, like where we left our keys, to having family members and close relatives suffer day by day from memory loss and confusion, we at BigBrother decided to create a product that could act as a second set of eyes, keeping track of all the minor details so that nothing could slip through our fingers.

What it does

BigBrother is an app that enables individuals with memory loss to quickly and effectively recall various events they've experienced. Our custom computer vision system detects any action or event and logs it into a timeline. Additionally, event summaries are provided for each event node that can be referenced at any time. Finally, users can query the data using the chatbox or voice detection, where Gemini will analyze and return the relevant event.

How we built it

BigBrother uses computer vision and Gemini AI to analyze video recordings of events. It automatically detects and logs them on a timeline, and allows users to query past events through a chat system using text or voice commands.

Challenges we ran into

Throughout the hackathon, we ran into hardware issues when it came to our personal devices, and more prominently, several issues with git and GitHub when it came to sharing and merging code. At one point, due to a miscommunication, several files were wiped from the repository with no way to retrieve them; however, they were reclaimed through one of the group members having not pulled code.

What we learned

We learned how to build fleshed out applications with image processing. While one of us had worked with OpenCV briefly, the quirks of the library really showed through while camera functionality was being configured. We also had a big of a nightmare with Git, that forced us to improve our knowledge of version control and Git. This hackathon was also one of our member's first experiences. He learned how to use Git in order to share and work on project files with everyone else. We also all learned to communicate efficiently what we were each working on and how to integrate what. This was incredibly important when it came to pushing and pulling edits in Git.

What's next for Big Brother

Currently, Big Brother works off of free tokens in APIs and runs on locally hosted servers. For Big Brother to truly serve people all over the world, those servers would need to be hosted online, and funding for tokens and domains would be vital. With that, however, the models used could be upgraded to better suit users and process environments faster, thereby improving the rate and quality at which it could assist people.