RemberU

Main Menu
Hardhat Interior
Feed
Hardhat Side View
Login
Face Detection

💡 Inspiration

When meeting new people, especially in huge numbers, it is not uncommon to forget people’s names immediately after–or even while–making small talk with them. As first year students who just went through Orientation, we can confidently say that we are no strangers to such situations. Social events can often be massive, intimidating, and overwhelming, so we wanted to help make such events feel a little cozier!

⚙️ What it does

RemberU is an AI powered device that attaches itself directly to your Waterloo hardhat to make your people-meeting experience as smooth as possible. By reading the lips of those you’re speaking to, RemberU ensures that you’ll never forget or miss important information or have to sheepishly ask “what’s your name again?”.

Our app stores essential aspects of each conversation and takes notes for future reference. Users can access these notes at any time to brush up on conversation starters, and each friend’s file will be updated using facial recognition after subsequent interactions.

💻 How we built it

Hardware: RemberU runs on a Raspberry Pi connected to a hardhat, with a webcam strapped to the front to measure lip movements and record interactions. We guarantee you’ll have the coolest hardhat on campus!

Backend: Our backend runs on Firebase, managing each account’s personal information and conversations. We used Symphonic Labs’ lip-reading API and OpenCV to record conversations, ensuring that our app works even in loud event environments. The Gemini API then processes the conversation, summarizing it into relevant facts and information to be displayed using Python and Flask in the app’s main feed.

Frontend: Our frontend is built using Flutter, and features aesthetically pleasing log-in and register pages, as well as the main feed containing all of the user’s conversation data. Our very own mascot, a teddy bear named RememBear, greets users at our home page and represents the very message of coziness we want to convey.

⚠️ Challenges we ran into

We struggled a lot determining what exactly we wanted RemberU to accomplish. We eventually settled with a scope wide enough to be useful, but not so wide that it would be impossible to complete. Learning Flutter was also a big hurdle for us, as it had a steep learning curve compared to using HTML and CSS, and most of us had little experience in it.

Small challenges such as implementing APIs and fixing buggy libraries and hardware quickly turned into multi-hour nightmares. Putting it simply, Firebase + Flutter cooked us. Our Raspberry Pi also just decided to not work, so we had to improvise last minute. Being ¾ of us’ first times at Hack the North, we were of course faced with the dilemma of sleep or no sleep, but here we are!

🏆 Accomplishments that we're proud of

Integrating the Gemini, Symphonic Labs, and OpenCV APIs together.
Figuring out how to use Flutter and Firebase in such a short period of time.
Building our own model for facial detection.
Submitting this project.

📚 What we learned

Technically, we had very little knowledge of front end development, specifically using Flutter and Firebase together. We also had little experience with the APIs we used such as Symphonic and OpenCV, so we’re very proud of how the final design and functionalities turned out.

Thanks to HTN, we’ve had the opportunity to develop our technical and team skills, which we hope to take advantage of in our upcoming coops and careers. We will certainly look back fondly on this event, and we will cherish the bonds we’ve made, the people we’ve met, and the RBC food tent.

🚀 What's next for RemberU

If we had more time, we would probably consider building a mobile app for RemberU so that it could be used even easier by people at events. This would allow us to use notifications to alert people of any potential updates to content or information.

We could also incorporate the RemberU hardware into other kinds of headwear, as our current design targets UWaterloo Engineering Undergraduates. More professional designs could include making a RemberU tie, a name tag, or glasses.