Inspiration
Everyone's heard or experienced the misfortune of forgetting a birthday. In fact right as the hackathon was starting, we realized we had forgotten one of our friend's birthdays in the rush about CalHacks. That's why we created ForgetMeNot, a way to remember when your friends birthdays are, and to make them great gifts at the last minute, so that way you can gaslight anyone into thinking you spent weeks preparing!
What it does
ForgetMeNot serves to recognize when the person you are looking at is upset, and tries to figure out the cause. Whether it be an anniversary, birthday, or they're just a little under the weather. No matter the cause, ForgetMeNot has the solution for you! Whether it be a custom love letter, teasing birthday collage, last minute flower delivery, or even a custom 3d printed gift, just for them! No matter the reason, or situation, there's something on hand.
How we built it
Firstly, we used Deepface for facial recognition and emotional analysis. We would match them with the identities and dates of those in a scraped past year's google calendar and your phone's contact list and photos. By analysis for matches with occasions, we could determine whether it was because of a missed occasion, or maybe just something small. Based the level of anger or frustration they showed on their face, and the amount of time left in the day, we came up with a doom's day esq countdown clock, so you could figure out how long you had to come up with your definitely not last minute gift.
Depending on whether this was shorter or longer, we used Deepgram's emotionally intelligent AI to give suggestions about whether to comfort your partner, or gaslight them to buy time. It not only generated the text, but was able to say it aloud to one's earbuds to allow them more discrete advice.
Using Groq's chat completion features and context window system, we were able to feed in the data from the emotional analysis and facial recognition, and sent it to the Llama 3 8b generative AI model. After prompting, it was able to provide concise and heartfelt messages to replicate a love letter.
Meme Video Generation -OpenCV and ffmpeg (Continued to apply emotional analysis to determine which photos to use)
Scraped flowers for multiple occasions from BeautifulSoup to highlight and recommend flowers that can be bought for 4 hour delivery. The flowers were recommended by looking at the occasion. Through utilizing reflex's redirect button, we were able to send our users directly to the site to complete their purchase.
A similar process was applied for assembling and showcasing the 3d print files. We used a 3D slicer to computer the time taken for files to be printed.
Challenges we ran into
Some of the challenges we ran into, were around adopting a new framework for our front end. Reflex made some tasks simpler, however it also made it difficult to learn and deploy more advanced and interactive Front End Elements, such as a 3d Web based STL viewer like Three.js
Another challenge was that we had initially not received AR glasses, however we bootstrapped our own setup with wireless headphones, a camera, and our own glasses.
Accomplishments that we're proud of
The product itself was genuinely entertaining, and we were able to laugh at, and find uses for the majority of the content and gifts ideas and products that came out of ForgetMeNot.
Furthermore, there was a diverse range of tools used in building this, such as *facial recognition, text-to-speech, NLP, webscraping, AI based recommendations, and a brand new front end framework. * Despite this, we were able to successfully integrate the majority of the technologies we set out to incorporate.
What we learned
We gained a greater grasp of facial recognition and tuning, as well as exploring Reflex and Deepgram, which were completely new tools to us.
What's next for ForgetMeNot
From here, ForgetMeNot can be expanded to design custom STL files for your partner instead of recommending from a pre-existing collection. Furthermore, integrations with a google photos API and google calendar would allow a more dynamic recognition of events, while reducing the set-up time for users
Log in or sign up for Devpost to join the conversation.