EmotionsForDori

Face detection
Email Joy

Inspiration

Some people need a bit more connection between friends and longtime relationships. The distances make real connection difficult. Often, facetime or other real-time messaging app is the only way to see your loved ones. Wouldn't it be great, if you could savour the best moments of your facetime? Now you can with Emotions for Dori.

What it does

It is an application that runs in the background as you facetime your loved ones. It frequently takes image snapshots of your face and it uses machine learning algorithms and image recognition to detect your emotions. If, during an especially wholesome moment, where your smile is the brightest and your happiness shines through, our application saves the snapshot and automatically sends the picture to your loved one's email account so that they may relive that special moment. Just to make sure they got it, the application sends an SMS just to inform you that the wonderful photo has been successfully saved.

How we built it

We used Visual Studios with OpenCV which operates the laptop's webcam at optimal conditions. Then we used Vision API from the Google Cloud Platform to do face detection and emotion categorization. To ensure that we only save the best of the best pictures, the joy detected must be "Very Likely" with over 96% confidence rate. Then, with integrated Gmail API, we sent your loved one that picture. We also used Twilio API to send a confirmation notification to your loved one's phone.

Challenges we ran into

Working with email and vision api permissions slowed us down. Additionally integrating with open CV to take and manipulate images didn't work perfectly on all machines.