We really, really, really wanted to promote safe drinking (and make a janky robot).

What it does

Francesco is a robot that uses facial recognition and facial analysis to determine how you are feeling, and if he thinks you feel sad, he will begin to pour you a drink. When he sees you smile, he will stop pouring.

How we built it

The robot itself is built using arduino and C, and we do our facial recognition and emotion analysis using python and OpenCV.

Challenges we ran into

We had initially planned to have Francesco check your ID to ensure that you were 19, however the OCR we used ended up not being reliable enough and ended up getting scrapped for our final demo. We also planned to add Bluetooth support, but the Bluetooth stack on the laptop we used to run our facial detection code was somewhat unreliable.

Accomplishments that we are proud of

We managed to build and train an emotion classifier in under 24 hours, and we also brought absolutely zero components, and had to make 2 runs to the hardware store!

What we learned

OCR is HARD. I also learned that adding in a manual shutoff is generally a good idea (We might have screwed up a waterfall...)

What's next for Francesco

Lots! OCR for Ids, cutting people off after a certain amount of drinks, and adding in emotion detection for when you might have had one too many.

Share this project: