Let’s say after a long day of work/school you finally come home feeling exhausted, sad, angry or maybe even happy. As a kid, your mother was always there for you. Somehow knowing exactly what you feel and being able to cheer you up. For this reason, we decided to invent a virtual assistant to create a smart speaker that was unique from all others. A speaker that would have the capability to predict what the user wants before they even have to ask. The speaker would predict music that corresponds to your emotions. Studies show that the psychological effects of music are powerful. Music has been used to promote emotional health and help patients cope with stress. We believe that this will have a large impact on society to improve emotional states.

What it does

The solution to this problem was to have a home speaker is able to read emotions as soon as you enter the room and respond accordingly with the proper music for how you are feeling. Along with this, the speaker will have an array of other features: the ability to detect fire and extinguish them with sound waves, voice activation capabilities, app interaction, it is also able to detect when the user enters a room to welcome them and begin to play music.

How We built it

Hardware: We created a box to house all the components of mamaBOSE. Included in the hardware we used ESP8266- WIFI enabled microcontroller to control mama bose. We also used a PIR (Passive Infrared Resistor) sensor and MQ natural gas detector with some LED lights.

Software: To program the machine learning algorithm to detect facial emotions we used Python Programming with OpenCV, Fisher-Face Recogniser. We used XML files and face cascades in order to help train the algorithm to be more accurate. We interfaced with the BOSE SoundTouch API through post requests using the XML framework to the location of the media information stored on an internal server.

Challenges I ran into

Raspberry Pi---originally we hoped to use a Raspberry Pi to control our mamaBOSE, however we found that compiling openCV was much harder than anticipated. Voice Recognition--we found difficulties uploading pyaudio to the raspberry pi, uploading the packages took much longer than anticipated and was unsuccessful. However, the mamaBOSE still has speech recognition capabilities.

Accomplishments that I'm proud of

Accomplishments that we are proud of include extinguishing a fire using sound waves, integration of hardware, being able to detect facial cues, and successfully playing music that reflects that your current mood.

What I learned

We learned how to train models for our facial recognition software with openCV and how to use machine learning with facial cascades. We also learned how to use the BOSE APIs and sending post requests to it from thoroughly reading their documentation. We also learned how to use arduino sensors and incorporate with BOSE.

What's next for mama BOSE

In the future, we would like to get the Raspberry Pi to communicate with the BOSE speaker. We would like to use thermal frequency to better distinguish if there is a fire. We would also like to be able to detect more emotions and have preliminary data on when the user wakes up and when the user sleeps to play specific types of music. We want to also add capability to other music platforms.

Built With

Share this project: