HackMirror

HackMirror is the smarter smart mirror. Fitted with facial recognition, voice commands, and voice calling functionality, and live news and environment updates, this mirror extends smart home functionality and accessibility.

img_3054

Built at HarkerHacks 2018.

HH Shield

About

Take the mirror hanging in your bathroom or in your dresser. Imagine if you could do more than just see yourself with this pane of glass - for example, talk to a friend, get a glimpse of the surrounding area, or connect with the world. We bring you the smart mirror - a technology that does just that in a convenient fashion.

Aside from a personalized smart mirror display, the mirror offers the day’s top news headlines, as well as the top Reddit posts of the day. It also gives you a convenient number fact every few seconds, just to keep you on your toes.

HackMirror takes user interface to the next level. While touch screen on a mirror might be inconvenient, the smart mirror fully supports Amazon’s Alexa voice control technology, so you can ask it things like: What’s the weather forecast for the week? This integration allows the mirror to be a non-intrusive method for controlling other smart home features,

Another proprietary feature is facial recognition, allowing for a fully personalized user experience. As you stand in front of the smart mirror, your face is recognized and matched from a database of faces, formed from an initial training setup. With appropriate permissions granted, the smart mirror is then capable of displaying your own schedule and upcoming events from your profile after it recognizes you.

Beyond just interacting with your personalized data, the smart mirror can communicate with others. Integrated with Alexa, the mirror has an ability to make voice calls between two people using RTC.

Technologies

The user interface of the mirror was developed using vanilla Javascript. Using a multitude of free APIs, we were able to retrieve the data (customizable by user) that is set on the dashboard. The program is run on a Raspberry Pi 3, which powers a stripped monitor laid behind the 2 way mirror.

We integrated Alexa Voice Services (AVS) to make the HackMirror truly IoT. Beyond Alexa’s functionality, we used PubNub for voice call integration, supported by a custom built Alexa Skill.

For facial recognition, we used OpenCV. Because the Raspberry Pi is not powerful enough to run the processing algorithm to be done client side, a picture of the camera is sent from the Raspberry Pi to a laptop using Firebase, which will then identify the user, and set the respective profile on the mirror. This processing is done live and repeatedly, so the profile will update respectively if the user changes. In addition, there is multi-user support, so the mirror will recognize if there are two users currently using the mirror.

Built With

Share this project:
×

Updates