Inspiration

I wanted to work with APIs in my backend engineering, and I was able to utilize several in this project. I was also inspired by the well known statistic, "a fifth of Americans are unable to locate the US on a world map." Although this number was proved to be false, the reality wasn't very comforting. In addition, I always find myself very tired in the morning and low performing in my first period class. After doing a bit of research, I found that scientists say that conversation can improve brain function especially in the morning. I wanted my mirror to address both of these issues, a serious and practical issue like American illiteracy as well as a relevant and lesser, but nonetheless important, issue.

What it does

My mirror goes through the six main steps of AI (input in the form of audio, speech to text, etc.) in order to give the user an enjoyable experience. For now, it can be asked a variety of questions ranging from questions about the weather to questions about the meaning of life. It only activates when it can recognize the user's face as well as when the user sayes the passphrase, which in this case is "mirror mirror on the wall."

How I built it

I built it with a Python backend and it utilized a Javascript frontend. Many other installations like Ruby, Homebrew, and OpenCV were used to give a full experience. It first takes in the user input in the form of audio, then it uses Google Speech to convert this audio into text. Then, I created a wit.ai project that works by defining a long list of intents (weather, holiday, time, etc.) as NLU and worked on training the model. Wit.AI helps make the user input understandable by the computer, so that it is able to generate the appropriate knowledge to "answer" the question. I used many APIs to get the knowledge necessary to answer these questions. Then, using simple NLG, it converts these answers to sentences understandable by the user (Instead of just 78 degrees, it was convert it to: it is 78 degrees). Finally, it uses Google TTS to speak the answers aloud. The main backend code was done in "bot.py" in the "Smart Mirror" folder.

Challenges I ran into

I went through issues with OpenCV location installations, as well as other issues with deprecated APIs and such. It was difficult for me to find APIs to use in some cases, so in addition to using RapidAPI, I had to scour different Github communities to find matches for what I was looking for. The installations also took a lot of space on my computer so I had to optimize everything I could.

Accomplishments that I'm proud of

I'm proud I was able to create such a viable product in a short amount of time from scratch. Everything works perfectly and it will be easy to turn this into an actual smart mirror in the future. This is a product I always knew I wanted to make, especially because there are so many uses and problems it fixes in one easy solution. I can comb my hair while listening to a joke, and have a whole conversation with a mirror in the morning to make me more alert. I'm proud that it's so usable and I was able to get it done in time.

What I learned

I learned a lot about open source code and integration of APIs, and I'm excited to utilize what I learned in future projects. I also learned more about OpenCV and other installations that I used, as well as some machine learning topics that came up here and there as I was training my computer.

What's next for Smart Mirror

Continue to add more intents, for example integration with Google Maps in order to be able to give directions. I will also build a cardboard frame and use a one way mirror sheet and old display to make this mirror into a reality easily.

Share this project:

Updates