We are entering an era where IoT devices will be all around us, in our apartments to our car. The way we interact with these devices is using mobile applications which can be quite cumbersome, as we won't always have mobile devices with us. We wanted a technology which recognizes our commands through speech and understand our intent and act accordingly. How cool it will be to have a voice-controlled light bulb to follow your commands instead of the old and boring switches.
What it does
HeyIlumi provides seamless voice interaction with Ilumi bulbs. It uses Natural language processing to analyze your speech and understand your intent and follow your commands. Imagine, you say 'HeyIlumi, I am in a romantic mood', and the bulb would change its lighting to suit a romantic mood.
Wana hear more? You can also give a name to your bulb and let's say you are on your way to home and want to romantic mood already set, you can just tweet on your twitter profile and HeyIlumi will follow your commands. When you reach home, the ambience will be ready.
How we built it
We built an Android application which constantly listens for your commands to HeyIlumi. We used Google's Voice API for speech to text conversion and then applied NLP algorithms to fetch the intent. For the twitter integration, we used Twitter4j along with Java.
Challenges we ran into
The Ilumi bulbs have very limited flash memory and can only interact with Bluetooth. We faced issues when the Bluetooth strength was not full, due to interference of nearby devices. We had to tweak Ilumi's API a little to get our work done. The most important challenge was processing speech for supporting natural languages.
Accomplishments that we're proud of
It was our first time interacting with a smart device (bulb) and it was a great experience. We got hands on experience using NLP algorithm. Our proudest accomplishment is building heyIlumi so smart, that it understands commands not only in a direct way (Turn blue) but also in an indirect way (I feel like being under the ocean).
What we learned
Programming application for Android device, Google voice API, twitter developer library and Natural Language Processing.
What's next for HeyIlumi
Integration with more smart devices, like Android TV, Apple TV, Amazon Alexa