We were intrigued by the Tap Strap and saw a lot of potential for it to improve people's lives. We wanted to make an intuitive gesture-based option to control smart devices for those who are unable to use voice commands.
What it does
Users can point at lights, and use gestures while wearing a Tap Strap to turn them on and off. They can also control their Spotify stream with gestures. It is an alternative control system to voice-based smart home control. Our program can be extended to add gesture controls to any home automation device, such as smart speakers, smart lights, smart thermostats, or even smart refrigerators.
How we built it
The frontend application is an Android application that uses the Tap SDK to create an app that can interface with a Tap Strap. The app detects the position of the hand, and any gestures performed, and sends them to a backend Python Flask server running on a Raspberry Pi. The Raspberry Pi sends requests to Wifi boards to toggle the LEDs attached to them on and off, much line a smart home hub would. The Raspberry Pi also sends requests to the Spotify API based on the gestures the user makes.
Challenges we ran into
The primary challenge we had was figuring out what data to use for detecting the position of the hand. We also ran into problems with determining how some of the values we were using were being measured, particularly whether they had to do with position, orientation, or both.
Accomplishments that we're proud of
It was really satisfying to finally see our project come together and work. It felt awesome to be able to control lights with the flick of a finger and change the song with the swipe of a hand.
What we learned
We learned to use the Tap SDK, and how to develop an Android app. We also learned to integrate with the Spotify API, and communicate among multiple individual devices.
What's next for Hand of God
Adding more gestures to allow for more actions for a device. For example, changing the color of the lights rather than just turning them on and off.
The Hand of God has potential to integrate with smart home devices such as Google Home or Amazon Alexa. With the calibration feature in our app, it is easy for users to use the Hand of God with any set of devices.