Smart Jacket

User interacting with Smart Jacket UI

Inspiration

The project is inspired from Pranav Mistry's sixth sense device which introduced the world to a whole new way of augmenting one’s reality with information. The parts just included a cheap projector and a camera along with a bunch of color bands; things that are too trivial to be able produce such cool device. We as a team of three, inspired by this thought came up with an idea of a futuristic smart jacket with an inbuilt camera and projector.

What it does

The jacket is equipped to get user inputs through a hardware remote and hand gesture. The jacket has a pico projector that can display user apps based on his/her convenience. The jacket has a camera that can take first person pictures. We built a few apps that caters to the smart jacket users such as to scroll through their gallery, take a picture on the go, control a music player and get some weather updates.

How We built it

We tried our best to use most of the hardware that was available in the hackathon that were relevant in the context of a smart jacket. The Makey makey classic is attached near the wrist of the user which helps the user in navigating through the smart jacket apps. Gesture Sense is a proximity sensor which is used to detect the person’s hand gestures. The particle Photon board which is interfaced with the gesture sensor, processes the RAW values and detects hand gestures. The entire app development is done on the laptop which will be on the backpack of the user. The apps are developed on LabVIEW that communicates serially with the photon and the Makey Makey. The laptop’s screen gets casted in front of the user through the pico projector which provides visual feedback for the user in his/her interaction with the jacket.

Challenges We ran into

Learning to work with the Particle photon board and figuring out how to interface it with the gesture sensor took some time. The inbuilt swipe right/left gesture recognition didn’t seem to work well for our case. We tried processing the raw sensor values but were only successful in reliably extracting a single gesture – a gentle tap above the sensor. Establishing reliable communication between the photon and laptop (LabVIEW) took some time too. Positioning the hardware on a jacket was equally challenging too.

Accomplishments that We're proud of

We were able to integrate a lot of different systems within three days and onto a single jacket too. The response time of the system is pretty real time.

What we learned

Almost all the hardware were new to us. It was fun trying out new things that we have not tried before.

What's next for Smart Jacket

Miniaturizing the entire hardware. Having all the processing done on a single board computer such as the raspberry pi would be decent upgrade to our current setup.