Gesture Controlled Hologram

• 

  Picture of hologram in a well-lit room

• 

  Bird eye view

• 

  Picture of hologram in a dark room

Inspiration

The inspiration for this project is our favourite superhero, Iron Man. Or atleast mine :) His hologram-centric workplace which he controls with his hand gestures looks futuristic and fitting, and of course, very handy. It helps visualise things easier, edit and build quicker, and see both sides of grass. So, it occured to me, why not implement this in classrooms to help students visualise the subject better. Cause at that time, we were learning about organic structures in Chemistry class, and man, I wish I had something to help visualise it better, cause the textbooks just had circles and lines. So, that's what got us "Gesture Controlled Hologram". A device which helps us visualise 3D objects in 3D space, and an exciting step towards visualising quantum science.

What it does

Gesture Controlled Hologram, does exactly what the name says, it gives us a hologram which we can control with our hand gestures. So, we can look at digital 3D models from all angles, and even interact with them!

How we built it

We got the acrylic hollow trapezoidal prism up with a lot of (hard) knife work and angles. The program to recognise gestures and accordingly move the 3D model was the fun part. We made the 3D model work in Unity and had the gesture recognition run with Python.

Challenges we ran into

Fixing the angles of the prism was a bit tough. The gesture recognition part was also relatively wonky, but managed to pull it off.

Accomplishments that we're proud of

The prototype works pretty decently!! The interactions with the 3D model is limited but it works, and I'm proud of it.

What we learned

There were a bunch of learnings but the one that stood out the most was that we should never stop ourselves and always try, cause believing that the project could be made, that thought itself completed 50% of it.

Ideal Product

The ideal product looks like this - there is a foldable LED screen and the foldable prism. Prism is made of acrylic sheets and has hinges at the joints to be able to fold and make space. Now, we install the project app in our phones and connect it with the LED screen. We can choose whichever digital 3D model we want to present on our phones, or even upload our own ones. For the gesture recognition, there are two methods. One, we have a transparent phone holder with a tag and have the phone lie on the chest with the camera facing away from you at your hands. Two, we can connect our phone with a button camera/clipping camera, or even better, if it is affordable, even have cameras on all sides of the hologram. Cause the mission is to make it affordable, interactive and a successful solution. Reducing the amount of space and resources required to maintain physical models, and the cost spent on them is what we're trying to reduce. Along with having easily updating models with perfect ratios and proportions.

What's next for Gesture Controlled Hologram

Have to make it into a usable and publishable app, so that schools can start using them. Also should think about manufacturers willing to make the prism. Have to get a bunch of 3D models (like heart 3D model, organic structures models, 3D shapes and math volume models) in the app pre-made and also should give the option to add own models.

Built With

• python
• tensorflow
• unity