Current 3D modeling software costs thousands of dollars for any enthusiast to use. We wanted to create a simple, easy-to-use, and cost-effective way to quickly model and prototype our ideas. A software that allows users to efficiently create and build in 3D space would be incredibly useful to budding engineers and designers alike around the world. We decided to apply mixed reality towards solving this problem, providing a solution that allows users to simply wave their hands to build anything they choose.
What it does
Build.Me is a mixed reality system that allows engineers and designers to build and create 3D models using the power of your own two hands. It creates an artificial environment where the user can move their hands under the Leap Motion to effortlessly create, rotate, and scale different objects. The user can choose from a variety of different objects to build whatever he/she chooses, switching between different objects and textures using voice commands. Finally, the user can save his/her designs to a Vuforia object, rendering it and viewing it augmented reality at their choosing.
How we built it
We used the Unity engine in collaboration with the Leap Motion to implement our mixed reality solution. The Leap Motion uses infrared scanning to detect hands and fingers, which can be inputted and used in the Unity engine. Once we were comfortable with using the Leap Motion with Unity, we worked on using hand motions to create, scale, and rotate objects. This involved using vector calculus to perfect the physics and movement objects within our artificial environment. Using Amazon Transcribe, an AWS service, we used voice commands to change the object or shape that we were manipulating. Finally, we allowed the user to save designs by mapping the 3D user-created objects to an image so that Vuforia artificial reality engine could view their creations in the real world.
Challenges we ran into
Vector Calculus is hard. Physics is hard. Rotations are hard. Scratch that. Rotations are nearly impossible. But then again, as our beloved president John Fitzgerald Kennedy once said, “We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard.”
Accomplishments that we're proud of
Getting rotations and the rest of the mathematics to work was perhaps the greatest obstacle we faced.
What we learned
We successfully learned how to use the Leap Motion, having no experience in mixed reality previously. We also successfully implemented the Leap Motion in Unity and were able to quickly learn the ins and outs of Unity in order to suit our tasks.
What's next for Build.Me
We want to add the ability to simulate different environments, like certain weather conditions, wind, and natural disasters, to user-created objects. We also want to increase the database of objects that the user can call into existence. We also want to build an Android/iOS mobile app that we can use to more easily take advantage of Build.Me while toning down the use of hardware. Finally, we want to push for better physics and graphics. In the long run, we hope that one day Build.Me will be used by engineers and designers worldwide.