3D modeling software poses several difficulties due to the overwhelming amount of tools, controls, time, and effort required to create a design. Existing applications such as AutoCAD, Maya, and Cinema4D pose high barriers for beginners who need these tools in order to bring life to their concepts. Consilio, meaning 'design' in Latin, aims to combine create a seamless experience in ideation through incorporating intuitive controls with 3D models to allow users to design the future.

What it does

Consilio is a web app that creates an interactive workspace, combining a virtual 3D studio with real life gesture controls. Hand motions are recognized and recorded through a Leap Motion. The motions are then translated into controls within a 3D workspace on a web app, and allows for saving of design projects.

How we built it

Consilio uses Leap Motion combined with Xesto Wave API to track hand movements, and record specified motions as gestures. We created an online 3D workspace using webgl, also using Firebase API to host and provide an authentication system. We translated gestures into controls through JavaScript to allow movement of modeling blocks in the workspace using hand motions.

Challenges we ran into

Implementation of the Xesto Wave API took much more time than anticipated due to many cases of troubleshooting and server issues. Combining the Leap Motion data with the Firebase API, front-end display and webpack was another challenge with creating a server instead of using a local host. Several fixes and edits were needed to build and pack all the programs in order to work well on the web.

What we learned

API implementation can require lots of time depending on the program and the intended use. We learned how to utilize API such as Xesto Wave and Firebase, in combination with the Leap Motion hardware and positioning API. We also learned the steps to host and deploy a web app through Firebase authentication and database. Additionally, we learned to create a 3D environment and render models. Finally we learned to combine different API and aspects of the project together.

Accomplishments that we're proud of

The entire process consisted of new experiences, so learning and successfully applying Leap Motion & the Xesto Wave API is an accomplish in itself. We are also proud that we were able to build our own 3D interactive environment from scratch. Additionally, overcoming some confusion and errors when using API or web-packing required critical thinking and problem solving as well. Merging all aspects of the project successfully was quite challenging but creating a functional end product in short span of time is definitely our source of pride.

What's next for Consilio

Consilio is currently only comprised of simple models and limited gestures. Moving forward we aim to include many more basic models and a toolbar with all, such as a simple wheel or axle. The display and graphics could also be improved, such as making a more detailed model for the motion tracker. Adding more gestures will also increase the functionality, as well as ensuring that each gesture is distinct and will not be mixed in recognition. Our main goal is to push Consilio as an open collaborative software which can be used in almost every field imaginable. From engineering models to biological diseases, Consilio will provide an intuitive platform for all people to make a difference with their designs.

Share this project: