Every student who has studied chemistry must have run into a particular situation: trying to interpret the structure of molecules based on a chemical formula. Any molecule drawn on a two-dimensional paper may not seem so interactive, which inspired us to integrate augmented reality and gesture control to enhance that learning experience.
What it does
Myonchemist works with Myo to provide users with gesture controls as a means of interaction with molecules, which are visualized using the Vuforia AR features. Users can perform simple gestures wearing the Myo wristband to zoom in, zoom out, and rotate the molecules in an AR experience. Moreover, further detailed information regarding the molecules, including electron pair geometry and molecular geometry, is included in the app.
How I built it
Grounded in Unity, the basis of Myonchemist consists of GameObjects of molecules. After making the molecules, a simple UI was also made in Unity. Once the GameObjects and UI were ready, we integrated the Vuforia ARCamera to visualize the molecules. The last technical step was to integrate Myo into Unity in order to enable gesture controls over the zooming features of molecules. Combining all these elements together, Myonchemist was made to enhance the learning experience in chemistry.
Challenges I ran into
Throughout the entire process, the challenges were primarily associated with the integration of Myo into Unity. Without any official documentation on how to use the Myo SDK in the Unity environment, we had to figure out how the built-in functions work by guessing, experimenting, and googling. We also ran into problems when trying to combine myo-controlling scripts wrote in C# with unity objects.
Accomplishments that I'm proud of
First, based on careful research of the precise data of bond lengths of particular molecules, we made the GameObjects in Unity with great detail and precision. Second, we came to a realization on how much potential unity engine and Myo wristband have and successfully made something productive out of them by constantly failing and trying. The most important part is we really made it as a team, squeezing this complex project into a narrow time span of 36 hours.
What I learned
Chemistry wise, we learned the properties and visualization of various chemical molecules. During the developing process, we had an overall understanding of the mechanisms behind objects' rotation and material changing, the interaction between Unity and outside plugins or SDKs like Vuforia and Myo, as well as the app building functions of Unity. Our path of learning was filled with many setbacks and challenges,but we enjoyed that.
What's next for Myonchemist
There are two major future plans for Myonchemist: adding chemical reactions and integrating more gestures in controlling the reactions and visualization of molecules. Adding chemical reactions would require changes to the UI and the adding of animations. Furthermore, it would require more gestures working with Myo in order to seamlessly control the reaction process.