Currently, the United States is suffering from a shortage of scientists and engineers. According to recent studies, only four in ten students choose to pursue STEM career paths, with even lower numbers for women and minorities. Thus, in order inspire an interest in science and technology in the next generation of students, we decided to create “Blast Off Into Learning!: AR Planets Experience”, an augmented reality Solar System flashcard deck and corresponding android application which teaches students about the planets through a unique and interactive experience.
With an easy to use interface, our app gives students a new way to engage with STEM study materials, as hovering over each Solar System flashcard triggers a state-of-the-art augmented reality (AR) 3D Planet Model and corresponding Planet fact card, while a narrator gives details about the Planet makeup and origins. This app is primarily aimed at Elementary school children who are learning about astronomy for the first time in their Science classes. By getting students interested in science and technology at a young age, we can help close the United States engineering shortage, as well as the minority and gender gap in STEM.
What it does
“Blast Off Into Learning!: AR Planets Experience” is an augmented reality app that works with a deck of Solar System flashcards (linked below), to provide students with a unique new way to engage with STEM study materials. The flashcard deck contains contains all eight planets in our solar system including pluto, and hovering the android application over each flashcard reveals an extremely realist, spinning 3D AR Model of each Planet along with a AR Planet fact sheet, while narrator audio details its makeup and origins in the background. The augmented reality features are dynamic and move/resize as the flashcard is picked up, moved around, and away from the phone camera. This application truly provides a unique and fun way for students to engage with their STEM study materials through the use of technology.
How we built it
We built this application using Unity with Vuforia and C#. The models of planets were made using textures from the NASA website, and the audio source and planet facts were all found online as free sources. The Solar System flashcards were purchased on Etsy as digital downloads. We believe that the combination of audio, hyper-realistic accurate AR animation, and straight information can create a extremely engaging learning experience for students that will not only teach them astronomy/science, but get them interested in the technology behind this educational program.
Challenges we ran into
One of the challenges we ran into was dealing with the learning curve for Unity and Vuforia. Learning a new UI and AR platform took a lot of our time in the beginning. Specifically, Vuforia requires some very detailed instructions that were not always given in the software manuals. In addition, we struggled with using Github for our Unity project because we were dealing with merge conflicts in 3D models not with code, as usual. We were eventually able to resolve these issues through lots of googling, blood, sweat, and tears lol.
Accomplishments that we're proud of
This project was our first time using Unity and C#. As complete newbies to this gaming engine and the augmented reality (AR) platform, it was really a challenge to master these tools, but we are so SO happy with the outcome of our project.
In particular, we are really happy with how realistic the Augmented Reality models came out, after a lot of trial and error with different planet textures found online, we decided to go with the images on the NASA website and the stunning AR result truly exceeded our expectations.
As this project comes to a close, we truly hope that the application we created will be able to inspire an interest in science and technology in young people through both its subject matter and the technology which was used to build it.
What we learned
During this project we learned a lot about how augmented reality works and also about the computer vision behind it. By using an API as advanced as Vuforia, we were able to work with the best image tracking tools available allowing us to push the envelope on the dynamic features and movement of our 3D models in relation to our image targets (the flashcards).
In addition, this project allowed us to gain experience with Unity which we found to be a much stronger android app development tool than Android Studio due to its basis as a game development framework. Though tricky to learn at first, we believe that the experience we gained with Unity will be a great basis upon which we can build hyper-realistic and smooth games and apps in the future.
Overall, we are very happy with our learning experience during this Hackathon and hope to use our newfound skills to create even more amazing applications in the future!
What's next for Blast Off Into Learning!: AR Planets Experience
Due to the hackathon time constraints, we only had time to animated Venus, Jupiter, Neptune, Mercury, and Earth. However, we hope to include all planets in the future. Outside of the Solar System, we hope to someday (or maybe at another Hackathon? XD) make a similar augmented reality flashcard project aimed toward Medical students. In particular, we hope to create an augmented reality flashcard deck with flashcards and 3D models corresponding to different bones or organs of the human body to help Medical students better prepare for their anatomy exams.