AnnotateAR

Inspiration

Imagine learning about Jupiter by interacting with a 5-foot tall 3D model annotated with its facts and properties. Then imagine being able to view the relative sizes, shapes, and revolutions of Jupiter compared to other planets. Augmented reality adds a new dimension to the 2D pictures we grew up seeing in textbooks, and it has the potential to transform the way we learn. 65% of people in the world are visual learners, hence augmented reality has the potential to become a major tool for education.

We built a real time, collaborative platform that allows teachers to annotate the physical world and virtual world as simply as it is to add a comment to Google docs.

What it does

We built a peer-to-peer platform that allows users to "annotate" 3D models in real time. This comes in the form of an iOS application where multiple devices interact with the same world through multi-peer connectivity. There are two modes: a "teacher" administrative mode that can add annotations and models to the world, and a "student" mode that simply views the world as laid out by the teacher. The teacher can place 3D models, arrows, and annotations on the physical world while the student views these placements in real time.

How we built it

AnnotateAR is built on iOS using Apple's ARKit. All devices know what 3D models exist, and display the proper model in the proper location when selected. The devices then share all data and relative locations of added annotations which are also placed within the world. Additional functionality, such as math plots from Wolfram Alpha, were added through a Flask app hosted on Heroku.

What's next for AnnotateAR

AnnotateAR is real time and collaborative - applied to 3D models. While developing this project, we found numerous applications of AnnotateAR's technology. Using a similar concept, product designers and architects can collaboratively come to a consensus about the best approaches and ideas while developing for a new release. It can also be expanded to a physical representation of Yelp where users can view and upload reviews of local businesses by simply pointing their devices at the business. Education is limitless when students can finally see a visualization of 3 dimensional saddles and give data representation with space.