ARtful Presentations

Inspiration

We are inspired by the medical record AR app from a previous echoAR competition. We thought it was a great idea to present 3D objects in a virtual setting, so that it’s easier for the presenter to clarify and explain, as well as making it a more engaging process for the audience. We decided to explore what other areas could benefit from such an AR application. We decided that considering the COVID pandemic, teachers could really benefit from an app like this especially in a K-12 setting, where many abstract concepts are taught and the audiences have a shorter attention span.

What it does

Artful creates a virtual classroom environment, where teachers can demonstrate and explain abstract concepts to students through AR visualizations and gesture-based interactions. Teachers can select presentation items to be visualized from our comprehensive 3D object database, including geometric shapes, planets, landmarks, sculptures, art pieces, and other educational materials. Using a face-tracking application, the object is then projected next to the teacher and the teacher can interact with the objects through gestures such as zooming in/out and turning to instruct in a clearer way, helping students to absorb new knowledge.

Link to project slides: link

How I built it

Project building platform: Unity Major Extension: Agora.io Step 1: Create an open channel video call application using Agora’s extension in Unity Step 2: Build a front facing AR Camera App to be able to present through Face recognition is applied to place 3D objects relative to the presenter Hand recognition libraries are used to allow the presenter to flip through their presentations and 3D objects that are being used to present Step 3: Integrate the AR camera rendering functions of Agora into the camera feed of the presenter Step 4: Leverage EchoAR for backend management to deploy across PC, Mac, iOS, and Android platforms

Challenges I ran into

Incorporating multiple different apps to create one app that has all the features we want and navigating through hardware limitations

Accomplishments that I'm proud of

We have a fully thought-out product plan and use cases for our application. We believe our app adds significant value to the K-12 virtual classroom. Given more time, we would be able to build a fully functioning prototype.

What I learned

We learned through this process the benefit of AR in a classroom and the versatility it presents in a virtual class room. We also learned the complexity of building an AR app, especially since it requires 3D knowledge, as well as the ability to leverage multiple APIs with limited documentation.

What's next for ARtful Presentations

Future Expansions: Features: Presenters can add custom 3D objects to their presentations AR notepad projected next to presenters for presentation audience to write notes (into an AR document that is later shared) without distraction Use cases: Business presentations where 3D objects add significant value to the audience