DibuAR

Free Drawing
Nature and Geometric Object Placement

Inspiration

Aside from our personal investment into social reform regarding children with disabilities, there exists much research regarding the benefits of art therapy in improving the socialization and communication skills and calming children with Autism. Art therapy is mostly conducted using physical or tactile art, but given the ubiquity of cell phones and no application listed on the app store regarding art therapy, a readily accessible therapeutic solution where children can free draw and interact with animals and nature using Augmented Reality, an utility that allows us to mimic physical space, would be beneficial for parents, caregivers, and children of Autism.

What it does

Our application is an artistic therapeutic solution targeted towards children with Autism. We have created a venue where children can free draw and interact with animals and nature using Augmented Reality. More specifically, children can free draw in 3D-space using a variety of colors and can place trees, geometric objects, and animated animals on detectable planes. All objects created in the 3D-space are viewable omnidirectionally.

How we built it

We built the application using Unity's AR foundation toolkit. Specifically, plane detection was identified by Raycast Manager in Unity's AR Core and placement of objects and animation was identified and targeted by tap input. Free drawing was created using Unity's line renderer and start/stop associated with touch was programmed. Trees, rocks, shrubs, and animals were created using blender. Animal animations were sequenced in blender as well. UI was constructed with Unity's Canvas and UI elements. UI icons were edited using Gimp.

Challenges we ran into

Having minimal prior experience with Unity’s AR Core, we had to quickly learn the components of AR and understand how AR elements interacted with each other. A theoretical understanding combined with a practical understanding of implementing our ideas was difficult to acquire and physically manifest.

Accomplishments that we're proud of

We are satisfied that we were able to create an all-encompassing technologically artistic therapeutic solution for children with Autism. The free drawing implementation was however our most proud therapeutic element. The animal animation and the spontaneous grass were also significant elements that solidified our therapeutic experience.

What we learned

We quickly became well-versed in manipulating Unity’s AR Core and creating animations in blender and importing them to Unity. Additionally, our understanding of Unity UI design was tested and solidified by creating complex UI connections. We also learned about Unity’s line renderer and understanding how to edit images in Gimp.

What's next for DibuAR

Our goal is to create the most well-rounded immersive AR therapeutic experience for children with Autism. We believe that we could increase our immersive experience by designing animals that walk in the Unity AR space. In other words, when an animal spawn, then the animal can keep walking and moving its position forward. Additionally, we would like to create a more kid-friendly UI to remove any user ambiguity.