Anansi

Screenshot of our Unity plugin in Wix
The Unity plugin in our app page
Our mobile home & launch page
The about page in our app
An example of a visual marker for our AR illustrations
Debugging AR markers
Breakthrough!
AR Caterpillar as seen from within app.
Angled view of previous screenshot

What is Anansi?

Anansi serves as a interactive storybook for children. This Unity based web application projects animations of events happening in-story, bringing the words to life and creating fresh interest in reading for younger children. In addition, Anansi translates and projects foreign language words above text on the page, helping readers to develop their vocabulary in English and other languages.

Inspiration

Anansi grew from the realization that oftentimes, parents may not have the time to teach their children a second language or read many books to them in their early years. 2015 statistics show that around 20% of parents read bedtime stories to their children[1]. The solution is to give children an interactive and fun-filled way to learn and grow on their own.

How we built it

In searching for a simple and effective way to provide this, our team chose a combination of Unity and Wix Code. These are typically used for gaming and hosting webpages, but we saw potential in using them together to provide a mobile and web app to achieve our goals. We also aimed to create something that would help aid the children of today to become global citizens, and improving language learning and literacy is an effective way to do so.

Challenges we ran into

A particularly difficult problem that we encountered was in AR marker detection. We initially wanted to attempt to perform optical character recognition (OCR) in order to translate words into events on the fly; however, we realized that OCR was too reliant on receiving a clear image.

We pivoted to using the existing illustrations in picture books as an optical marker for our AR, as they were sufficiently easy to detect even without a clear image. Once we began testing, however, we realized that most of our clipart markers were too round, and thus too similar to differentiate and display the correct visualization.

At last, we moved back to the original illustrations from The Very Hungry Caterpillar by Eric Carle, as they had very distinctive silhouettes. After a brief scare in which none of our AR marker detection worked due to a license error, we were finally able to differentiate our markers and display our AR visualizations!

Another challenge we face was displaying translated words in AR; even though we successfully integrated with the Google Cloud api to gain actual translational functionality, we were unable to display the translated text in the GUI.

Accomplishments that we're proud of

We strove to present a polished product by the end of this hackathon, and we hope we succeeded!

We were able to combine Unity and Wix in an unconventional way in order to make second-language literacy accessible to anyone with a smartphone and an internet connection. Despite many bumps in the road, we are very proud to have successfully created and displayed a "living" story with 3D illustrations.

What we learned

"This was my first experience working with Unity and C#, as well as Open WebGL and Wix. I was really amazed by how easy both Unity and Wix are to use, and I'd love to see how far I can push the limits of both platforms in future projects." - Lydia "This was my first time working with AR and Vuforia as well as Google Cloud platform and Wix. I thought it was really cool the way Vuforia creates a database of images and recognizes them to add AR elements on top. I also liked how simple Google Cloud platform can be to implement into an app regarding translation services. We didn't get to fully implement in our end product, but I was able to test translating with code in the AR space." - Daniel

What's next for Anansi

If we had more time, we'd definitely tackle optical character recognition; on-the-fly translated text AR overlay would be amazing, and would allow us to tackle books that aren't in our database. A really cool extension would be to allow readers to tap on words and overlaid text to hear pronunciation or definitions for difficult words.

We'd also love to give children using our app the ability to interact with the AR animations produced by our app! We'd really love for children to be able to help their favorite picture book characters solve puzzles and save the day.

In additon, we would love to improve the app by testing it on groups of children in different age brackets. We plan to use that research to perfect the application in both learning methods and what the children find the most fun, finalize our method of teaching children, and place it on the market for families and children to bring stories to life.