Our team was inspired by the current lack of interactivity in virtual reality, particularly the inability for children and adults to play together in a virtual world. We were also inspired by the fact that children who are experiencing virtual reality for the first time will benefit from exploring the technology with a trusted adult. Many adults are also new pioneers of this medium themselves. We set out to create an experience where children and adults can can explore virtual reality, together.
Careers have become global. Service men and women, and countless others currently can video chat with their family and children, but with TreeHouse anyone living abroad can play with their kids in and help them achieve educational milestones through the game. Other inspirations for this game include that we want to offer incarcerated parents can interact with their children in the beautiful virtual environment that is TreeHouse.
Most virtual reality experiences are one-sided, and currently none exist to facilitate an educational and familial bond between young children and caregivers. Technology has presently allowed text communication and voice, but we want to bring a collaborative and immersive experience to families. Our mission is to ensure that no matter where you are in the world you can connect with your children, because family time should have no boundaries.
What it does
TreeHouse is an educational platform that allows parents and kids to play and explore together in virtual reality. TreeHouse is an educational platform that lets kids access multiple educational worlds. Children start out in a magical tree house space; this is their homebase. From here the player has access all platform activities. Inside, the child meets Mr. Pouf, an adorable robot companion that acts as a guide throughout all of the experiences. We envision Mr. Pouf as a helper for all the different activities throughout the game and a tool to keep children on task during gameplay.
For the purposes of the demo, we have created Exploratorium: the introductory experience in TreeHouse which enables parents and kids to get acquainted with all of the base-level functionalities of the game. In Exploratorium, children can learn about logic and shapes through playing with blocks, expand their creative skills with free-form drawing and connect with their parents to complete fun and educational tasks such as drawing their favorite animal and favorite food.
The parent is able to access the game in the same room as their child or remotely through a mobile phone, laptop or desktop computer. If they want a multiplayer experience, the parent can use a headset if they would like to (any headset can work including HTC Vive, Samsung Gear, Google Cardboard etc.).
Portal Mode access is the second option, to enter the game through our unique portal-style video feed. In this view, children can both see and hear their parent in the game, additionally the parent can also manipulate objects in the game space remotely from their phone or computer. This innovative functionality has endless opportunity for education. One example demonstrated in the game is counting and stacking with blocks. In our Exploratorium, we combine virtual-physical space to building math fluency. This innovative way to teach children spatial reasoning, logical reasoning, and tactile learning offers a sensory experience; which is paramount to early-childhood development.
Parents can also also “gift” their children objects that are accessible from the TreeHouse App Store. These objects are instantaneity placed in the game for the child to interact with in real time, as the parent can observe, interact, or instruct. Additionally parents can also use TreeHouse to teach their children responsible technology usage practices from an early age.
How we built it
TreeHouse was built in Unity Game Engine, heavily using the Unity Networking framework and going into low level coding concepts like sending packets of byte arrays. In order to send video streaming data, we had to take the textures produced by the webcam and simplify them down to byte arrays in order to send them over the LAN network. There were many problems, including being restricted on packet size and syncing commands and object transforms through unity and C#. However, we found solutions to our problems by compromising things like resolution and low latency for more reliability.
Challenges we ran into
As a team, it was very important to us that the functionality of our product is realistic. Initially, we started out with a plan to allow the child and the parent to each use an HTC Vive headset to enter the game and interact with each other. However, we realized that very few households have access to two headsets. Our solution was to allow the parent to access the game (with or without a headset) in the same room as their child or remotely through a mobile phone, laptop or desktop computer.
We also ran into challenges with the video feed for the interactive portal but we were able to reason our way through it to make it a reality!
Accomplishments that we're proud of
Our team is most proud of the technology we developed to enable the interaction between children and adults during the game. Our interactive Portal Mode technology has far-reaching applications for virtual reality far beyond gaming. This technology can be used for learning and training in multiple industries. It can also be used for research since researchers will be able to watch children learning, playing and exploring in real time.
Our team is very proud of the aesthetics and music of the game. We believe that music, when utilized properly, can help guide the experience of a game and unleash creativity. To accomplish this, we developed an original soundtrack for TreeHouse. The TreeHouse design and infrastructure is custom-made by our team from the beautiful wood panels down to the nails on the floor board.
TreeHouse is more than just a game or an experience for children of all ages to have fun, it is a particularly fantastic tool for young children to have their first experiences in virtual reality. We believe young children should be able to see and hear their parent with them in immersive games. Unfortunately, in the educational game market, content and interaction between parent and child is few and far between. With TreeHouse, we’re bringing a product to market that has immediate need and endless applications.
What we learned
Our team learned more than we ever thought would be possible in this time period. We each expanded our capabilities with the hardware and software we used. We also learned how to add a new functionality to VR, with the introduction of our interactive portal-mode technology.
TreeHouse is not just one game, it is a platform for games and interactive learning activities. Parents and kids can access the TreeHouse App Store to purchase more “worlds” for their kids to play in. While we did not have the time to build all of the worlds for this hackathon, we would like to build out more games and experiences for TreeHouse.
Our team would also like to conduct research on how to get the best controller experience for children in TreeHouse. We would also like to conduct further research on education and media literacy in VR so we can optimize our games and experiences in TreeHouse.