Spherecraft

Unity development environment
Spherecraft World
Interacting with a sphere created from the voice command "make money"

Inspiration

As students, we spend a decent amount of time studying and trying to recall lots of information before examinations. It is often difficult or time consuming to do rote memorization, so we often seek ways to study more effectively, such as mnemonics or memory maps.

In this project, we chose the latter to produce interactive visuals to easily recall information or tangible objects. One notable example is to “make math equation”, which produces a sphere whose texture features some notable equations such as the quadratic equation. As a result, this project provides an innovative approach to encourage creativity in education and entertainment.

What it does

This project uses virtual reality with Oculus Rift and Leap Motion to allow the user to create a world full of spheres representing tangible objects or ideas. The user can use the Leap Motion’s hand gestures to interact with the spheres to further enhance the user experience, such as grabbing and throwing the spheres away! Hand gestures are also used to move the player around using the touchpad on the left hand.

With speech recognition, the project supports voice commands to create spheres (“make <something>”) and change the size of the spheres (“increase”, “decrease”).

Technical Details

We used Unity 3D to develop the map and C# to write the scripts. To create the spheres, we used a sphere prefab with a texture to be rendered using WWW.texture. We used other WWW functionalities to call a GET request from the Google Custom Search Engine API to obtain the link to the image.

For speech recognition, we used UnityEngine.Windows.Speech as the basic implementation of the algorithm, which we refined to integrate with the Google Custom Search Engine API more properly.

In the realm of Virtual Reality, we used the Oculus Rift as the core functionality and Leap Motion to accommodate user interaction with the environment. Specifically, we used raycasting to allow the user to select a sphere to interact with, and added physics and collision to properly grab spheres and throw them.

Challenges we ran into

Although our team had some experience with Unity, we ran into compatibility issues that we were unaware of when integrating Leap Motion with Oculus Rift, which we resolved by downgrading Unity to an older version to safely accommodate the features.

We also had some difficulties implementing speech recognition, so we modified the algorithm to produce more accurate results.