An innovative audio visualizer specifically designed for the GEAR VR. This is being created using FMOD for Android (for audio analysis and playback) and Unreal Engine 4 for Mobile. Unreal Engine 4 was chosen because the ease of integrating C++ code.

A major flaw I have noticed with many audio visualizers is that they are either unrepresentative of the audio being played, or they are very difficult to integrate into other programs. Poor integration and poor representation greatly hamper the VR experience, as VR technology is dependent on immersion and interactivity, which cannot exist if the visuals poorly model the audio. First and foremost, this will be my goal: to use my dual background as a Computer Science student and a Music Technology student to create one of the most immersive Audio-Visual experiences to date.

That alone is not innovative, however. I intend to make the audio interactive as well, so that reverb and other effects can be added in real time, altering the music and, ultimately, the visuals that result from it. This is where the name Thesia (clearly based on the condition Synesthesia) comes from: Thesia is an environment in which every change in sound can be experienced as a change in sight as well.

I have familiarized myself with the limits of Unreal Engine and FMOD and am currently designing the parent classes for the visualized objects. If members of the community know of any particularly interesting real time processing APIs and middleware, I would love to know of them. My current priority is simply making the objects work in a way that isn't too resource heavy on a galaxy note device, as latency would significantly hurt the experience.

//Milestone 2 Additions: There is now a central controller in the center of the stage which will ultimately allow the user to alter parameters while the visualization is occurring. Object spawn points have been placed at eight strategic locations. These will organize the visuals into different zones encompassing the player.

Visualization is not yet fully functional but work is under way. I will be posting a video once visualization works successfully.

Built With

Share this project:

Updates