Moodify

• 

  An image of the mountains in the Moodify environment

We bring you Moodify, a VR environment simulator for mood improvement, targeted at helping increase productivity, promoting wellness and calm mental states. Moodify highlights and utilizes the powers that music and imagery have to indicate, influence and improve mood. Mood is indicated by the music playing, and appropriate images are fetched by the simulator to augment the original simulated scenery: a quaint art gallery nestled in a grove.

The simulator starts by processing the music that the user is playing and uses a mood estimating algorithm to assess the mood of the user and decide what kind of images to add to the gallery. The mood estimation algorithm and image retrieval, along with the surrounding python functionality were implemented by Aisha, based on a pair of research papers, using the Librosa python library for audio processing. The virtual environment was implemented by Aditya and Soumyaraj, who, respectively, designed the architecture and controls of the simulator using Unity and C#.

Like any enterprise that sets out to make a mark, Moodify also faced its share of setbacks. The difficulty of matching image mood estimation and music mood estimation was compounded by the unavailability of open image and art datasets annotated by mood. Further, the lack of frameworks enabling simultaneous execution of C# and Python processes proved to be a major hindrance to integrating Unity's C# scripts with our Python data analysis scripts.

The biggest knowledge takeaway for the group was the experience of creating and managing both the hardware and software aspects of a virtual reality application. Being music aficionados with diverse tastes themselves, building Moodify's mood classification framework was also a great opportunity for the trio to see their subjective approaches to classifying sounds they love be reinforced (and even, countered) by the quantitative techniques recorded in literature.

In the future, we would like to implement functionality to pick music based on the mood, implement more detailed and diverse environments based off of mood and specific paintings associated with every mood. We would also like to explore using style transfer from renowned artworks to generate the aesthetics of the simulator environment.

Built With

• c#
• oculus
• python
• unity