A conversation at a VR meetup was the catalyst. Eric, a VR audio dev came up to me and said, "It would be awesome if you could make an experience where you can play a marimba in VR!" As someone that studied Marimba and other mallet instruments for many years I agreed wholeheartedly. Marimba was my absolute favorite instrument growing up.
I came into the hackathon as a team of one, pretty much set on making this experience. As the hackathon progressed however, several others that shared the same passion for Marimba and xylophone joined the project. Like myself, I think once the idea was planted, my teammates HAD to work on this project.`
What it does
“Marimba” is a performance platform created so that people can practice, play, and share performances on mallet instruments. The dynamic reactive environments would serve to be a backdrop for broadcast performances. So, if you can, imagine people performing and sharing their content on Twitch like they do “Let's Plays” and game walkthroughs. This game-like creation platform focuses on sound and audio more than visual components for player creation. Among our influences is Minecraft, a game that became popular from people just wanting to share what they made with their friends. More than gameplay (which may be added over time as an afterthought), Marimba will provide people with the tools to create music and soundscapes.
How we built it
We built the experience in Unity, using FMOD as the audio middleware. HTC Vive was the VR hardware of choice.
Challenges we ran into
Turns out the environment was our biggest enemy. The reflectiveness of the floor made the Vive lighthouse sensors have difficulty synchronizing. A significant bit of our time was spent trying to set up the Vive so that it didnt (1) interfere with nearby Vives (2) didnt pick up light reflections from nearby whiteboards and the floor. I guess the Vive wasn't exactly built for classroom settings.
Carpet is better and keep reflective surfaces to a minimum.
We also had trouble with FMOD. The documentation for FMOD and Unity integration is a bit dated.
Accomplishments that we're proud of
Our marimba is playable and sounds really good. As a marimba player I feel like the experience is comfortable and fun! I can spend ours in my Marimba world.
What we learned
I experimented quite a bit with physics in the experience and eventually pulled the majority of them out. I really had to change the way I was thinking of the experience. Instead of trying to match it as close to reality as possible I needed to think about how or if a certain physics rule actually helps me play my instrument. If I can play better without elastic collisions then why add them? Are there things I can do in VR that I can't do in real life that would make me a better performer?
What's next for Marimba
(Hopefully) Coming soon to Steam and potentially other platforms.