The recent times changed a lot about how things are organized and processed, and the effect is no different in case of music industry and casual music jam sessions. Music artists have been struggling to conduct their jam sessions or concerts in an efficient hybrid manner because the conventional platforms for video/audio conferencing are not very well optimized from the concerts' point of view, having various issues, some of them being:-
• high latency
• low quality of audio/video
• plain raw UI/UX
• no added personalization
• no custom virtual instruments to play
So this was our inspiration and that's how we came up with chordz, a web based platform to organize and attend hybrid music jam sessions, from literally anywhere and on any device, with best quality!
❓ What it does
Chordz is a web application that lets users create or attend online jam sessions. Its an audio conferencing application, but very much themed up towards music sessions and optimized well enough to overcome the normal barriers. As a plus point, we've 2 default virtual music instruments available in chordz, for users to play, namely:-
Any performer can play them or they can use their physical instruments as well.
Basically any user would need to login/create an account on chordz. Thereafter, one member of the performer team would create a new jam room, they'll fill in the requirements of what instruments they'll need virtually, and then they'll get a jam room created! After that, they can easily share the details of this room using in-app share feature, and then any common user who'd like to attend the music session can just enter the room code and they'll be redirected to that specific jam session. The attendees would have the permissions to listen to audio, and the performers would have the permissions to mute/unmute themselves, play virtual instruments and end the meeting. The main aim to create it was to integrate the hybrid environment correctly and letting people have a sense of place, instead of a boring screen show.
Attendees can also show multiple reactions to express how they're liking the session. It goes like this:-
🎼 How we built it
• For the basic process of authentication, storing user's data (profile picture URL, email), we used Firebase's Auth and connected it to our react application.
• For making the virtual instruments, we used simple logics and got the free to use sounds for these instruments and implemented them in our react components.
• For creating virtual conferences, passing music audios, letting users join, and everything around that, we used Dolby.io's APIs with functions involving creation of audio conferencing room, mute/unmute permissions, sharing screens (multiple at a time) and leaving meeting.
• For deploying, we deployed our react app on Netlify using Netlify cli, the website is live at https://chordz.netlify.app
• For designing the mockups and logos, we used Figma and Canva, and for editing the demonstration video, we used Alight Motion (android edition)
👀 Challenges we ran into
We ran into a couple of challenges while making chordz web application work. The APIs we used were relatively new to us so we took a while to familiarize ourselves with it and debug our code properly. The major thing we struggled with was finding a way for transmitting and synchronizing music from virtual instruments in real time. Initially, we were trying to keep a track of keystrokes and render it on the client side, but we realized that it was really slow and out of sync, thereby affecting the experience drastically. So we changed our approach and ended up implementing multiple screenshares without rendering the video (to increase efficiency), and after playing around with permissions for a while, we were able to successfully implement this feature. Also, we used a new library for CSS (tailwind CSS) instead of the normal CSS, so that was also a bit intriguing but fun enough!
💫 Accomplishments that we're proud of
Our team is proud that we were able to complete our project and we made something that we think could really impact a lot of people in a very positive way. We're also very proud that the following features are totally under working condition:-
• creating new room
• joining room
• streaming audio
• playing virtual instruments
• leaving room
• seeing participants
📖 What we learned
We learnt a lot throughout our course of hackathon. First and foremost, we got familiar with dolby.io APIs and saw some of the cool features that we'll potentially use in the future as well. Apart from that, we learnt how to make responsive designs, give them a royal look, using hooks in React.js, deploying web application on Netlify, and more!
📈 What's next for chordz
To be honest, we saw the capabilities of different APIs and how they can be integrated in chordz to make it even better. For example, we loved the Spacial Audio API, and using it in chordz would make the experience even better, so we'll be looking forward to that. We would also like to make it more interactive and give some more customizable controls to the organizers of the music jam sessions!