Inspiration
In offline gatherings all the guests don't usually just listen to one person, instead they form groups and talk about different topics. I wanted to bring this feature into online meetings.
What it does
It is a online conference call application with some brand new unique features. The guests of the call are in a 3d world where they can interact with each other. The sound from the other users depends on their position and distance from the main user which allows formation of groups without interfering with each others conversations. The live movements of every user are reflected on every computer.
How we built it
- The audio call uses webrtc to transfer audio streams from one computer to another. Webrtc only supports peer-to-peer connections so I used a multi party architecture implementation of SFU called ion-sfu to allow multiple users to join the call.
- Webrtc data channels are used to communicate position in world space with each other.
- Audio based on position in the 3d world is implemented using the panner node of the webaudio API.
Challenges we ran into
- No way to send user data with audio stream: I tackled this issue by creating a separate webrtc data channel which sends a unique id to identify the user along with position data to make the necessary updates in the 3d world.
- Managing complexity: I solved this by implementing audio calling and movement of characters separately and then joined them together and added dynamic sound. This method really helped in minimising the number of bugs.
Accomplishments that we're proud of
- Realtime movement of guests in the 3d world
- Establishing audio calls using webrtc
- Dynamic audio which changes based on relative user position
What we learned
- It was my first time using webrtc so I learned a lot in terms of video and audio conferencing apps
- Learned some new concepts related to realtime movement in games
- Managing complexity and modularising code
What's next for Searr
- Add filters to make audio sound cleaner and remove white noise
- Improve UI/UX
Log in or sign up for Devpost to join the conversation.