Spacial Audio Conferencing

Inspiration

Tired of boring meetings with small squares on Zoom? We certainly are. In real life — in a group setting — you can have small side conversations with friends or colleagues. In Zoom, on the other hand, you have to separately message or call your friends, absorbing your attention. Shouldn't there be an in-between?

For Spacial Audio Conferencing, we concluded that spacial audio is how sound is meant to be, making any video call more interesting and productive!

Overview

Our project for LA Hacks 2021, Spacial Audio Conferencing, is an adaptive spacial audio communication tool for calls, meetings, conferences, and talking to friends. Spacial Audio Conferencing adds spice to video calls, immersing users in the spacial audio world.

What does our app do?

Spacial Audio Conferencing serves as a high-quality video conferencing application. We implemented peer-to-peer video and audio data tracks using WebRTC and developed a distance algorithm to calculate space between users and their environment. The algorithm automatically adjusts sound volume between users for an immersive feeling. Users can build their own rooms using the wall placement/removal tool, which automatically adjusts output volume based on user proximity.

Want to ask a friend a question? Simply move aside or to another room. Want to form small groups that aren't talking over each other, but can still hear the gist of the other conversations? Spatial audio is the solution!

How we built it

We began the project using a Table, however decided to switch over to raw Canvas for a smoother experience. We designed and implemented our own pan, zoom, and move effects for the environment.

We used WebRTC for our video and audio. The WebRTC handshake was facilitated through our own signaling server, which allowed us to take total control over who people connected to. We used Google's public STUN server to handle our ICE candidates.

We created a Click to Deploy button using Google Cloud that automatically configures the necessary services for a working tool in approximately two minutes.

We chose Big Table for logs due to its ease of deployment, close integration with other Google Cloud services, and Wide Column structure.

Other technologies include:

  • SvelteJS for the frontend
  • Sapper, ExpressJS, and SocketIO for our backend NodeJS
  • Cloud Run for our Docker container

In Review

What were our challenges?

Spacial audio was no joke. We originally used the A* pathfinding algorithm but decided against it as it wasn't as useful for audio adjustment purposes, opting in to create our own instead. Hooking WebRTC up to spacial detection was tough, though we managed to work it out after hours of effort.

Accomplishments that we're proud of

It was our first time setting up a WebRTC handshake, and Spacial Audio Conferencing was, in general, a large project to put together. We're proud we got everything working in the end!

What's next for Spacial Audio Conferencing?

In the future, our goal is to add more environmental features such as room designations, variations of walls that adjust audio levels by different amounts, doors that can be toggled between opened and closed. We plan on adding more productivity-related features like interactive calendars and kanban boards. Finally, we hope to add customizable user profiles and user maps.

Or deploy your own

Built With

Share this project:

Updates