NOTE: Our website is available to use and is included below, however it is only tested on Google Chrome!

PowerPoint: https://docs.google.com/presentation/d/1NjbLDOykrrj_UvKTT6Vg2dLIdEgFW4o6gQfkeCrEc_w/edit?usp=sharing

Inspiration

The coronavirus pandemic has significantly uprooted our lifestyles. Quarantine and social distancing, the recommended methods to help stop the spread of the virus, have forced 297 million Americans and billions of people around the world into isolation. This isolation has cancelled social gatherings and destroyed the possibility of big events. Like many other Americans, we’ve been impacted heavily by the pandemic as college students, and miss walking around campus with friends, meeting with interviewers at job fairs, and going to in-person hackathons :( . To help remedy the situation, and to simulate the feeling of being with friends, we built Connect.us.

What it does

Connect.us is a platform that allows us to walk around and chat with others, just like in real life. With fun and playful 3D environments and characters to choose from, it allows us to hang out with friends, attend conferences, and more! All you have to do is move with the WASD keys! The main functionality of the app is proximity-based video calling, where you can walk up to other people in the world and start chatting and video calling them on the same page. Users and organizers can also make custom rooms and invite attendees with a simple room code. Overall, Connect.us is adaptable to so many different social events that were once impossible in quarantine.

How we built it

We created the character models and environments using voxel editors called Goxel and MagicaVoxel. Voxels are 3D pixels that can be used to quickly build various objects similarly to pixel art. The platform itself is powered by 4 different servers working together. The Web server is a nginx server hosting the https://connectus.dev domain and provides the local HTML and Javascript code used for the platform. This code includes a web-based javascript game engine called three.js to render the players and environments, and handle events like collisions. The Game server is a python Flask server which handles the networking aspect of the platform by exchanging and synchronizing player positions.

The video and audio communication aspect of the platform is powered by a framework called WebRTC, which allows native web-based video and audio transmission without any third party libraries/APIs. This system uses 2 different servers to perform its task; a signaling server and a TURN server. When a user first connects to the platform, it communicates with the signaling server to set up the video and audio communication channels. Since most users are behind home routers, direct peer-to-peer communication is not possible, so another server called the TURN server is used to proxy data between the users once the connection is established.

All these servers are hosted on Amazon Web Services.

Challenges we ran into

Servers are a huge pain. :( Especially when you have to run 4 different servers at once to handle the game engine interaction, turn server and signal server for the video calling, and display and serve the app itself.

Accomplishments that we're proud of

We’re proud that we were able to make a cool interface that overlays video calls over a 3D environment that we can freely walk around in. We are also very happy that we got through the many bugs that came with coding our own servers to handle user interaction data and also https protocols.

What we learned

We learned how to use new tools such as Goxel and MagicaVoxel, new frameworks and libraries like three.js and WebRTC, and gained more experience with servers, networking, concurrency, and security! Jumping right into it, we were not sure how robust the final product was going to be as we had a lot of preparation to do before we could write our first line of code, but overall the challenge in making our vision a reality is what made it fun!

What's next for Connect.us

We want to improve Connect.us by adding more features to help Connect.us feel more and more like in-person connection. These features include things like mini games, the ability to join a lecture, and collaborative whiteboards. We hope to release Connect.us for the general public by using it with our friends and spreading it through college social groups. Connect.us is already available on a public domain, so this shouldn’t be too hard. We want to spread the use of Connect.us for event and conference organizers to use to to make their events more lively! To make sure this runs smoothly, we would like to test the scalability of Connect.us and make sure it can run with hundreds, or even thousands, of users.

Share this project:

Updates