ARcade

ARcade Logo

Inspiration

The inspiration for this project came from observing a real life version of Pong reimagining a similar interactive experience emulating classic retro video games in a virtual environment. The choice of the competitive augmented reality platform was made with the goal to play Pong on the go, requiring nothing but your phone and a flat surface.

What it does

ARcade creates a platform for you and your friend(s) to compete against each other using AR in real time. It makes use of multi-peer connectivity, a rather new approach to host multiple participants in the same network to join the same game. The game initializes once the host player’s phone camera scans a flat surface suitable to host the Augmented Reality platform. Once a suitable surface is found, a second player wanting to play against the host can join the game using the app on their own phone. Upon initialisation, the game plays very similarly to classic pong, with each player’s objective being to prevent the ball from crossing their end of the table using pong paddles. Every time the ball crosses either end of the table, the player on the opposite side receives a point.

How we built it

To support a peer-to-peer application, we sought help from the detailed Apple Developer Docs and came across MultipeerConnectivity, a powerful package that allows for the creation of shared P2P experiences. On the more technical end, we integrated this package with ARWorldConfiguration() of ARKit so that devices nearby could all share one configuration, and add entities and anchors to the same ARSession. We kept track of the camera position and syncronized the paddle positioning with the respective device.

Challenges we ran into

Building a game in AR is not that hard, but creating a multiplayer real-time game against two users on the same network is ground-breaking. Our biggest challenge was implementing multi-peer connectivity such that two users can compete against each other simultaneously in real-time on the same domain.

Scenekit vs Realitykit
Our first big decision was to decide which ARKit framework to use, SceneKit or RealityKit? Although SceneKit has been around for a while and has much more support, we chose to go with RealityKit as it allows for better in-game physics and makes the multiplayer P2P connection instantaneous and real-time!
Surface calibration - AR One of the largest starting hurdles was detecting and keeping track of planes in AR. To place our game table, we first had to map out a plane based on what the phone camera sees. But what was even harder was to deploy and fixate other objects and movable entities onto this table we just created, so that our puck and striker are always confined to the game board. We achieved this by treating every plane as an individual Node instead of a plane, and then constraining further smaller nodes like the puck to the TableNode so that it would never exceed the bounds of the tableNode.

Accomplishments that we're proud of

Our main goal was to expand our individual technical skill-sets. In a short span of time, we managed to learn the fundamentals of Swift and ARkit and other required frameworks and tools. Considering the complexity of our project and the ground-breaking development in the world of AR with the introduction of multi-peer connectivity, we are particularly proud of accomplishing our tasks under a time crunch and creating a one of a kind real-time multiplayer AR game. Coming from diverse academic backgrounds, ranging from Computer Science, Chemical Engineering and Mathematics forced us to adapt to varied skill-levels and perspectives despite which, we were able to collaborate efficiently.

What we learned

Our biggest takeaway was the technical skills we gathered over the past 36 hours. With some of us with no coding background to others with rudimentary experience in mobile app development, we were able to broaden our technical skillset and are now proficient in working with Swift, Realitykit, and ARkit in general. We also understood that delegating tasks based on our individual skill levels was beneficial and most efficient, and our unique time management and scrum techniques paid off! Lastly, dealing with complex algorithms using core Physics and Linear Algebra taught us how to be patient, and the errors we ran into along the way taught us how to persevere through and improved our problem solving skills.

What's next for ARcade

Our focus during the course of this hackathon was limited to developing a functional application to allow the user to compete with their friend on the same network in a quick game of Pong. We envision adding additional functionality including, but not limited to, additional levels (with variations like multiple pucks and faster puck speeds), a highscore tracker, and more visual customisation options.