We're all fascinated with augmented reality, and a smartphone has the perfect level of convenience and capability to make it an excellent resource for AR. We were brainstorming small applications we could make to demo the capabilities of a smartphone for AR, and we landed on a version of pong. Pong has a good mixture of being easy to play and implement while still exemplifying something cool.
What it does
We made a version of pong where the camera is straight behind the paddle instead of being a top down view. We display the game on a computer and then use a smartphone to capture movement as the paddle. The smartphone doesn't actually display anything, but seamlessly sends its position and orientation so that it feels like the pong paddle on the computer. A key component was multiplayer, so you can play with another person on the other side of the game.
How we built it
We built three major components to get our project to work. The first one was the actual game side on the computer. We used C++ and OpenGL to make a simple 3D pong game. To get the paddles to move, we needed to collect the sensor data from the phone. We created a web server, also in C++ using sockets, that relays the sensor data from the phones to the computers, as well as computes the physics of the ball. Finally, we created an Android app that would initiate a connection with the server and transmit the sensor data.
Challenges we ran into
It took a lot of time to set up boilerplate code because we built it all from scratch in C++. We probably spent 10 hours before we even had the code necessary to start our project. We also ran into issues where the paddles and ball were jumping around the screen because of lost or delayed packets.
Accomplishments that we're proud of
We wrote pretty much everything from scratch (maybe not the best idea for a Hackathon...). This means all our software is very fine-tuned for our project, and we are able to achieve high performance and low latency. Despite sending sensor data from two phones across the internet to the two laptops, the latency is almost unnoticeable and the game is very playable.
What we learned
Use existing tools because it takes way too long to do it all from scratch.
What's next for Pong^3
We would like to support more than 2 players and add more interesting physics to the ball (see a game called Curveball). We would also like to make the system more robust and improve performance and tracking.