We are located at table 41 near the control booth.


We honestly didn't have this idea until the day before Bitcamp started. We had a group meeting over dinner to discuss potential ideas that could utilize all of our strengths. Mikhail and Muyiwa both had good coding experience, Gerry was on electronics, and Sam and Bryce were great with CAD design and robotics implementation. In the end we decided to come up with something that tied all of those together. Mike was able to register a Oculus CV1 for our group, and we wanted to be able to utilize that as well. At this point we definitely wanted to have a project that utilized both software and hardware. From there it was just brainstorming until we came up on the idea of making a interactive version of Rockem-Sockem robots. In the end, this worked perfectly because everybody had something to contribute.

What it does

Essentially, there are two parts to this game. In terms of hardware, we have the Oculus Rift CV1, the Kinect 1st Gen, and the two 3D printed robots (controlled with 4 servo motors each). Essentially, the Kinect records user input in the form of arm movements. It can record a left punch, a right punch, a block motion, and an unblock motion. The Kinect (which is running on Unity) then sends a value (based on which movement it registered) to the Arduino which powers and controls the servo motors that control the movements of the robot. Essentially we are replicating the movements of the player on the robot.

The next part is the Oculus CV1 implementation. The robot has a webcam mounted as the head, and it relays that information back to Unity which then places it on a plane and relays the image to the Oculus. As a result, Player 1 gets a real time, first person view of the robot fight. Additionally, in Unity - a Heads Up Display (HUD) is added in order for Player 1 to have an augmented reality experience. The HUD will contain player health and opponent health.

Player 2 (Since we only had one Oculus) is played by an XBOX controller. So instead of having the Kinect read movements, the input is instead generated by the wireless controller. However the actions are still replicated on the Player 2 robot. Player health and opponent health is also indicated by LED's for the convenience of player 2.

The game is played between Player 1 and Player 2. The battle goes until one player hits the other player 4 times. Once that happens, the game is over, and the player who made it to 4 points wins.

How we built it

Both robots were both designed in CAD and 3D printed during the 36 hour period of Bitcamp. Different colors were used to differentiate between the robots.

The arena was made with sheet of wood, and some pens glued in. Rubber bands stretch around the arena as well.

We have a breadboard where the power and ground is connected to the power and ground of each motor, and the control wires are connected to the Pulse With Modulation (PWM) pins on the Arduino.

The Oculus is worn by the Player 1, the XBOX controller is held by Player 2, and the Kinect Sensor and the Oculus Sensor are wired into the computer and both pointed at Player 1.

Challenges we ran into

We literally spent the first twelve hours just trying to install the Oculus software. There were serious incompatibility and permissions errors that took us a lot of time to troubleshoot. So Oculus integration was one of the last things we started working on. This issue was widespread and affected all of the other students using the CV1s. At around 3 PM Saturday afternoon, we were able to get a fellow student (who magically had figured out the issue) to help us install Oculus AND get it working well with Unity (which was another serious issue).

Additionally, we originally planned to use the Intel Edison instead of the Arduino, but it was not being recognized by the computer, and after spending a lot of time on it, we realized (with some help from an Intel employee) that there were not even enough PWM pins for all the motors we were using anyways, and we ended up scrapping the idea. The Arduino proved to be much easier, and much more flexible to code and implement.

Once we had that up and running it was relatively smooth sailing. Of course there is always bugs in code, and things wired up wrong, but those are issues every project faces and we were able to deal with them

Accomplishments that we're proud of

Basically the whole thing. This project was ambitious at best and while we had high hopes going in, it was really a lot to complete in just 36 hours. Needless to say we didn't get much sleep. And basically everything from getting the Oculus to work, to finally getting the Kinect to register punches, to having the robot be able to replicate the actions of the user were all massive accomplishments. Additionally getting together as a team and working together to produce something amazing was another accomplishment. It kind of makes all the hell we go through as engineers worth it when we can actually bring our minds together and build something amazing.

What we learned

We learned a lot, individually and as a group. For most of us, this was our first Hackathon, and we learned a lot not only from our project, but from all the other projects around us as well. We learned the great things that can happen when a lot of minds come together.

What's next for Rockem-Sockem-Oculus

Who knows, we might keep working on it! Although since we have to return all of the gear we rented it might also just turn into a cherished memento of a great Bitcamp.

Built With

Share this project: