A couple weeks ago, while playing connect 4, I mentioned to a friend that Connect 4 is a solved game. In other words, given optimal play by the first player, he or she will always win. My friend, getting the gist of the concept, then asked: "So if I could remember how to play optimally, I could always win?" Sadly, I had to explain that there were far too many moves for him to remember. However, I suddenly realized that if I could combine computer vision and a perfect AI, I could actually always win against my friends.
How It Works
Our Google Glass application takes a picture and sends it to an EC2 Apache Tomcat Server using a POST request. Then at the server, OpenCV is used to transform the picture into a computer readable board. The board is then passed as a string of moves to a Connect 4 perfect play engine, which returns the optimal move to the Google Glass.
Using OpenCV to get the board from a picture. Installing all the necessary components to an AWS server. Using JNI to run C code in Java. Meshing JNI with Tomcat. Creating a Google Glass application for the first time. Android file transfer protocol.
Being able to return a computer readable board from a picture. Porting a 16 bit AI engine to a 32 bit operating system. Being able to return the optimal move given a Connect 4 board.
What We Learned
First and foremost, we learned a lot about working with computer vision. Secondly, we learned a lot about cross platform development, as we had to integrate a mobile Google Glass with a Java server which had to further call C-code.
The computer vision is the part that could use the most improvement. Computer vision is still a relatively new field, and due to our inexperience with the technology, we had to stick to pretty simple approaches for the object recognition. In the future, with more time for experimentation, we should be able to develop a more versatile computer vision system that can recognize a board in even the most tricky environments.
Looking at the bigger picture, although this project may not seem to have any significant impact, it represents everything wearable computing could be. Wearable computing's niche is in its impact on daily interactions. Many of the things wearable computing can accomplish are impossible for phones and computers. Wearable computing has the potential to assist you in every facet of your life, whether it be remembering names, knowing where to go, or even playing connect 4.
As of now, the product that is the closest to what I thought Google Glass could be would be the Microsoft Hololens. Although simply displaying the column of the best move is functional, it would be far more amazing if one could project a hologram onto the connect 4 board, for example. Perhaps that's where this app will head to next.