Picture Picture Perfect

Before Picture Picture Perfect
After Picture Picture Perfect

Inspiration

We wanted to do something that truly challenged our programming and Data Science skills. We decided to attempt the puzzle solver challenge. We were interested by computer vision, and wanted to see if we could solve an advanced computer vision problem.

What it does

The assignment requires a machine learning algorithm to unscramble a given image. A given image has been split into four even pieces, each piece rearranged, and the image reassembled. After the model’s training, it returns a string with the solution containing the correct placement of the pieces to solve the image.

How we built it

We used Python, Numpy, TensorFlow, and Keras in order to build this project. We built the main orientation system using a using a Convolutional Neural Network Machine Learning model. We took in training data from the given data and from popular online datasets of scrambled and unscrambled images.

Challenges we ran into

One of the most complicated challenges we faced is that there were not enough unscrambled images to train our model off of. Our model was incorrectly classifying many images because it was receiving so many scrambled input images. We resolved this issue be taking a sample dataset online and training out model on it. This allowed for the model to become much more accurate.

Accomplishments that we're proud of

As noted above, there were many unforeseen challenges in both the iterative development of our machine learning model and the implementation for scrambled photos. On the night before the submission was due, our team troubleshooted a number of challenging unforeseen problems in order to submit the deliverable before the project deadline. Seeing the program run successfully after the long uphill climb was immensely gratifying.

What we learned

Our team learned about digital image processing concepts including pixel distance measures, pixel neighbors, and adjacency. We also gained experience working with Pytthon’s Numpy module for large-scale data manipulation. Additionally, we deepened our understanding of how to implement machine learning technologies with Tensorflow. We learned how to work under a strict time constraint, how to plan for complex problems, and most importantly we learned a lot about machine learning.

What's next for Picture Picture Perfect

Our next plans for the picture perfect project would include scaling it so that it can handle images that are split up in more sections than a 2x2 grid.