Puzzled? AI is the solution!

• 

  Architecture of the model used

Inspiration

Classic jigsaw puzzles, while being a simple hobby and entertaining pastime, are of great educational value for the human brain as they train its pattern recognition abilities, improve coordination and develop spatial awareness. Since in computer science pattern recognition is typically solved by image processing, we decided to develop a computer vision program for the automatic assembly of a jigsaw puzzle.

What it does

Our project aims at developing a model that would take a scrambled image, returning a string representing the ordering of the pieces required to unscramble this image.

How we built it

• We first divided each image into four sections denoting each piece of the scrambled image. Next, we created a stack of these four puzzle pieces for each of the images.

• Now, this stack of four sections was used as training data for our model. This input data was passed through 5 units of convolution layers followed by batch normalization and dropout layers.

• Then we passed the feature vectors extracted from the model into a multi-layer perceptron network of 5 dense layers and a few dropout layers.

• Finally, the output feature vector was passed into a dense layer having 24 outputs where each of the outputs denotes the probability of the possible permutations of the 4-piece puzzles.

• Thus, we used the above-obtained permutation to unscramble the puzzle pieces.

Challenges we ran into

• Tuning hyperparameters for obtaining the highest accuracy proved to be a challenge.

• We also had to ensure that dimensions were matched according to the layers' requirements.

Accomplishments that we're proud of

We are glad to have developed a lightweight model to solve 2x2 jigsaw puzzles correctly 9 out of 10 times tested, at minimal computational power.

What we learned

We learned how to use a custom data generator and also effectively applied different mathematical functions for learning rate annealing to obtain the best results.

What's next for Jigsaw-AI!

We plan to deploy this model as a web application using flask in the future. Our automatic jigsaw puzzle solver also has several potential applications, among which are reassembling damaged documents and stitching satellite images into maps.

Built With

• keras
• matplotlib
• numpy
• python
• scikit-learn
• tensorflow