Using statistics to unscramble an image

Inspiration

We were asked to create a program that would output a string describing how to unscramble a 2x2 image.

What it does

We created a program that outputs a string after being given an image.

How we built it

We used the package imageR to use images in r since it doesn't natively support it. Our code takes every possible permutation of the 2x2 picture. It stores the positions in a dataframe. The last column of the dataframe is a score metric. The score metric is heavily based on the statistics term correlation. We cut out the two middle columns to get just the "seam" of the image. The columns were made up of vectors of length 3, with the columns as a whole being 128 pixels long. We flattened out the vectors to make the vector describing a single column be 128*3=384 values long. We found the correlation between the two columns. We then averaged that with the similar metric from the two middle rows. When the seams match up correctly, the correlation is usually very high. Another common scenario where the correlation could be very high is when the left and right edges meet in the middle. The left and right images are often the same color, leading to perfect correlation. We decided to look at the variance of the middle columns, and if the variance was too low, we rejected that image. At this point our output has created a dataframe with 24 rows. Each row has a score that can be between -1 ( essentially perfect opposites matching) and 1 (essentially perfect matching). We took the row of the highest score and returned the number based on the position of that image.

Challenges we ran into

The problem of the left and right sides matching due to very high correlation down the vertical seam was a tough problem. Changing our method from absolute vector difference to correlation between flattened vectors started to fix this problem. Rejecting images based on too low of correlation was another step in the right direction.

Accomplishments that we're proud of

We are very proud that we used a different method than everyone else. We stuck to what we were good at instead of forcing ourselves to use machine learning.

What we learned

We learned a lot about the R package imager. We collectively had no experience with images until this project, so there was a big learning curve at the beginning.

Built With

• imager
• r