Real Time Sudoku Solver - CV/DL

demo_pic2
demo_pic1

Inspiration

I made a Sudoku solver using PyGame some time ago and I wanted to do something like this ever since. Then I saw that this hackathon was on, so I decided to implement the neural network using PyTorch too.

What it does

As the name suggests, it solves Sudoku puzzles by looking at it and it's on the lines of Augmented Reality. Using the camera, it searches for a 9*9 Sudoku puzzle in the frame, extracts it, solves it and overlays the solution on the puzzle itself. All you gotta do is show the incomplete puzzle to the camera!

How I built it

For the digit recognition part, I used pytorch to build a Convolutional Neural Network and then trained it on the MNIST dataset of handwritten digits. Training Test with an image

For the image processing part, I intensively used OpenCV.

And to solve the sudoku, at first I was applying normal recursion and backtracking, but then it was too slow, so I used Peter Novig's optimized version of backtracking and a medium article by Naoki Shibuya, which reduced the time taken to a great extent!

Challenges I ran into

Detecting the empty blocks in the Sudoku puzzle was one major challenge. Also, superimposing the solution on the puzzle itself was a bit challenging but then there were OpenCV functions which made it doable.