Inspiration
The inspiration for this project came from our interest in deep learning/ computer vision. For years now we have seen these captcha puzzles and even up to a certain point in my life I didn't even understand them or what they tried to achieve. So, we put together our interest in deeplearning/computer vision coupled with our curiosity about captchas and incorporated it all in this project.
What it does
We take random sampling out of a dataset of roughly 11700 images and display them on a 3 by 3 grid. We then use a convolutional neural network to classify the images on the grid and we highlight the AI's predictions.
How we built it
To build the AI, we used popular AI research papers as a rough guide/reference to construct the overall architecture of the model. Some papers we used were the paper on AlexNet and VGG16. Our unique model was then built in tensorflow with the functional API. For the back-end of the website we had a flask application. For the front-end, we used basic HTML, CSS, and JS as well as bootstrap and jQuery.
Challenges we ran into
There were two major challenges we ran into when designing the project. The first was that the AI had a hard time classifying at first because the pictures often contained more than one of the classes we designated. As an example, a picture with crosswalks also contained street lights and cars. To solve this, we designed a global model and then used that to transfer weights to several models designated to one class. The second major problem we had in the project was mixing flask and react.js. We had a lot of trouble connecting react to our backend and python code so in the end we decided not to use it.
Accomplishments that we're proud of
The best accomplishment about the project we are proud of is how it came out in the end. We are happy with the results overall and hope you guys enjoy seeing it too!
What we learned
The main thing we learned throughout this project was dedication. We barely had any sleep last night because a lot of things regarding the front-end were not working out but we stayed determined and we eventually got everything working.
What's next for Beating Captcha (b-captcha)
Eventually we see ourselves adding a game so that people can vs the AI and we will increase the speed in which it computes the answers as players progress throughout levels. We would have added it for the dev-post but time did not permit us to do that.
Log in or sign up for Devpost to join the conversation.