imNotARobot

Inspiration

After running a few sample images through Clarifai's API, we saw how powerful their image tagging capabilities were and wanted to put them (and us) to the test.

How it Works

ImNotARobot is a Chrome browser extension intended to autocomplete Google's reCAPTCHA fields. When the user executes the extension, our javascript will traverse the document held in the captcha's frame to isolate the source image being displayed. After slicing the image into the respective grid pieces, we encode these images and send them to Clarifai's REST API to be tagged for us. After obtaining the list of tags describing each of the sectioned images, we send these and the captcha's target word to ESA Semantic Relatedness API to acquire a numeric value for comparing semantic relevance for each of the tags. Using the total values of all the tags describing an image, we can then guess whether captcha intends that image to be selected or not.

Challenges We Ran Into

Throughout developing ImNotARobot we kept running into several roadblocks. First we had trouble figuring out how to tag the data to be sent to Clarifai through the REST api. We also had issues with parsing the document provided by reCAPTCHA because they do not provide IDs or any other means of identifying the members of the document. We also had the problem of the reCAPTCHA sometimes having a 4x4 grid of images that followed a different format of question than the 3x3 grid of images.

Accomplishments that We're Proud of

Using good agile development practices, and almost always keeping productive and working together to complete our task especially with so many challenges.