We all hate when we see undesired content like some news about topics that make us uncomfortable. So we asked ourselves if there would be a way to remove all this content.
What it does
Kittyfy is a browser extension that allows you to set some keywords for content that you don't want. These keywords are then searched through all the web pages. If there's a match in the text then it's replaced with Meow or Purrr. If the match is in an image this image is replaced by a kitty image.
How we built it
To build the extension we used the Firefox extensions API to be able to modify pages' content. The most innovative part was using Azure's computer vision API to analyze every image and get important tags about that image or if the image contains a celebrity. With the image's tags then we can decide whether to replace or not the image.
Challenges we ran into
The first challenge was that none of use had never build a browser extension before so we had to learn from scratch how to do it.
The second one was to be able to detect all the images in a website. Since many websites load images through data URLs or add them with the CSS background rule we had to read all the requests for images and decide whether to resolve the image or to redirect to a kitten image.
Moreover, when handling the requests we found that Chrome browser does not let you cancel a request asynchronously so we moved the extension to Firefox.
Once we had all the images we had to decide whether or not ask the azure API to analyse the image searching for tags based on size, format and other heuristics.
Also, replacing text without breaking the HTML document was not an easy task so we had to search the best way in doing it.
Finally, we wanted to expand the keywords provided by the user so we had to search for another API that provided us with keywords related to the main one.
Accomplishments that we are proud of
We have released the internet from thinks we hate. Making the world a little bit happier by allowing the user to remove undesired thoughts.
What we learned
- How to create a browser extension.
- How to use Azure's API for image recognition
- How to work with ECMAScript and Typescript
- How to properly divide the work for a good teamwork.