We're motivated by how pervasive advertising is in the modern era, especially by companies that encourage or directly contribute to environmental problems. With our "bad"-blocker we envision a future where using AR we can remove unwanted advertising, in particular ads or logos from companies that make the environment worse. We take our project a step further by replacing the logos with environmental facts about the company.

What it does

Our app hit a number of key milestones that we're very proud of:

  • Processing realtime video with Python and OpenCV
  • Detecting logos of companies at 25 fps (only one at a time)
  • Using SIFT and other similarly advanced CV techniques to detect logos with skew, rotation, scaling
  • Detecting multiple logos
  • Blurs the logo out
  • Replaces the logo with an environmental fact about the company

How we built it

We take video from a webcam using FFMPEG and pass it to OpenCV with a python pipe. Within OpenCV we have multiple ways to detect logos. The fastest is using color matching to find strong areas of color that match a desired logo's. This works well (and extremely quickly), but it's not very resilient and can't deal with different logos. The next is template matching: template matching worked well to identify logos and put bounding boxes around a single logo. Unfortunately it's too slow to run at a full 25 fps but it gives a very good bounding box for replacement. Finally we have SIFT (Scale Invariant Feature Transform). This uses OpenCV SIFT to find the keypoints in a logo and in webcam streams and matches them. It uses a Flann Tree to search the webcam image and the RANSAC algorithm to find homography between the two. This method does well in most areas, but it doesn't get a perfect bounding box all of the time.

Challenges we ran into

We ran into a few challenges which turned into great learning opportunities:

  • Using OpenCV video capture: This is not in the standard open CV install, we used FFMPEG instead.
  • SIFT. SIFT is a hard algorithm and OpenCV takes a little massaging to get it to work.
  • Speed. It's hard to get computer vision to go as fast as we'd like. Writing better code and improving our skills helped a lot here.

Accomplishments that we're proud of

We're most proud of the fact that our product works. We reached all of our goals that we had when we set out and feel that we've demonstrated the viability of this product.

What we learned

We learned a ton about image processing, Linux video internals, and computer vision.

What's next for Badvertsing blocker

We weren't planning on getting to a proper AR setup at this hackathon, but in the future we'd like to put this into an AR headset.

Built With

Share this project: