Say you wish to take a picture of your friends at a public place and people keep crossing the scene that you wish to capture. Usually, you would wait for the people to pass by and take the picture afterwards. This however might take a while and forces you to pay attention to your surroundings. Enter 'Decrowder'. 'Decrowder' is an iOS app with which it is possible to take a picture of your friends, say, in front of a monument but without any pedestrians that were crossing the scene at the time when you were taking the picture.
The application records a short video, applies operations to each frame and eventually returns a single image as a result.
The flow of the operations is as follows:
- video stabilization
- background subtraction for motion detection
- image inpainting
Video stabilization is needed to even out movements of the smartphone. In our implementation, we find homographies between images and apply perspective warping. In order to achieve background subtraction we rely on an OpenCV implementation of a statistical Mixture Model. This allows us to detect blobs of moving objects in each frame. To conclude, we look for the frame with less false-negatives, and we inpaint the moving objects in it using the other frames of the sequence. This step represents the main strength of Decrowder. We don't just simply wait for people to get out of the field of view to take the picture, but we rather offer a new solution which fits well with crowded spaces.