Inspiration

Surprisingly, there is still no easy way to do precise outline-based extraction of individual objects from a photo. Tools such as photoshop have fancy cropping tools, but they require some practice and skill to use well - which an image editing novice like myself does not have. There just are not any tools to crop out entire objects from their background with only a few clicks. So, this was a hacker-y attempt to address that.

What it does

It is a website on which users can upload a photo, surround an object they want to crop out with a drawn rectangle, and extract the object from the photo with one click. Since the machine-learned model is hosted on AWS, it would also be easily modifiable to be a REST API for other applications to use.

How I built it

I had already had experience with DeepMask when starting on this, since my project for Stanford's AI class was to modify DeepMask to see if it could be used to crop objects with a single click. My conclusion from that was that I was still better off using Facebook's approach, but my experience with AWS for that project came in handy. Specifically, I reused my previous AWS DeepMask computing solution: paying for an AWS EC2 instance with a GPU, installing the relevant dependencies on it, and making it host a REST server. I had to modify the DeepMask code slightly, but for the most part I just set it up to run on the cloud and used Facebook's pretrained models.

The bulk of the remaining work was in implementing the http://objectcropbot.com/ web demo. I made this with basic HTML/CSS/JS parts, except for the excellent open source visual cropping library Cropper.js. After an all-nighter tweaking the demo to look and feel right, I set it up to be hosted on GitHub and arranged the domain to be what it is after buying it from NameCheap.

Challenges I ran into

I am not a frontend developer, so this project being primarily focused on a (fairly tricky) UI for using the AWS-hosted model did not match my strengths.

Accomplishments that I'm proud of

Primarily making the UI as smooth and not terrible-looking as it is.

What I learned

I started out thinking I'd implement the cropping manually, and was lucky to think to search for a cropping library - it proved crucial. So, as ever got to remember to be as lazy as possible.

What's next for CropBot

The cropping precision still leaves a lot to be desired, even if it is state of the art for AI. So, ideally i'd be able to keep iterating and making it better at the task.

Built With

Share this project:
×

Updates