Fish ID

HoG features for fish detector
Part of the VGG architecture utilized

Inspiration

Getting statistics about fish is hard. Particularly, quickly identifying fish is difficult for beginners and time consuming for experts.

What it does

The application processes a simple white-background picture of a fish and attempts to identify the species using computer vision techniques. It also provides a simple fish detector, although this is not working as well.

How we built it

The process leverages the deep VGG-19 network (ILSVRC-2014 competitor) to extract high dimensional features from a simple picture of a fish. It then trains a support vector machine (high dimensional, linear classifier), to classify these features on a fish species. The picture simply requires a flat white background fish which is then pumped through the process.

Challenges we ran into

Initially we wanted to be able to identify fish in the hands of people. This is a two step problem. First you have to locate a bounding box for the fish, then you have to pump that cropped image through the network. This turned out to not work too well because the strong detector we used (histogram of gradients). Does not work well across fish types (only 20-30% recall). So we'd have to train a separate detector for each species. We decided then to use simple white-background fish pictures without detection, which worked decently well.

Accomplishments that we're proud of

We are able to classify fish based off high dimensional features!

What we learned

Fish shape varies widely. This is a blessing and a curse. It's a curse because it makes detection a pain. It's a blessing because it makes recognition tasks easier.

What's next for Fish ID

Refine the workflow and get the accuracy up for detection to make in-hand or in-any-context identification possible (no longer require a white background). Will probably require a deep net trained on fish detection (which requires a LOT of data).

Build a simple front-end to take a picture and upload to our pipeline.