Seafood fraud is a global issue affecting everyday consumers, disrupting the global fish market, compromising the health of the marine ecosystem, threatening endangered species, and much more. As a result, government inspectors, enforcement personnel, and according to the Oceana study, even the most conscientious seafood consumers may be unknowingly selecting overfished, illegally caught or even endangered IUCN Red List species (e.g. Atlantic halibut for Pacific halibut). In the same study, Oceana found 78% of sushi restaurants in the U.S. had at least one mislabeled fish.
We believe seafood buyers should not be deceived when purchasing their favorite fish.

What it does

Dory is a mobile app that verifies the labeling of your fish, using IBM Watson’s visual recognition technology. Thanks to machine learning, and IBM’s continual innovation, the fin-tastic database becomes more sophisticated over time. In addition, Dory serves to raise awareness about fish fraud in an accessible way and encourages everyday users to make sustainable choices, incorporating information from the Monterey Bay Aquarium and the National Oceanic and Atmospheric Administration.

This ensures you are always eating a great catch!

How we built it

We used design thinking methodology to thoroughly explore the problem statement. Then we had a design sprint, including ideation, rapid prototyping and UX/UI principles for user flow to arrive at an MVP that would communicate our vision for Dory.

The next step involved field work with our partner Bowery Whole Foods to collect data to train IBM Watson’s visual recognition technology. We took 50 photos of scallops and 50 photos of cod that could be mistaken for scallops which we uploaded to Watson as positive and negative data sets respectively. A test data set of 10 images including scallops and cod were used to improve the machine learning capabilities. We used this information combined with seasonal and geolocation information to devise the Dory Score.

Challenges we ran into

Initially we thought we could use pre-defined characteristics to guide consumers to verify their own fish i.e. color, texture, length, but we came to realize that differences are indistinguishable to the human eye. After we realized this we looked for visual recognition technology to support our mission.

Accomplishments that we're proud of

We’re proud of simplifying a challenging issue and for raising awareness about fish fraud in an accessible way. We’re excited that we built a product that gets better over time and is unique in the marketplace. Our app is scalable, cost-effective, and user-friendly, with the potential to be applied to other use cases.

What we learned

We learned that we may have been subject to fishy situations ourselves! We learned about the many issues that are caused by the mislabeling of fish and feel empowered to raise awareness and hopefully bring to market an elegant solution to help others consciously consume seafood.

What's next for Dory

First the ocean, next the world! It’s a good thing oceans make up 71% of the Earth. In the short-term, we hope to work more closely with IBM Watson to refine the fish identification process and address other stakeholders in the fish supply chain i.e. distributors. Swim along for the ride.

Share this project: