Inspiration
Fish form the backbone of marine environments. However, due to chronic overfishing and environmental pollution, crucial fish populations are in critical condition and need to be protected (82% of fisheries worldwide are below sustainable yield)! Although fishery legislation exists online, the rules are dense and often overlooked by both recreational and commercial fisherman. Our team knew that technology could be utilized somehow to alleviate this issue; instead of requiring fishermen to know fishing legislation in a given region by memory, why not have technology do it for them? Equipped with web development skills and a love for machine learning, our team took the opportunity to build a model and web application that can make sustainable fishing fun and easy!
What it does
Our web application, Fish AI, allows recreational and commercial fisherman alike to take a picture of their catch and identify if it is a regulated fish species or not. First, users can hop onto our vercel app and upload a picture of their catch. From there, our machine learning model on the backend will identify if it is an endangered fish, and if it is, provide information about the species, catch amount, season, size requirements, and a brief description. We built a database of endangered fish specifically for the New England area. For now, we decided to focus our efforts on the fisheries of the New England region because many of the protected fish in the region are important not only to the local environment but also commercial harvest.
How it works
Hop onto Fish AI on any device Upload a fish image (on mobile, this can be done directly from camera) The image will be ran through our neural network model (YOLOv5 model built with pytorch) , and the species of the fish, along with its accessory information, will be output onto a results page
How we built it
Our project components include the machine learning fish identification model, database of endangered New England fish, and the web application.
We built the machine learning model using YOLOv5 object detection model, and trained YOLOv5 to identify endangered fish with a dataset we built. We also built a database of New England endangered fish using CockroachDB. We then built an API wrapper around the YOLOv5 model and CockroachDB with a Flask server. This API takes in an image via POST request and returns the fish information from the image. Lastly, we built the web application using Next.js (React). This web application allows users to upload an image of the fish and retrieves data from our API wrapper.
Challenges we ran into
- Connection with the API wrapper
- Smaller sampling sizes and training set for machine learning model
- Learning CockroachDB and connecting with it
- Styling our pages for PC and mobile
- Hosting our Flask server on the cloud
Accomplishments that we're proud of
- Built the API wrapper around both the ML model and Cockroach DB
- Learned CockroachDB and successfully used it!!
- Utilized YOLOv5, a very novel machine learning model
- Built an intuitive and fully functioning web application
- Connected all components together.
What we learned
From this project, we learned a lot about using object detection using machine learning. We were able to build a custom dataset and train YOLOv5 to specifically detect species of fish. Our team had never previously worked with CockroachDB, but we wanted to learn after hearing about the challenge. We were able to use CockroachDB to store information about New England Fish. We also learned how to connect these components together using Flask and the web application.
What's next for Fish AI
We hope to continue Fish AI’s development in two main facets: improving our machine learning algorithm and adding more informative features for users. For the machine learning algo, we hope to expand beyond the endangered fish from New England and look at other key fisheries in the U.S, such as fisheries in Florida or Alaska. We also hope to continue labeling more fish data to further improve the accuracy of our model. For improved user features, we hope to include more data about sustainable fishing practices and how fishers can best avoid having endangered fish as bycatch.
Log in or sign up for Devpost to join the conversation.