We wanted to automate the entire process for fisheries, not just collect data about fish features. We also noted that the amount of fish that needs to be classified is huge. We think that the process of image acquisition needs to be automated as well. That would decrease the time of classification considerably.
What it does
In order to fully automate the classification process for fisheries our solution is able to classify and physically separate fish (either dead or alive) and store relevant information about its features.
- We transport fish trough the system without human intervention. We do it fast.
- We classify the fish species using software.
- We physically separate and keep it alive protected fish species using the water tubes.
- We physically separate fish that is ok to use.
- We show the data in an user friendly way.
How we built it
1. Recognition phase:
We use a cellphone camera and a mobile app equipped with a computer vision system built using OpenCV that detects specific features of the fish (i.e. color, length, fish fin) This information is sent to a computer.
2. Classification Phase:
We implemented machine learning techniques and software to build a classifier that can make more accurate predictions about the fish species.
For the physical classification we created two stages: the first stage works with water tubes that is able to separate fish keeping them alive. The second stage works with a conveyor belt and is capable of separating dead fish, it also works as a rectification phase of the first stage, this process is made with additional cameras.
We built this system thinking about keeping alive and returning to the sea protected species, fish that is before its reproduction period or other protected animals that are not even fish. i.e. Turtles, Sharks.
3. Data Visualization
This step is made with a Software that use an authentication system to get all the information relevant to identify who is doing all the process: fishermen, fishery, location, etc.
All the data acquired in the process is stored offline in a small computer, it gets published to a server once the computer gets access to Internet or if the computer has already access to it.
The information can be visualized by workers on site using the computer or by any other member of the organization using a website.
The data can also be accessed by other people (researchers, developers) interested in work with this information to make projects related with sustainable fishery or for scientific and statistical purposes. We created a web API to archive this.
Challenges we ran into
Recognizing multiple species of fish was difficult and we couldn't find an efficient data source or web API to work with, so we bought two real fishes in our local fish market and used them to build our own dataset, train our machine learning algorithm and test our proof of concept.
The integration with the hardware controller was not easy at first. We needed to test a lot to send the signal just in time to separate the fish. Timing was crucial.
Keeping fish alive was a huge challenge. We wanted to be able to do it, it was really important to us to probe if it was possible to make a system that could do that.
Accomplishments that we're proud of
We were able to fully identify our two species of fish using basic parameters Our machine learning module was able to learn from it's own errors. It did not made a mistake with a sample again after the feedback was received.
We found a solution that not only improves work conditions for fisheries and fishermen by creating a faster fishing selection process but it also keeps alive and returns to the sea the fish that is a protected species or is too young to be fished. We managed, in less that 48 hours, to develop a minimum-viable-product integrating software and hardware. We’re proud to say that this solution is applicable to other different industries and uses.
What we learned
We learned a lot about the fishery industry and also about fish. We also got a lot of information about the ocean and the problems that we need to solve in order to preserve it.
Interdisciplinary teamwork is huge in terms of problem solving. As the brainstorming process was being done, each discipline representant was arguing why each idea was strong or not from their point of view. This was crucial to consider not only one-point-of-view-solutions but holistic solutions, which are more robust.
What's next for fishclassifier
Physical prototypes are required to validate our first design and keep on redesigning until there’s a feasible and profitable solution. We consider it’s important to do this in collaboration with fishermen in order to receive qualified feedback from the future real users of our solution system.
For the software the next step is to build a more robust fish detector system and improve design and user experience. We consider important to include additional fish features and also a larger amount of data is needed to improve machine learning.