Fishing Vessels Aggregator

Me, Whisker and Volunteer on Stage

Inspiration

There is a lot of data available on the internet about fishing vessels. From pictures of vessels to information about permits and quota; from owner/operator details to unique vessel identifiers. But, even though all this information exists, it doesn't exist in an integrated, standardized form that law enforcement, like the Coast Guard, can use effectively. This is a problem around the world; at-sea law enforcement teams in the Pacific and Atlantic, and everywhere in between, need a comprehensive, simple way to access all the information they need: vessel name and ID number, owner/operator information, permit types, fishing gear, and much more - in a simple, comprehensive format.

What it does

This product gather data from different sources of data about fishing vessels i.e. websites, apis, individual inputs and etc. The data is available on the different websites, vessels information providers and individual inputs.

How I built it

It is build using nodeJs, angularJs, mongodb. I write scraper application which collects the data from websites, restful apis and local csv files. This scraper runs periodically to get latest information about the fishing vessels and store to mongodb in un-structure data format i.e. JSON.

Challenges I ran into

The format of data from different sources are different and challenging to identify the information. Few sources have only references to third party source of data. Some sources require login credential to access the more detail data.

What I learned

I learned fishing industry has a key domain to develop a lot of stuff in aspect of software applications, phone applications, restful apis, analytics and etc.