Piplook

A collaborative wildlife tracker app powered by AI.

Comment 7

Inspiration

In a previous hackathon Clara built a bird sighting visualization app. There, she realized there were huge differences in bird sighting reports between countries, with some countries having very few reports despite having a healthy bird population.

Accurate bird sighting reports are incredibly important, they provide a picture of wildlife and ecosystem well-being. Hence, we decided to make bird sighting reporting easier.

What it does

With Piplook you can take a picture of any bird and it will identify its species for you using AI. Then, you will get to know some more facts about that bird species. You will also find a global map with the locations where the bird has been spotted before.

How we built it

For the backend we used Flask. We have several endpoints. Here are some examples:

  • Bird capture: this endpoint gets an image, uploads it to Google Cloud Storage, and sends it to Google Cloud Vision for tagging. Afterwards we filter the tags to get only the species.

  • Bird pokedex: when passing a user id, this endpoint will return the list of species this user has unlocked, which are stored in a Heroku hosted PostgreSQL database.

For the front end, initially we wanted to create a web app with React. However, we realized an app might make more sense, so we used React Native with Ionic. We have used Google Maps for Javascript API in order to display the locations to the user. We are hosting it in AWS and it is served through Cloudfront.

Challenges we ran into

One of the greatest challenges was getting the camera feature to work and post the picture to the API in the right format. We had to work with our phones continuously to make sure the camera would work as expected.

Another challenge was the connection to the database. Initially we wanted to work with a Google Cloud hosted database. We didn't manage to make the import function work, so we ended up switching to the Heroku/AWS hosted database, but we discovered some very interesting features on the Google DBs for the future.

Accomplishments that we're proud of

We are proud to have been successful on using Google Cloud Vision for species classification, since it was not it's original purpose. Having an already trained AI model saved the team a lot of time and allowed us to focus on other meaningful features.

We are very proud of Alex, for having overcome a lot of bugs in this project. Working with Ionic and React turned out to be an impossible mission for us. Even though it was our first time using it Alex was persistent and managed to make it work.

What we learned

It was our first time working with Ionic-React, so we definitely learned a lot on this front. 😉

We also worked with well-defined roles within the team for the first time: front-end, back-end, DevOps, UX/UI. Due to this everyone had ownership of a certain part of the codebase or the processes, which made organizing and discussing features a lot easier.

What's next for Piplook

Something we really wanted to develop was an AI model to classify species by their bird call.

The next step is to collaborate with scientific communities to make sure our bird species assessment is accurate. In the future, we will be uploading the user's submissions to global ornithological databases to make the world a better place.

+ 34 more
Share this project:

Updates