In a previous hackathon Clara built a bird sighting visualization app. There, she realized there were huge differences in bird sighting reports between countries, with some countries having very few reports despite having a healthy bird population.
Accurate bird sighting reports are incredibly important, they provide a picture of wildlife and ecosystem well-being. Hence, we decided to make bird sighting reporting easier.
What it does
With Piplook you can take a picture of any bird and it will identify its species for you using AI. Then, you will get to know some more facts about that bird species. You will also find a global map with the locations where the bird has been spotted before.
How we built it
For the backend we used Flask. We have several endpoints. Here are some examples:
Bird capture: this endpoint gets an image, uploads it to Google Cloud Storage, and sends it to Google Cloud Vision for tagging. Afterwards we filter the tags to get only the species.
Bird pokedex: when passing a user id, this endpoint will return the list of species this user has unlocked, which are stored in a Heroku hosted PostgreSQL database.
Challenges we ran into
One of the greatest challenges was getting the camera feature to work and post the picture to the API in the right format. We had to work with our phones continuously to make sure the camera would work as expected.
Another challenge was the connection to the database. Initially we wanted to work with a Google Cloud hosted database. We didn't manage to make the import function work, so we ended up switching to the Heroku/AWS hosted database, but we discovered some very interesting features on the Google DBs for the future.
Accomplishments that we're proud of
We are proud to have been successful on using Google Cloud Vision for species classification, since it was not it's original purpose. Having an already trained AI model saved the team a lot of time and allowed us to focus on other meaningful features.
We are very proud of Alex, for having overcome a lot of bugs in this project. Working with Ionic and React turned out to be an impossible mission for us. Even though it was our first time using it Alex was persistent and managed to make it work.
What we learned
It was our first time working with Ionic-React, so we definitely learned a lot on this front. 😉
We also worked with well-defined roles within the team for the first time: front-end, back-end, DevOps, UX/UI. Due to this everyone had ownership of a certain part of the codebase or the processes, which made organizing and discussing features a lot easier.
What's next for Piplook
Something we really wanted to develop was an AI model to classify species by their bird call.
The next step is to collaborate with scientific communities to make sure our bird species assessment is accurate. In the future, we will be uploading the user's submissions to global ornithological databases to make the world a better place.