Web App Main Page (Click on search to upload an image to search organism,Click on flahcards to redirect to that page)
Web App Flashcards
Uploading feature on webapp
Output Type anlong with recommendations
Recommendations on webapp
Mobile App taking a photo feature
Mobile App Photo Uploaded
Mobile App Added Flashcards Page
Remember watching the cartoon Pokemon as a child and being fascinated by how Ash would click a picture of an Pokemon from the wild and it would identify the Pokemon and would tell him information about it? It sounded so magical as a child. Hanging out at the beach sometimes you feel helpless looking at the incredible array of sea creatures and not being able to identify them. Ever wished if there could be an application which could help you identify the creature and know a little something about it, just like the cartoon? That's where our inspiration comes from. This application will be magical for children exploring the beach or you as an inquisitive explorer.
What it does
This application has the ability to recognize a sea creature species based on a picture you take or upload. It will tell you what creature it is as well as some information and references about it. This is an web application accessible through mobile and a potential mobile application as well.
Based on the image you input SeaSurfer also provides you with recommendations of the other kinds of common species you might spot as well and a little brief about them so that you can identify them easily and just so that you are aware.
SeaSurfer includes another feature which is beach exploration flashcards. You can view various flashcards as a retreat to remember the amazing organisms you have got a chance to see and show off your personal virtual collection of sea creatures when you go back home after you vacation. You will be able to see the photo you had clicked/uploaded as well as the name of the creature and a little information about it.
How we built it
We built this application in the following way: Image Recognition Feature(TensorFlow,flask using web scraping and CNN(we experimented with results of both, whichever have more accurate ones according to our data set))
We tailored and created a image repository/A dataset which is best suited for our project
Using that data set we trained a model which gives predictions based on which image is uploaded keeping our data set as it's base
Those predictions based on images and existing data gave us output.That output displays the species of creature spotted as well as a little information about it.
We displayed this in a card format and below that we included recommendations of similar kinds of creatures found on the beach.
As mentioned the recommendations as well as results show in the flashcard format.
There is another section we created for the flashcards by displaying the received data in form of a flashcard collection
Challenges we ran into
The main challenge we ran into was finding an appropriate dataset which could cater to all our needs, however we did overcome by picking an existing image repository/data set and tailoring it according to what we wanted to add for our project. Another challenge we faced was the accuracy of the model and by using multiple techniques and shifting our approach a little we were able to immensely improve the accuracy of our model. Also please note that in case the image is not of appropriate dimensions it might not be visible.
Accomplishments that we're proud of
We did it! This was a challenging task as there were a lot of aspects we had to cover but we finally got a working model ready. Apart from this our magical childhood dream which seemed unreal then was brought to life today.
What we learned
We learned to collaborate as a team since we all were specialized with very diverse skills, bringing all of them together for this project and integrating them was a great learning experience. Each of us got to explore each others technologies as well as finally we had to tie all the knots to make such a product.
What's next for SeaSurfer
There were a couple of things we could improve which would definitely make this the ideal beach app, although if we had more time we could fine tune it to add anomaly results:
We could aim for the most accurate model by improving it and expand the data set to include more categories(The more vast the data set is the more sea categories of result we can consider)
We would definitely love to see this as a complete mobile application as well, though a mobile accessible web application will work but in form of a mobile app it will cater to a lot of other advantages like take a photo on spot feature etc.
We plan to make our recommendations dynamic and better as we work on it's algorithm and the database