Venomous spider bites are potential fatal - every Australian wants to avoid them! But most people can't easily identify spiders, which is important to ensure people are appropriately careful if one comes into their house. In the event of bites, the spider also needs to identified to administer the correct anti-venom.
What it does
Uses machine learning to classify images of spiders, and identifies if they are venomous or harmless. Can be used through a web interface (upload image) or app (uses phone camera).
How we built it
We had available Accenture's machine learning API as part of the hack, but it was trained on the ImageNet dataset, which focused on identifying any object to a reasonable degree of accuracy. We customised this API with training data for specific species that are most commonly found in Australian households, to teach the API how to identify them. This API is the backend for our app while we used Flask for our middle tier and an HTML/CSS/JS front-end.
Challenges I ran into
We wanted the app to be multi platform, so it could always be easily accessed. Making the experience stable and consistent between platforms was challenging. This is why we decided to make a webapp and try to integrate it with PhoneGap to create the mobile versions. We also had little or no experience in deep learning, making training the classifier and hooking with the API a new and interesting experience. We ran into difficulties at first when trying to train the API with our dataset, with the API returning incorrect and sometimes quite odd results, but we've managed to discover more about machine learning and now get fairly accurate results.
Accomplishments that I'm proud of
We had a diverse set of talents in the team, and we managed to combine them to make something cool in just 24 hours.
What I learned
I learned about how useful online computation is for machine learning. The classification algorithm was too expensive to run on a phone, but offloading the computation allowed the app to be powerful and portable.
What's next for Spidr
A natural step would be to integrate it better with emergency services, to allow the app to better ease the burden on emergency workers. It can also be extended fairly easily to identify other types of dangerous animals such as snakes. Another extension is to allow users to describe a spider in words (if, for example, it ran away before they could take a photo) and provide a list of possible spiders to choose from to help in identification.
3rd party APIs used
- Accenture's image classification API (version 2)