We wanted to experiment with visual recognition, machine learning and dialogue flow by developing a messenger tool to identify car damage on the spot. By learning how severe damage is, along with the current worth of the car, we can determine the cost of repairs and whether it would be more effective to go a mechanic or to make an insurance claim.
What it does
Within the user's text messages, they are asked a series of questions, including the age and for a photo of the damaged car. With the photo, the visual recognition algorithm classifies the severity and area of damage along with a level of confidence (currently around 90% accuracy as we're still in the process of training our algorithm).
How we built it
The visual recognition component of the project was built using the IBM Watson Visual Recognition API. We trained our model by adding various pictures for each type of accident. With this API, we are able to upload a picture and it will be able to detect what type of accident the car was most likely involved and the certainty of it. The photo will be provided using SMS. A Twilio API was used to receive MMS for the visual recognition to classify.
Challenges we ran into
Integrating Twilio API to transfer the photo received in the messenger to a link accepted by Watson Studio
Accomplishments that we're proud of
We're proud to put together a working product in the end!
What we learned
Python, training machine learning models, HTTPS requests, AWS dialogue flow, connecting API's, parsing data in JSON
What's next for Crash Buddy
We want to add more use cases for the Crash Bot in the future. One would be to make use of the information the user enters to better estimate the cost of repair. We also intend on integrating a location feature to access a nearby mechanic or insurance firm if required.