Car Crash Buddy

Making car repair incidents efficient through SMS

Comment

Inspiration

We wanted to experiment with visual recognition, machine learning and dialogue flow by developing a messenger tool to identify car damage on the spot. By learning how severe damage is, along with the current worth of the car, we can determine the cost of repairs and whether it would be more effective to go a mechanic or to make an insurance claim.

What it does

Within the user's text messages, they are asked a series of questions, including the age and for a photo of the damaged car. With the photo, the visual recognition algorithm classifies the severity and area of damage along with a level of confidence (currently around 90% accuracy as we're still in the process of training our algorithm).

How we built it

The visual recognition component of the project was built using the IBM Watson Visual Recognition API. We trained our model by adding various pictures for each type of accident. With this API, we are able to upload a picture and it will be able to detect what type of accident the car was most likely involved and the certainty of it. The photo will be provided using SMS. A Twilio API was used to receive MMS for the visual recognition to classify.

Challenges we ran into

Integrating Twilio API to transfer the photo received in the messenger to a link accepted by Watson Studio

Accomplishments that we're proud of

We're proud to put together a working product in the end!

What we learned

Python, training machine learning models, HTTPS requests, AWS dialogue flow, connecting API's, parsing data in JSON

What's next for Crash Buddy

We want to add more use cases for the Crash Bot in the future. One would be to make use of the information the user enters to better estimate the cost of repair. We also intend on integrating a location feature to access a nearby mechanic or insurance firm if required.

Share this project:

Updates