I'm a massive car guy, and a lot of what I do relates to cars. Last year, my friend and I wanted to create a mobile application with image recognition capabilities using Clarifai's API to be able to identify specific models, makes, and trims just from pictures. Unfortunately their machine learning algorithms were too general for our purposes and there wasn't a feature where we could train our own models, so we abandoned the idea. This year however, I found a way to incorporate the Clarifai API with one of the API's from this years sponsor, Wolfram Alpha.

What it does

Essentially, you can take a picture of a car, and the picture will be identified as a specific car, and information pertinent to that make and model sourced from Wolfram Alpha will appear for you to read. However, I realize that this could be scaled to many more objects than just cars, and using more developed models, we could expand its capabilities many-fold.

How I built it

I used Android Studio to create an Android application using Java, as well as the imported API's from Clarifai and Wolfram Alpha. I also tested a lot of the functionality using PyCharm and a few Python scripts. The Clarifai models were trained using images from Google search, imported through a Python script I wrote for the Python interface for Clarifai.

Challenges I ran into

I had a lot of issues with the Wolfram Alpha API, mainly because recognising the query and consistently receiving a valid response. I had to switch versions and formats for it to work. I also ran into some issues when I needed to port the image data from the camera straight to Clarifai for analysis, as opposed to just storing on the device first. Clarifai also gave me some issues when I first started training the models, because they weren't different enough.

Accomplishments that I'm proud of

Mainly, I'm just proud of finishing a working demo in this short span of time, especially since I was working alone. I'm also happy with the fact that I was able to learn two different API's and merge them together.

What I learned

I learned quite a bit about machine learning through the Clarifai API, and I learned that Wolfram Alpha has information and data on much more than just math problems. I also learned how to work with the camera on a mobile phone, as well as how to integrate multiple API's together.

What's next for Alphafy

Next, I want to try and expand the models that I try to be able to identify more cars and eventually branch out to more objects that people maybe don't know a lot about

Share this project: