House Style Identifier

Inspiration

The project was inspired by the "Best machine learning in commercial real estate" prize category.

What it does

The goal of this project is to develop an android app that identifies the house style of a picture. Users can upload their own images or take on using their camera, and a machine learning algorithm will identify the house style. This will allow home buyers to get a better gauge on what home style they like or what style they are looking for. The home styles that the model can identify are cape cod, country french, colonial, Tudor, cottage, Mediterranean, modern, traditional ranch, contemporary, and townhouse.

Currently, the machine learning model has been trained by Google Vision. This is a Google Cloud service that allows users to import data files, label, them, and then trains a pre-made neural network using those files. The Model has an accuracy of 64%, precision of 80%, and Recall of 44.9%. The low accuracy is in part due to modern and contemporary house styles being near identical, leading to 75% of modern home images being identified as contemporary. Since these styles are effectively the same, this accuracy measure is not fully representative of the model's true accuracy. In retrospect, modern and contemporary should not have been included together.

The android app currently allows users to take a photo and stores that image in a bitmap. This is the only functionality working at the time.

IDEALLY: The android app would allow users to upload or take a photo of a building. The user can then press a "process image" button, upon which the TensorFlowLite file will process that image and return a predicted house style.

How we built it

The machine learning model was built using Google Vision, implemented through an android app.

Challenges we ran into

The primary challenge was building the machine learning model and implementing it in Android Studio. Initially, I tried to create a neural network from scratch using TensorFlow. This took up around 4 hours of research and model building attempts before a mentor told me about Google Vision. This made the model creation process significantly easier.

Implementing the TensorFlowLite model into Android Studio was another challenge. We were unable to implement the machine learning model into our App.

Accomplishments that we're proud of

This was my first attempt at machine learning, so I am definitely proud of the model that we produced. While the model wasn't built from scratch, I now have a much better understanding in how supervised learning models are created.

What we learned

How to create a machine learning model through Google Vision.

Increased our fluency in creating a Java-based application in Android Studio.

What's next for House Style Identifier

Implementing TensorFlowLite into the app would be the next step. After that, allowing users to upload their file to a database in order to process the image. Next, importing more training files to the model would increase its accuracy. Currently, around 80 images were used for training for each of the 10 house types identified in our project. Increasing the amount of house types the app can identify would be the final improvement.

Built With

• android-studio
• google-vision
• java
• tensorflow
• xml