Cloud Spotter

☁️ Inspiration ☁️

We really liked the theme of this hackathon, as it was filled with so much of creativity and colors. We wanted to make something which is fun to make and use, and also helpful to people in a way. Also, we wanted to try a low code hackathon, and this one by snapchat seemed to be the best one. All this inspired us to make a fun powered educational lens, Cloud Spotter, which can let users spot and classify clouds.

☁️ What it does ☁️

Cloud Spotter is a snapchat lens, which requires the user to point at the sky, and then it tells the users the category in which the clouds fall into. Let us give you a little brief over here. The clouds that we see in the sky are very much classified into categories based on their appearance, height, the amount of water they carry and more such properties.

There's a branch of study that comes under meteorology which deals with the study of clouds, their types, properties and how they effect the weather around us. This study is called nephology, and it also includes the classification of clouds into different categories. Our lens is very handy in achieving this, through it's inbuilt features.

So, for our lens, we chose the 9 most common clouds categories, that are:-

altocumulus
cumulus
cirrus
stratus
cirrostratus
nimbostratus
stratocumulus
cirrocumulus
Cumulonimbus

The lens tells the categories detected and the probability of top 5 of them. It is installable on each device with snapchat installed in it. The snap code and link are given below:-

https://www.snapchat.com/unlock/?type=SNAPCODE&uuid=205e7fa6457c471fa12ad74fce56b290&metadata=01

☁️ How we built it ☁️

To build the lens, we used Snapchat's Lens Studio. Since our lens idea required a lot of machine learning based calculations, we heavily utilised the SnapML feature, which was very helpful to us. Also, we saw the templates for binary classification and multi object classification to integrate them into our project. For making and testing the model, we used tensorflow's libraries and google colab notebooks to run our code for training/testing and exporting the models.
The dataset that we needed to make our machine learning model was not available publicly, probably because this niche is very unique. So, we used our self made dataset which has around 9000 images in total. We downloaded and crowdsourced around 500 images and then augmented the dataset to reach 9000 images.

☁️ Challenges we ran into ☁️

There were few challenges that we ran into while making our lens. They include:-

This whole idea of ours was very new and there were no previously available tools to identify cloud types through pictures and videos. So, doing that was very new and was challenging.
importing the model correctly without getting the quantization error.
tuning our dataset and model to get accurate results.
adding styles to few of our objects.

☁️ Accomplishments that we're proud of ☁️

We are very proud that we were able to come up to make this fun lens which is very logical and educational and can actually be used to classify clouds. We're hopeful that people using it would like it. The idea was something unique and we are happy that we carried it nicely. One more thing to be proud of is that we were able to complete the lens in time.