Character Mood Classifier

Beginner artists sometimes struggle to convey their character's emotions. This tool helps these artists see if their drawings convey emotion properly by using Deep Learning.

Comment

Inspiration

As someone who draws, I often found myself struggling to convey my character's emotions when I started out, and I have noticed that many beginner artists encountered this problem as well. I wanted to create a learning tool for artists to see if their emotions are clear enough for a computer to understand the character's mood.

What it does

Users upload their drawings in the website. Then, the deep learning model will try to predict the character's mood and display the result.

How we built it

The project was built using PyTorch and a pretrained ResNet-50 model which I finetuned and trained using a mix of a dataset I found on Kaggle and my own screenshots.

Challenges we ran into

Due to the small size of the dataset (around 430 images from Kaggle and 150 screenshots I took), the model would end up overfitting to the training data and would not learn the features properly. This resulted in a very high accuracy for the training set (over 90% most of the time) and a low accuracy for the testing dataset (around 40% at the start, now around 58%). Getting the accuracy to where it is now was challenging, since I had to take my own screenshots and learn transform my dataset to perform data augmentation.

Accomplishments that we're proud of

I'm proud to have achieved an accuracy higher than what I initially aimed for (expected: 50%, actual: 58%). I knew I would run into the dataset size issue and I am satisfied with how I managed to deal with it to get satisfying results.

What we learned

I learned how to use PyTorch for deep learning, more specifically for image classification. I also learned a ton about datasets and how to make them more reliable (increasing the size of the dataset, preprocessing images, aligning faces, etc.)

What's next for Character Mood Classifier

I am planning on improving the model's accuracy to at least 70% ideally. I want to learn more about making my own dataset and automating the process to hopefully get much more than the 680 images I currently have.

Built With

Share this project:

Updates