Easymoji

Our logo.

Inspiration

We were inspired by the difficulty to find the correct emoji on time during a conversation, as well as our interest in machine learning and image recognition.

What it does

It uses Microsoft's Azure API to detect the emotion on one's face and finds a corresponding emoji. This emoji gets copied into the user's clipboard and then they can easily paste it into whatever they're using.

How We built it

We used tkinter to create a GUI and OpenCv to get a video feed and used Azure to get a face's emotion.

Challenges I ran into

At first we were going to train a neural network of our own. However, we ran into several difficulties and were not able to train it in time in the end. When we discovered Microsoft's Azure API, we found a way to easily solve all of our problems and make all of our frustrations disappear.

Accomplishments that I'm proud of

We discovered how use the azure api, opencv's facial recognition features, and learnt a massive amount about machine learning.

What We learned

We learnt a ton of machine learning and pytorch during this hackathon. Although we weren't able to fully train a working neural network, we were able to train a neural network to 40% accuracy during our hack, which I feel like is pretty good for some people who have never made a large neural network before and our one teammate who has only used Tensorflow and Keras. We realized that the Azure Face API was much more convenient and switched over to that.

What's next for Easymoji

Firstly, we would like to have more emojis and have our app recognize hand gesture emojis in addition to faces. Secondly, we would like to create our own PyTorch model that we will have full control over to use for the recognition of emotion and hand signs.

Built With

Submitted to

Hack the North 2019

Created by

I worked on the machine learning aspect of our project with Rohan and Borna. I worked on our machine learning model and transformed our dataset to include the locations of facial landmarks. Although our model only reached up to 40% accuracy on its cross-validation tests, we were able to make something that was (somewhat) functional without much prior knowledge in pytorch.

Trevor Du
hi, i like coding
I came up with the idea and worked on training a machine learning model created in PyTorch.

Borna Sadeghi
Tryna improve my skills
I worked on the frontend aspect of our app using Tkinter and other Python libraries and connected it to Azure's Face API for emotion detection.

Eric Xiao
CSE student @ UW-Seattle Paul G. Allen School
I made the OpenCV facial tracking system that gives visual cues to the user when their face is visible and can make a dot map of their face in real time.

Rohan Shetty