Wave+

Inspiration

I am just one of an estimated 150million league of legends players worldwide. As a league of legends player though- I'll let you know that I have problems being conscious of my mental health- especially when I'm losing games one after the other.

Now you may ask- "okay, if you're not having fun and it's negatively impacting your mental health, why do you keep playing it?" Great question. Maybe one of the other 150 million players can give a better answer, but honestly- winning is addicting. Losing... not so much.

I tend to have a "can't end on a loss" mindset when it comes to league, and this has honestly affected my mental health in a negative way. In hindsight, it would have been much better for me if I took a break after a bad game, and came back to it later. That's what Wave+ aims to do. It aims to keep mental health at the forefront and give you incentives to take a break once in a while if you're constantly losing games.

What it does

Wave+ runs in the back of your local machine- it works with a webcam or an attached camera input to identify the emotions you have when you play league in real-time. For the sake of the demo, I set it to trigger the moment it detected anger- but one can hypothetically set it to collect the overall time you have an angry look on your face throughout the course of the game. The more angrier you are- the longer your "anger score" and especially if you lose your game- that's when Wave+ hits certain perimeters and sends you a text, advising that you should stop playing (at least momentarily).

The text itself isn't that special- but it contains data specific to the user (the user manually has to set this up), and it scrapes data off of op.gg and riot's api's to tailor custom responses to the player. Running a few scripts off of my data revealed for example, that after losing a game, I win my next game 48.74% of the time- and that number gets lower and lower the more I lose.

How we built it

OpenCv, Tensorflow, Kerias, python, a LOT of time letting the computer train itself on data, the twilio api (for text support), the riot api, and some beautifulsoup packages for data scraping.

I honestly think this has been the most technically difficult project I've ever done, considering I knew nothing about ML/neural networks before I started this project.

I trained the neural network on the FER-2013 data set- which consists of 35887 grayscale, 48x48 sized face images with seven emotions - angry, disgusted, fearful, happy, neutral, sad and surprised. Opencv was used to integrate the camera with the dataset, while tensorflow was used to classify frames from the camera feed into distinct facial categories. The category that was the best match % was displayed on the screen, in real time. I set certain "trigger" categories like anger showing up- to start the messaging python script. The python script would pick a random message template, and load in data from the player's past games (mostly win/loss counts)- and send a text to the player with their data displayed.

Challenges we ran into

So many. Considering I had never worked with ML/AI stuff, I was in over my head at a lot of points on this project, and the only places that had answers were research papers. I had issues even getting the camera working- which was the most basic step honestly and I almost gave up at that point. The code remained buggy, packages and dependencies had a habit of breaking over all sorts of seemingly random stuff. Despite this, I was able to get the practical parts of the ML working and the texting script right at the 11th hour. There were several other technical difficulties (at one point my computer crashed and I lost almost everything) but in the end I persisted and was able to deliver.

Accomplishments that we're proud of

It's finals week and it took 2 days of all-nighters, but I managed to build something cool.

What we learned

So much. I'm not really sure I processed a lot of it, and won't get a chance to process it until finals are over haha. My mind kind of went on autopilot of "this doesn't work, let's brute force until it does" and stuff somehow worked out. I learned about the math, the process, how to set up a deep-learning neural network, the sheer amount of work that goes into these kinds of things, and a lot about my technical abilities- my limits of what I can and can't do.

What's next for Wave+

I already have some interest from some friends who are interested in beta testing this if it becomes an actual product. The issue is that it takes a lot of time to train the deep learning model on the FER-2013 dataset like I said before. Also downloading the packages/dependencies... is something like several gigabytes of data... so probably if I'm able to somehow ship it in a smaller package that might be the most rational step.

Built With

• machine-learning
• network
• neural
• opencv
• python
• riotapi
• tensorflow
• twolio