Life is an emotional rollercoaster. You go through happy moments, sad moments, angry moments, and get surprised with the amount of ups and downs throughout the week. Sometimes, it's difficult to convey these emotions to others let alone identify them ourselves. We often forget to check in with what's going on in life and ask "How am I feeling right now?" and "What can I do about it?"

What it does

Light-UP detects facial emotions with machine learning and uses colours and light patterns to help improve your mood and light up your day! If you're feeling sad, Light-UP will detect it and give you flowing, calm, warm, and soft lights to cheer you up. When you're happy, it will reflect your emotions and show you twinkling warm joyful colours. If you're angry, Light-UP will calm you down with calm, cool, flowing blues. And finally, if you're surprised, Light-UP reflects your excitement with fast flashes of random colours!

How we built it

The project is a web app built with Vue.js. It uses the javascript library Tensorflow.js, along with KNN Classifier and MobileNet to access Machine Learning features. It can be trained to recognize facial expressions through a prebuilt Webcam component. The specific emotions that are recognized in realtime are saved to a Firebase database in the Google Cloud Platform. The arduino board pulls in the current emotion from firebase and updates the array of LEDs according to the emotion's matching colours, pattern, and speeds.

Challenges we ran into

To no surprise, this hackathon was also emotional rollercoaster. We initially wanted to use Google Vision's Auto ML api and spent many hours trying to figure it out, but after many technical difficulties, we had to restart halfway and switched to learning how to use vue and tensorflow instead. Similarly we had a lot of difficulties with connecting the firebase database to the arduino, but we're super thankful that we had a mentor to help us along the way!

Accomplishments that we're proud of

We were not sure if this idea was feasible in the first place and even though we faced many difficulties and had to switch frameworks halfway, we're proud that we decided to keep on going and got the lights and facial recognition to work in the end.

What we learned

Facial recognition is super cool! but also requires a lot of careful planning to set up. Also learning how to set up firebase to connect with the Arduino.

What's next for Light-UP

We imagine Light-UP could be used beyond personal use. Such as hospitals, daycares, or health facilities to help improve people's moods and encouraging healthier and positive emotions.

Built With

Share this project: