Emosion

• 

  Emosion

Inspiration

My inspiration for Emosion slowly developed over the period of the Covid-19 pandemic and remote learning as a result of it. Remote learning was nothing like the original in person learning, it was especially hard for teachers and educators who couldn’t properly tell whether they’re students were able to understand the taught material. Sure there were assessments, but that was only after all the material was covered, many students. Along with that, many students, especially introverts never spoke out about their understanding of the topic, making it harder for the teacher to understand as well. As a solution, Emosion was created.

What it does

Emosion allows a teacher/educator/presenter to set up a virtual classroom, to which students/listeners would be able to connect to. Upon connecting, Emosion would use a pre trained machine learning model on the client side to process the user’s emotions from their webcam. The emotions would then be broadcast to the host, who would see a summary of the students emotions as well as individual emotions of each student in real time, this would make them better understand the audience’s feelings.

How we built it

Emosion consisted of two main parts, the machine learning model that would actually get the users emotions in real time, and the infrastructure that would display the emotion data to the host. The emotion detection system was essentially built with tensorflow and an open source library called face-api.js, the frontend was built with Vuejs, a javascript framework, and the backend was built with Nodejs and socket.io, a js runtime/library.

Challenges we ran into

1. Initially deciding on an idea that would incorporate AI/machine learning and realtime technologies
2. My first time using AI/machine learning on images/videos in real time
3. Getting the real time system working without any bugs
4. Integrating the emotion detection system into the browser seamlessly without bugs

Accomplishments that we're proud of

1. Getting the webapp to work exactly as I wanted to with no detected bugs (as of now ;D)
2. Being able to design and develop the webapp in under two days

Built With

• javascript
• machine-learning
• node.js
• vuejs