Today life is brimming with pressure and tension and particularly in metropolitan urban communities circumstance is the most noticeably terrible. Everybody is occupied with his life, no time for family and himself even. More than 2 in 5 people globally reported symptoms of depression and anxiety during the pandemic. As students, we have realized that a lot of our time is spent in front of a laptop and first-hand watched our mental health deteriorate during the pandemic. Entertainment can include salt in this distress and tacky life. Music, dance and movies are an extraordinary method to relieve the everyday pressure. Pretty much every individual is keen on any sort of music, dance or movie. And that's why having an application like Entertrain is so important!
What it does
Entertrain is an app that allows users to enjoy music and movies personalized according to their current mood and even provides AR/VR dance step demos.
We analyze your mood using just two inputs - your video and your text. The features of our product are numbered below:
First, it can register the "mood" of a person. This is very important to provide automated action and suggestions from the website and to the user. The product can detect a total of 4 moods: happy, sad, angry, and neutral through the webcam of the user. With different moods, the website can perform different actions. If the user is happy, the website suggests if it should play the user's favorite Spotify playlist. If the user is sad, the website suggests if it should tell the user a joke and play upbeat music. It can also ask "check-up" questions. The website also plots a graph of your emotions in a fixed time necessary and even suggests discovering various rooms such as meditation, relaxing , etc.
There is a friendly chatbot to answer your questions and can even alert the user's relative/friend through SMS if the user gets suicidal and depressed to a certain level. This can be of great help to save someone's precious life.
The app can also be used to get a movie suggestion according to a selected genre and mood.
Finally, we have a flowing UI catering to the needs of the users. We have tried to make the overall look and feel to make it calming, soothing, safe and engaging enough for our users.
We also have a AR/VR Dance demo section.
How we built it
Video Emotion Detection
Emotion detection was also doing using Face recognition API. We used a custom open-source model that we found online to help us process user's emotions. We then extracted the user emotions from the API and connected them to timers on the frontend. This allowed us to have a Hashtable (using chart.js) that contains all of the data on how many minutes the user feels each emotion during the duration of the video recording.
We suggest playlists according to the mood of the user using Spotify API.
Chatbot alert system
Movie Recommender System
AR/VR Dancing Zone
We used echoAR to implement few dance step demos in AR/VR so that our users can learn dance steps from the ease of their houses.
Challenges we ran into
We were literally racing against time. The building of a good UI with all kind of functioning of different APIs and integrating each of them according to our user's need was indeed a big challenge for us. There were a variety of bugs to fix till the last moment. Implementing AR/VR through echoAR was also very difficult due to various errors showing up.
Accomplishments that we're proud of
This project was especially an achievement for us because we created many additional components on the front-end in order to have a completed project. We are also particularly proud that we were able to include a vast array of different features for the user, which we felt makes Entertrain such a serviceable form of digital entertainment. Lastly, we think the impact our project could have is a significant accomplishment. Especially in the digital age combined with remote day-to-day communications, this could really be a product that people find useful!
What we learned
We have a learnt a plenty much investing time and efforts in this project. Use of various APIs and integrating them plus a UI design to implement in front-end in such a short span of time was unbelievably challenging.
What's next for Entertrain
Entertrain has so many useful features, so we'd love to further expand the project to handle a variety of online shows and podcasts! Overall, we hope that one day this project can be widely used as an excellent resource for entertainment.