My Therapy Bot

Your Emotions, Your Words - Chatbot that Listens to Your Face, Understands Your Feelings, and Lets You Speak from the Heart

Comment

Inspiration

At the heart of our creation lies a profound belief in the power of genuine connections. Our project was born from the desire to create a safe space for authentic conversations, fostering empathy, understanding, and support. In a fast-paced, rapidly changing world, it is important for individuals to identify their mental health needs, and receive the proper support.

What it does

Our project is an innovative chatbot leveraging facial recognition technology to detect and respond to users' moods, with a primary focus on enhancing mental health. By providing a platform for open expression, it aims to foster emotional well-being and create a supportive space for users to freely articulate their feelings.

How we built it

We use opencv2 to actively analyze video and used its built-in facial recognition to build a cascade around individuals' faces when identified. Then, using the deepface library with a pre-trained model for identifying emotions, we assigned each cascaded face with an emotion in live time. Then, using this information, we used cohere's language learning model to then generate a message corresponding to the emotion the individual is feeling. The LLM is also used to chat back and forth between the user and the bot. Then all this information is displayed on the website which is designed using flask for the backend and html, css, react, nodejs for the frontend.

Challenges we ran into

The largest adversity encountered was dealing with the front-end portion of it. Our group lacked experience with developing websites, and found many issues connecting the back-end to the front-end.

Accomplishments that we're proud of

Creating the cascades around the faces, incorporating a machine learning model to the image processing, and attemping to work on front-end for the first time with no experience.

What we learned

We learned how to create a project with a team and divide up tasks and combine them together to create one cohesive project. We also learned new technical skills (i.e how to use API's, machine learning, and front-end development)

What's next for My Therapy Bot

Try and design a more visually appealing and interactive webpage to display our chat-bot. Ideally it would include live video feed of themselves with cascades and emotions while chatting. It would be nice to train our own machine learning model to determine expressions as well.

Built With

Share this project:

Updates