Happy Test - Facial Expression Detection
Sad Test - Facial Expression Detection
Angry Test - Facial Expression Detection
Neutral Test - Facial Expression Detection
Watson IBM Dialog Diagram
ChatBot Demo 1
ChatBot Demo 2

Lords_of_the_pings_HM21

BuddyBot - Your Emotional Assistant App

link to repo: https://github.com/celebrimbor22/Lords_of_the_pings_HM21

BuddyBot is your Emotional Assistant App. It detects your current mood through Facial Recognition and Facial Expression Detection (implemented using Machine Learning). After you know your results, you start a conversation with our ChatBot (powered by IBM Watson Assistant) that helps you with some advice or, if needed, provides you with the contact information of health institutions.

Face Detection and Facial Expression

The library used to detect the facial expressions of the pictures taken is face-api.js, which is a high-level interface on top of TensorFlow.js

The Neural Network used to detect faces in the provided picture is SSD Mobilenet V1, model pretrained on the WIDER FACE dataset.

Additionally, a second model, employing depthwise separable convolutions and densely connected blocks, is used as part of this interface, to detect the facial expression of a person based on the following categories:

neutral
happy
sad
angry
fearful
disgusted
surprised

How it works?

The users will take a picture of themselves with an integrated webcam. After that, with the press of a button, the face detection algorithm is run on the picture and sends back the result of the aforementioned categories.

IBM Watson Assistant

IBM Watson Assistant is a service on the IBM Cloud to help you build and deploy virtual assistants. IBM cloud service provide you with the script to use in the java script code to desplay the assitant.

Watson work by nodes, each node has an specifict funtion that can lead to another node or end there. The firts half of the diagram it havr have a series of simple question to make feel the user more confortable chating with the bot.

The next part of the chatbot consists in train the AI to recognize four basic emotions: angry, happiness, sadness, and neutral. The node methodology was continued, but this time, each node detects an emotion and proceeds to process the feeling and provide a solution. Each node was trained with keywords, which help Watson's training.

Depending on the feeling detected in the sentence, the chatbot will give a result, selected in a bank of responses and gives the telephone number and the TeQueremos link in order for the user to keep track of their emotional state.