Current chatbots have no social skills. There is no way to connect friends with video chat and text chat to communicate with emotions.

What it does

Vdo2txt embeds emojicons in text based on facial expressions and speech from video chat. Text with emojicons are use to communicate with friends in group chat. Natural conversational agent uses social communication processing to interact with friends. Please see details in slides.

How we built it

We used python, incorporated Azure Cognitive API and our own model to analysis the facial expression and speech to embed the emojicons to text so that the agent(bot) can interact with friends with emotions.

Built With

  • microsoft-cognitive-services-api
  • python
Share this project: