Millions of people face anxiety daily when dealing with regular social interactions. Especially in today’s tech-driven society, social interaction has been at an all-time low, due to the increasing importance of screen interactions. However, it's difficult and time-consuming to overcome anxiety with current conventional practices. Furthermore, there currently aren't any defined and effective ways to practice social interaction. Confidant aims to change that: we'll bring confidence into your social interactions, while being efficient, convenient, and painless.

What it does

Confidant helps people learn to overcome their social anxiety, helping them in their daily lives. Using cutting-edge eye detection technology and speech recognition, Confidant tracks eye contact, attention, and response cohesion. From these factors, we can identify your level of social anxiety and attention deficit, and give you advice on how to improve. This technology powers simulated practice conversations that you can initiate from the convenience of any device with a front-facing camera. You can use Confidant anytime, anywhere to practice and overcome your anxiety.

Use cases

  • Individuals can use Confidant for interview and conversation training, including anxiety reduction and social skills improvements.
  • Confidant can be used to detect and improve the social interactions of people with social impairments, such as autism and ADHD.
  • Service employees can use Confidant to train how they interact with customers.
  • Similarly, companies can use Confidant to train their employees to better respond to their clients' needs.
  • (American Airlines) Confidant can be used to train flight attendants and gate attendants to better engage with passengers, in order to improve passengers' travel experience.
  • (The Home Depot) Confidant can enable store employees to elevate the level of customer service they provide to millions of customers nationwide. Furthermore, anxiety reduction will allow for employees to build stronger, deeper, and more meaningful relationships with customers.
  • (PIMCO) Our product aligns with PIMCO's ESG metric, allowing the company to decide potential investment opportunities by revealing issues in a company's treatment, training, or conduct practices.
  • (Facebook) Reducing anxiety during social interactions is a fundamentally unsolved problem we face in this world today. Confidant will build community by fundamentally improving the quality of day-to-day social interactions, bringing the world closer together.

How we built it

Confidant leverages the most cutting-edge and advanced technologies. Our frontend uses React.js + Next.js to render our website. Our eye-detection technology uses a library called WebGazer that runs on the client side. Our backend API is built using Flask, hosted on Microsoft Azure Virtual Machines (VMs). We extensively use Microsoft's Speech Services (both text-to-speech and speech-to-text) to process the simulated interaction between the human and the Confidant interface. Finally, the Confidant companion is powered by the VOCA speech to animation engine.

Challenges we ran into

We used cutting-edge libraries with limited documentation and unrefined code. It was quite a challenge to get a working prototype with them. Furthermore, due to how state-of-the-art our technology is, we had to work with unpolished academic paper code, taken straight from the most recent academic research.

Accomplishments that we're proud of

We were able to establish a pipeline to convert raw user eye movement information and voice into meaningful data and voice response by using multiple Microsoft Azure services and our custom backend API.

We're able to give users a comprehensive score that's able to judge their anxiety levels and empathy levels. In the future, we're easily able to extend this to more interpersonal metrics.

What's next for Confidant

Confidant addresses pervasive issues with technologies that have far-reaching potential to make people's lives better by giving them confidence.

In the future, we plan to integrate unconscious bias detection and training by allowing users to train on a diverse set of conversation companions (differing in skin color, gender, and age). We believe that respect for all people and equal treatment is a key aspect of a progressive society; Confidant is a significant facilitator to make that happen.

Share this project: