From showing up to interviews to performing well at school debates, we have all experienced how difficult it is to be confident. Oftentimes, as debaters, we do long, time consuming drills to develop our confidence, oftentimes requiring the help of one or more friends. Fidenti came from our realization that a lot of this feedback can be given through the computer.
What it does
Helps to train speaking skills and confidence with the help of text-to-speech and AI-powered eye tracking. Eye tracking helps you maintain eye contact while text-to-speech has many different modes that allow powerful analysis of word frequency, filler word filtering, and text matching with a provided script.
How we built it
We combined many different platforms to create Fidenti. Text to speech is done with IBM Watson, while AI eye analysis is powered by a custom deep learning algorithm. The frontend was made using React and Plasmic. Python is used to process and analyze transcript data. Throughout the project, we split the features we wanted to implement among ourselves and worked separately, integrating in the end.
Challenges we ran into
Obtaining API keys and dealing with free licenses and limited amounts of calls AI training: finding good training data and processing the end result. React is frustrating and buggy. Working continuously for hours on end with little sleep :)
Accomplishments that we're proud of
I am impressed by the amount of different features we managed to implement, including all of our different modes and the eye tracking. We are also proud of the frontend website built using react as it is one of the most polished and good-looking websites we have managed to create so far.
What we learned
We learned a lot about frontend development, including consistent theming, the use of Plasmic, and cooperation with Git. We also learned about AI image processing, text to speech, and text analysis.
What's next for Fidenti
We plan to implement a lot more features, including posture/gestures detection: analyzing posture using AI so you can portray confidence with your stance, online connection to help you compete in online leaderboard for additional motivation more modes, analysis mode that replays past speeches, distraction mode trains you to ignore sounds and other distractions