Inspiration

We're a culturally diverse team; we understand the obstacles that language can pose for us and our loved ones. Many people are capable of doing great things, but are inhibited by language and communication. We don't think this is fair. We believe actions speak louder than words, but understand that words are still vital. This is our take on a system that helps students like us develop our abilities to convey our thoughts without hesitating and without feeling nervous.

What it does

It takes a video of the student, analyzes it and performs various cognitive processes on it. Cognitive processes include tracking facial expressions/emotional changes over time and speech analysis. Using Google's speech to text API and NLTK, the recorded speech is converted to its equivalent text, stop words are identified and better alternatives are suggested for certain words (adjectives) to improve the overall quality of the speech. Using Microsoft's Cognitive Services API for Emotion Analysis the video is processed, emotions are computed over time and graphically displayed for better understanding of variations.

How we built it

The front end has been implemented using HTML, CSS, JavaScript and, Flask and back-end has been built entirely on Python. The video is taken which is then used to extract the audio counterpart. Frames from the video are sent to the Emotion API to retrieve the emotions. Audio is sent to Google Cloud Services API for speech synthesis

What's next for SpeakEasy

With more time, we'd work on incorporating sleeker visuals. We'd include database capabilities to allow users to track their progress over time and access their past recordings. Lastly, we'd love to make SpeakEasy mobile, and thus more accessible.

Share this project:
×

Updates