Inspiration

One member of the team is a part of a public speaking club at his work. There are several people with tasks such as timing, keeping track of crutch words, false starts, ect. We set out to develop an application that would allow people to practice public speaking in the comfort of their own homes with a similar form of feedback.

What it does

SpeakEasy is a web application that allows a user to record themselves practicing their speech. It will give real time updates of the users pace in the form of words per minute, allowing them to either slow down or speed up their talking, use of crutch words such as like, um, basically, ect so they are pointed out and the user can make an effort to remove them from their speech, and hand tracking which will alert the user if they are moving their hands around too much, as it could be very distracting to an audience.

How we built it

Audio SpeakEasy was built with several different technologies. The audio side was processed using Google Cloud Platforms Speech-to-Text. This was then processed by a python script to search for any crutch words being used, as well as keeping track on the users pace. Video The video was recorded and passed through a machine learning TensorFlow model to track the users hands. A Python script was used to calculate the distances the user's hands move. Next, OpenCV was used to display the video and show where the users hands were. Front End The outputs from the audio and video handling were then passed to our front end using a Flask server. The front end was built with HTML, JQuery and AJAX to display a dashboard of all the real time information being collected.

Challenges we ran into

Neither of us had any solid front end experience, so there were several challenges in the development of the application, including learning how to properly use all of the tools to make a user friendly dashboard. In the backend, there were many challenges with getting Python scripts to run concurrently on threads and dynamically access the scores while they were running. There were also some issues with our operating systems, as one of us was working on a PC and the other was working on a Mac, and the Mac was unable to run certain parts of the program.

Accomplishments that we're proud of

We were able to learn and effectively use new technologies that we hadn't gotten to work with before. We were also very proud when we managed to get all of the individual parts of the project working on their own, and we were especially proud when we managed to bring them all together in the web application even though neither of us had any experience building web applications before this.

What we learned

We learned how to integrate powerful tools such OpenCV all into a single functioning project. We also learned about front end development and learned some JavaScript. Another important topic and skill we learned was about multi threading and multiprocessing in Python.

What's next for SpeakEasy

There are many additional features that could be added to even further improve SpeakEasy. Some of these features include a timer on the webpage, the ability for the user to upload their speech and detect whenever they make mistakes from what is written down, a results page that allows a user to see their overall performance for their speech, the ability to playback their speech and watch/listen with the word per minute and crutch word counters. The user should also be able to save and download the videos with all their associated statistics for offline viewing, among many other possible additions to SpeakEasy.

Share this project:
×

Updates