We wanted to solve a problem that was real for us, something that we could get value out of. We decided upon Vocally as it solves an issue faced by a lot of people during job interviews, presentations, and other occasions which include speaking in a clear and concise manner. The problem was that it takes a long time to record yourself and re-listen to it just to spot any sentence fillers like "um" or "like". We would like to make it easier to display statistics of one's speech.
What it does
The user clicks the record button and starts speaking. The application first converts speech-to-text using React's built-in speech recognition. After analyzing the results various text processing techniques (e.g. sentiment analysis), it displays feedback.
How we built it
- First, we needed to see how keywords could be extracted from an audio recording in the back-end. We settled with React's speech-to-text feature.
- Next, we created API endpoints in Flask (a python web framework) for the React app to make requests from.
- Fuzzy string matching, grammatical, and sentiment analysis were used to process and return the stats to the user using data visualization.
- The last task was deployment to the pythonanywhere.com domain for demo testing purposes.
Challenges I ran into
Using flask as an API was easy, but we initially tried to host it on GCP, which proved to be difficult as our firewall rules were not configured properly. We moved onto pythonanywhere.com for hosting. For the front-end, we first decided to take a look at the Flutter framework to be able to make the application mobile accessible but the framework was introduced in 2018, and there were a lot of configuration issues that needed to be resolved.
Accomplishments that we are proud of
Getting the sound recorder to work on the front-end took longer than expected, but the end result was very satisfying. We're proud that we actually achieved creating an end-to-end solution.
What I learned
Exploring different framework options like Flutter, in the beginning, was a journey for us. The API that was created needed to delve deeper into the python programming language. We learned about various syntactical and natural language processing techniques.
What's next for Vocally
We may re-explore the concept of natural language processing, perhaps build our own algorithm from scratch and do more over a longer time period.