Inspiration

Upping Your Elvis Talk it Out is a simple way to improve mental well-being. The participant talks to a friend while walking; the walking helps the user to enter a state of "flow" and open u about issues in their life. The friend identifies emotional changes in the voice to highlight areas for the participant to look at. 60% of people won't talk to a human, so having a mobile app to take on the role of the listener could open this facility up to more people.

What it does

What's Your Problem looks through an audio file to detect emotional changes in the speaker's voice. This highlights the topics the user is emotionally involved with. Sections from near the emotional change are played back to the user to highlight the topics for them for work on.

How I built it

At the core of the programme is a tensforflow deep neural network that detects the emotions in the speakers voice. An audio file is uploaded and cut into 5 second chunks. Each is individually analysed by the code to extract the proportion of each emotion (anger, calm, fear, happiness, and sadness) in the section.

The emotion proportions are plotted against time to identify the changes in emotion, these points are identified by looking for a rapid change in the peak emotion. It cuts out the 30 seconds of audio near the emotional change to be played back to the user.

The Python code is accessed through a web app implemented in flask.

Challenges I ran into

Getting the machine learning algorithm to run in the web app: We tested two machine learning algorithms. Both produced reasonable plots with features that corresponded well with features that were audible in the file. The decision on which to use was based on the ease of implementing it within the web app.

Identifying the emotional changes within the code also presented a significant challenge. We achieved this by considering the most dominant emotion within the speech, and looking for points in time where this suddenly changed.

Accomplishments that I'm proud of

Producing a web app that can analyse audio files for changes in emotion.

What I learned

How to use Python machine learning programmes to

What's next for What's Your Problem?

Further training the machine learning algorithm with tagged data to increase the certainty of the flagged emotional changes. The web app should be further developed to be available on Android and IOS.

Built With

Share this project:

Updates