The Future of Computer Science Computer Science, Software Engineering and Information Systems are international qualifications, enabling people to work globally, and in a very broad variety of roles. Through this hackathon, we wanted to bridge the gap between men and machine in a creative way, so we decided to create a project that can make computers understand not only the literal meanings of our language but also the hidden emotions underneath them.

What it does

Our project takes in an recording and detects the emotion of the speaker. This is done by analysis of audio and analysis of the language used. Thus, our app can detect the language tone and emotional tone. Furthermore, a text input can also be analyzed for language tone.

How we built it

We utilized Bootstrap to create the website and Google Charts to represent data. For the backend, we used IBM Watson for language tone analysis and speech to text and Vokaturi to analyze emotional tone. The Node.js server built on the express framework is used to call the IBM Watson APIs and a Python script is used to interface with Vokaturi.

Challenges we ran into

We had to learn CSS and Javascript to overcome some technical difficulties when working on the front end, we also ran into problems deploying our Node.js server on the backend. Time constraints limited the amount and type of graphs we could show, otherwise, we would of wanted more informative charts to represent our output.

Accomplishments that we're proud of

Exploring Vokaturi and IBM Watson was very rewarding as we got to implement the basic features of both while also getting a glimpse of their full potential. Although using Bootstrap and Javascript were frustrating, we are proud of how much we learned and the final product we were able to create using them.

What we learned

We learned how to use Vokaturi, IBM Watson and Bootstrap. Beyond these, we have also learned to embrace our challenges with passion, stamina, and willingness to learn and improve ourselves.

What's next?

We envision expanding our app to respond to real world applications. To do this, we need improve use of analysis apis. IBM Watson's speech to text can differentiate between different speakers, and Vokaturi can be used to analyze smaller portions of the entire audio file. Inclusion of these features would allow us to detect the emotion for each speaker at a specific time. To represent these improvements, we will also need to make our data visualization more robust by using more informative graphs.

Share this project: