Meetings are painful and sometimes boring! What's the worse? Having to write the meeting minutes and try to remember what everyone has to take away from it! Can we make that simple with speech and speaker recognition to (i) help participant to pay attention when needed and (ii) provide them with meeting minutes and notes at the end.

What it does

WavaW is a webapp that records the meeting and shows in real timeline the conversation : who says what ? Additionnally the app can :

  • send each user notifications in case certain keywords, that he chose, are mentioned in the conversation.
  • take a short note when the user says : "take note : blablabla"
  • show the speaking time of each person

How we built it

The front end is a simple webapp using javascript. The backend is a server using the simpleHapi framework with node.js. For the audio management, we record from the browser and stream the data through a socket connection. The processing is done in the backend and uses the microsoft API for "speaker recognition" and "speech to text".

Challenges we ran into

The audio format to send to microsoft was hard to handle, in particular because the sampling rate depends on the client's device and we have to re-sample in the backend. Aside that, we have taken a decision to process the audio by small chunks for this hackathon. A bigger challenge will come, with the possibility to cut each piece of audio based on whos talks and thus generate a better speech to text, and also to possibility to interpolate and get a verification.

Accomplishments that we're proud of

We are poud of: the clean UI we have made in a record time! the fully working solution, with the complete stack! a interesting and original use of the speech recognition API!

What we learned

API's are powerful but not as precise as they seem and it is something to take into account, especially when considering how to cut or stream our data. And for some of us, we have learnt a great deal on light server setup and javascripting.

What's next for WAVAW for LauzHack

For LauzHack, we are done and we intend to participate to the challenge of Swisscom and Microsoft. For after LauzHack, there is a whole room for improvement on two mains topics :

  1. Improve the usability and the value that we bring to our users, by saving the minutes in a friendly format, adding reminder features liked to the calendar and many more.
  2. Improve the speech and speaker recognition, by using custom cut chunks of audio instead of cutting regularly, and by using interpolation. And get even more familiar with the API to be able to get the best results.
Share this project: