Our team’s mission is to build a tool that alleviates stress on individuals that partake in extracurriculars.
With individuals scheduling up to 5 meetings a week for extracurriculars, we noticed the rise of Zoom fatigue and the decrease in engagement as individuals were tasked with creating minutes. We decided to come up with a solution to take this additional work off their shoulders by creating min.it; a web application that can record meetings and output meeting minutes in real-time! With min.it, individuals no longer need to struggle with typing down unfinished thoughts all while participating in the conversation. With a click of a button, the tedious work from meeting minutes can be done for you.
What it does
min.it is a web application that makes meetings simple by generating meeting minutes in real-time. min.it will join a conference call and immediately start recording the conversation in real-time. As this is happening, the audio data is transcribed and simplified using natural language processing. After the text is simplified, phrases will begin to output on the min.it webpage. You get to see your meeting minutes come to life!
How We built it
We used Vonage’s Communication API to dial into group meetings to stream live audio data to IBM’s Speech to Text API via Websocket connection. Then we implemented a natural language processing model was built using NLTK, Tensorflow Keras, and BERT to perform lexical simplification (simplifying sentences by replacing complex words). We then used Firebase to store the data for the generated meeting minutes to allow users to edit the minutes in real-time.
Challenges We ran into
This was our first time developing a working full-stack project which brought us several challenges that we tackled head on! Firstly, setting up the WebSocket was a new concept that encouraged us to watch video tutorials to know where to start. Upon developing the knowledge to set them up, it was challenging to combine the WebSocket with the IBM API. We endured issues with relaying binary data between the two which was resolved by using a blob object (binary large object). Overall, these challenges allowed us to use our problem solving skills in tandem with our programming knowledge to understand the connections between all the technologies. Without running into these challenges, we would not be as well versed in these tools and we are happy to have persisted through them.
Accomplishments that we're proud of
Our team was incredibly proud of learning all the new concepts needed to complete this full-stack project. There were several ideas we had in mind to solve this problem, however, there were new tools we had to learn to increase the functionality to output the best results. Some of these concepts included using websockets, NLP and different API’s to facilitate the development of the program. Furthermore, we are proud of supporting a challenge that arose during COVID-19 that has brought team motivation down due to the increase in virtual meetings. As students, we are aware of how tedious taking meeting minutes are and the challenges posed from moving to an online platform. It has been an incredibly interesting program to embark on and we are excited to see what it can accomplish. Finally, we are most proud of completing all aspects of the program, executing a working, full stack program. This was the first time we applied all concepts in tandem with one another and we were able to bring the modularity and abstraction together.
What's next for min.it
Min.it has a bright future ahead! We want to incorporate more accurate speech-to text results by training our model with the additional time available. This would increase the accuracy of the speech recognition as well as the associated confidence scores while different individuals speak. An additional feature we would like to improve are the stylistic effects of the documents that get outputted so that users can customize to their preferred style. Finally, we plan to implement abstraction-based text summarization that will allow for better simplification of sentences to draw out key information. Due to the time constraints we were not able to incorporate these enhancements but are excited for the future of min.it!