SummItUp

Meetings, Minus the Manual Work

Comment

Inspiration

Post-COVID , virtual meetings have become an indispensable part of our lives. We all can relate to the struggle of getting a attaining a thorough understanding of a meeting we missed, or even when we need revisit our past discussions after some time has passed and this became the driving force behind our project. While exploring Atlassian's products, we identified an opportunity to seamlessly integrate meeting insights into Confluence . Fueled by this observation, we embarked on creating SummitUp, a solution that bridges this divide, streamlining the extraction of valuable insights from meetings, and making the process more intuitive , effective and automatic with AI technologies.

What it does

SummItUp is an app that automatically generates meeting summaries and action items, facilitating seamless collaboration and efficient meeting management. It extracts insights and identifies action items based on the meeting content. The app integrates with Atlassian Confluence by automatically updating Confluence pages, providing a central hub for teams to access and contribute to meeting summaries and related documents.

Benefits and Impact

SummitUp transforms meeting documentation and collaboration by:

  • Saving time and effort in documenting meetings .
  • Ensuring that action items are never missed.
  • Enhancing post-meeting follow-ups.
  • Accelerating Decision-Making
  • Allowing Atlassian Confluence Integration
  • Providing a AI powered insights to make confluence reliable and intelligent source of information for decision-making.

Therefore ,resulting in improved collaboration and productivity within teams.

How we built it

We built this project using a combination of audio processing, machine learning, and natural language processing techniques. Initially, we loaded and processed the audio file using the 'librosa' library. We then applied Gaussian Mixture Models (GMM) for speaker diarization to identify different speakers in the audio. The data was further analyzed using Agglomerative Clustering to group similar segments together. For text summarization, we leveraged OpenAI GPT-3, generating summaries for each text partition. We then identified action items the text summaries. To manage the audio data, we used the 'moviepy.editor' library to divide the audio into parts for further processing or analysis. This project was developed in Python and involved data management using Pandas DataFrames to organize and analyze the results, enabling us to extract meaningful insights and information from the audio data.

Our application’s backend was hosted on an EC2 server, utilizing Flask for development and an Nginx Gunicorn reverse proxy for production to facilitate communication between our API and the app. In the post-processing stage, we utilized the Atlassian REST API endpoint for content to create pages as child pages, complete with summaries and action points. For the computational aspect, we opted for a g4dn.xlarge instance, providing ample RAM for model training, along with a Tesla T4 for GPU computing.

Challenges we ran into

  1. Accurate Speaker Diarization was a notable challenge. We initially explored common approaches using libraries like pyannotate and spectral clustering but encountered issues, such as misattribution during long pauses. Pretrained models, including those from Hugging Face, also posed challenges. Ultimately, we opted to train a custom Gaussian Mixture Model (GMM) and use agglomerative clustering to cluster audio into speaker-specific frames, overcoming diarization challenges and emphasizing the value of custom solutions in complex audio processing tasks.

  2. Navigating the Atlassian Ecosystem was another challenge we came across. As newcomers, understanding Atlassian products and building an app on the Forge platform presented a steep learning curve. We encountered various challenges during the process, but with the help of documentation and tutorials , we found our way.

  3. We faced issues in permissions when connecting the external API, forge lint was helpful.

  4. Forge expects us to have SSL certificates , but certbot does not issue certificates to certain endpoints that includes AWS EC2. This means you should deploy our code on a domain registered server to be able to interact with forge seamlessly.

Accomplishments that we're proud of

We are proud of creating an innovative solution that simplifies meeting management, fosters teamwork, and enhances knowledge retention. Our app's integration with Confluence sets it apart, making it a valuable asset for teams using Atlassian products.

  1. Successful resolution of challenges in speaker diarization and development of a custom GMM model combined with agglomerative clustering to achieve this.
  2. Creating a Forge app and implementing a external REST API interaction.
  3. Successful integration of UIKIT2 and 'confluence-macro' template for app design, with no prior experience
  4. Model development and Training on an external server, since our local computers were not computationally powerful enough and Successfully hosted the app's backend on an AWS EC2 server .

What we learned

  1. Throughout the development of SummItUp, we deepened our expertise in AI model integration especially in audio processing and analytics like voice activity detection, noise reduction , speaker annotation , diarization etc. We gained insights into the complexities of natural language processing.
  2. We embraced a steep learning curve, continuously adapting to the tools and technologies ,from Atlassian Forge.
  3. Our journey underscored the critical importance of safeguarding sensitive data. We learned to prioritize data privacy and security throughout the development process, ensuring our users' information remains protected. We don't store any information, we delete all the data after every call .

What's next for SummItUp

The future of SummItUp holds exciting possibilities. We plan to enhance the app's AI capabilities for even more accurate meeting summaries and action item assignments. We want to perform actions based on certain triggers (such as emails and notification among teammates) .We're exploring integrations with additional collaborative platforms like slack and want to expand the app's language and region support to broaden our user base. Furthermore, we aim to implement more responsible AI principles. Ultimately, we envision SummItUp as an indispensable tool for efficient meeting management and knowledge sharing within organizations.

To have a look at our presentation , open the below link : link

Share this project:

Updates