Our smart note taking app revolutionizes the way you take and organize notes. With AI-powered features like auto-summarization and smart semantic search, you can quickly and easily find the information you need. It's perfect for university students and industry professionals alike who want to be more productive and efficient.


During our lectures, we spend most of our time typing and catching up to what the professor is saying. And after all that typing, we then have to spend time summarising, formating, and cleaning our notes! Not to mention the pain that comes from looking for specific information through dozens of note documents. Taking notes is just way too much time and effort. We wanted to simplify and improve the process of note-taking and give users an easy and efficient way to write, organize, and search through your notes.

What it does

NoteBuddy allows users to record audio or upload existing audio and auto-generate a transcript for it. You can record lectures, meetings and much more! The transcript is automatically saved to the user's Google Docs and the NoteBuddy platform. The AI-powered auto-summarization feature generates a short summary of the most important points. It also allows for smart semantic text search, which allows you to ask questions and find the answer through all your notes in just a few seconds.

How we built it

The frontend is built with Vite, JavaScript, and React while the backend is built with python and flask. We used Appwrite for OAuth 2.0 authentication to use Google's GSuite APIs. We also used Appwrite's database to store data related to the user and their documents. We used OpenAI's Whisper model to transcribe audio into text and Google's Flan-T5 model to summarize and search through text. Both of these models were deployed to high-performance compute units on the cloud as our machines were not powerful enough.

Challenges we ran into

The biggest challenge we faced was perfecting semantic document search. To get it to work, we needed to semantically search the documents and answer questions based on the information in the document. We needed to minimize false-positives because we didn't want the user to be directed to a wrong document to get an answer. We were able to overcome false-positives through some clever prompt engineering and model tuning.

Accomplishments that we're proud of

Seamlessly integrating Google Docs with our platform through Appwrite's OAuth 2.0 service.

What we learned

We learnt a lot about deploying machine learning models and fine-tuning their parameters to get accurate results to best fit our use case.

What's next for NoteBuddy

We want to further fine tune the models so they work better for our specific purposes. While we did do some tuning so we could make the models work for our needs, it could still be made more efficient and accurate. We also want to implement more features to allow users to collaborate with each other and make their experience better, such as the ability to import existing documents.

Built With

Share this project: