Inspiration

What it does

We will build a desktop buddy for an in person lecture. A student imports a slide PDF, starts microphone recording, asks for help with an “I’m lost” button, and receives organized notes at the end. Gemini turns speech into text and connects that text to the uploaded slides.

How we built it

We built a desktop app with Python using pyside6, and used the Google Gemini API to handle audio transcription, note generation, and summary.

Challenges we ran into

We ran into challenges with coordinating different interactions simultaineously, such as having the app constantly record the audio and update the UI in the background whilst also allowing the user to further ask questions about certain slides. Another challenge we faced was attempting to balance the time spent planning out the implementation details and the actual development work whilst making sure that we completed the project on time.

Accomplishments that we're proud of

One accomplishment that we are especially proud of was getting both accurate live audio recordings, note generation, individual slide explainations, and character animations to work seamlessly with eachother, accounting for different user interactions.

What we learned

This was the first time most of us worked on a proper desktop app, so it was really exciting to learn about how desktop GUIs are designed and developed. This was also the first time we were able to use Gemini to the extent that we were able to do, with live transcription generation, note summarization, and the combination of both to synchronize both the audio and lecture slides. This project allowed us to learn about how to combine all of these in a way that felt satisfying as a user and useful.

What's next for Study Buddy

We want to add more features such as better interaction between the mascot and user actions on the app to make it feel more alive.

Built With

Share this project:

Updates

Submission history