Daysence

Inspiration

What it does

How we built itAbout the Project: Real-Time Lecture Summarizer

The Inspiration The idea for the Real-Time Lecture Summarizer was born out of a common frustration: the difficulty of taking effective notes during fast-paced lectures. As a student, I often found myself so focused on writing down every word that I would lose the overall thread of the topic. The act of note-taking became a distraction from understanding the material. I imagined a tool that could handle the tedious task of transcription, allowing me to fully engage with the professor's words and ideas. This simple desire—to create a tool that makes learning more efficient and less stressful—was the core inspiration for this project.

What We Learned Building this project was a valuable learning experience that touched on several key areas of software development and AI.

Audio Processing: I gained a much deeper understanding of how audio data is captured and processed. Working with libraries like PyAudio taught me about audio streams, sample rates, and chunk sizes. For example, I learned that a sample rate of 16,000 Hz is more than sufficient for clear speech transcription, a fact that significantly impacts file size and processing time.

API Integration: The project was a hands-on lesson in integrating different APIs. I learned how to connect the audio output to the speech_recognition library and, most importantly, how to use the powerful Gemini API to transform raw text into a meaningful summary.

System-Level Programming: The challenge of creating a "real-time" recording process on a desktop environment, especially with non-blocking input, was a fascinating puzzle. I discovered the importance of platform-specific libraries like msvcrt for handling keyboard input without interrupting the main program loop.

How We Built It The project was built iteratively, starting with a simple proof of concept and adding complexity step by step.

Phase 1: Basic Audio Capture: The first step was to create a script that could simply record audio and save it to a .wav file. I used PyAudio for this, focusing on getting the record_audio function to work correctly.

Phase 2: Speech-to-Text Integration: Once the audio capture was reliable, I integrated the speech_recognition library. I learned how to handle the audio file and send it to Google's service for transcription. This was a critical milestone, as it turned a raw audio file into usable text.

Phase 3: AI Summarization: With the transcript in hand, the next phase was to tackle the summarization. I integrated the Gemini API, crafting a clear and concise prompt to guide the model's output. The goal was to distill a long transcript into just 4-5 key points, which is where the real value of the project lies.

Phase 4: Enhancements and Error Handling: I then refined the code to be more robust. I added a non-blocking input mechanism for the real-time recording to improve the user experience. I also implemented try...except blocks to handle potential errors, such as API key issues or file system errors, ensuring the script fails gracefully and provides helpful feedback.

The Challenges We Faced The project was not without its hurdles.

Non-Blocking Input: The most significant challenge was implementing the "real-time" recording. The initial approach of using sys.stdin.read(1) was blocking and would prevent the audio recording loop from running smoothly. This led to the KeyboardInterrupt error seen in testing. The solution required a deep dive into platform-specific libraries and understanding the distinction between blocking and non-blocking input streams.

API Key Management: As demonstrated in the test output, managing the API key was a key challenge. Since the script was designed to run locally, it couldn't automatically access the Canvas environment's API key. This required me to manually add a placeholder for the API key and provide clear instructions for the user to insert their own key, which is a common but crucial step in local development.

Transcription Accuracy: While generally reliable, the accuracy of the transcription can be impacted by background noise or unclear speech, represented by the ratio of signal to noise, or S/N. This is an inherent limitation of speech-to-text models and something that the summarization model must work around.

Handling Large Files: For very long recordings, the audio file size and transcription time can become significant. This highlights a need for future optimizations, such as streaming the audio data to the transcription service in smaller chunks.

Overall, this project was a rewarding experience that demonstrated how powerful tools and APIs can be combined to solve a real-world problem. It taught me the importance of a phased development approach and the value of robust error handling.

Challenges we ran into

Accomplishments that we're proud of

What we learned

What's next for Daysence

Built With

• a
• and-msvcrt-platforms:-a-local-desktop-environment-(specifically
• apis.
• cloud
• databases
• databases:
• for
• google's
• in
• it
• its
• languages:-python-apis:-google's-speech-recognition-api-and-the-gemini-api-libraries/frameworks:-pyaudio
• library)
• mentions
• no
• on
• project
• relies
• services
• services:
• speech-recognition
• the
• this
• used
• wave
• were
• windows-specific