NoteNest
Inspiration
The inspiration for NoteNest originated from observing the high-intensity research environment of a university professor. We witnessed a workflow where students participated in focused group discussions, summarizing complex research papers to extract core insights. The professor had even developed a custom chatbot to assist students in understanding these documents.
We realized that in any fast-paced setting—whether a 60-minute lecture or a 5-minute stand-up meeting—the ability to retain information drops significantly once the speaker stops talking. We wanted to bridge the gap between passive listening and active learning by creating an app that turns any spoken session into an interactive, persistent knowledge base.
What it does
NoteNest is an AI-powered study and meeting companion that transforms live recordings into intelligent "Nests." It captures audio, generates a refined transcript that restores context and fixes semantic errors, and allows users to chat with their notes in real-time. Key features include:
- Intelligent Chat: Ask your lecture specific questions, like "What was the car analogy used for?"
- Social Sharing (Nest ID): Every user has a unique ID to share their processed lecture bots with classmates or teammates.
- Automatic Summarization: Instantly generates quizzes and key takeaways from any recording.
How we built it
We developed NoteNest using a modern, scalable tech stack:
| Layer | Technology |
|---|---|
| Frontend | Built with Flutter for a cross-platform, responsive experience. |
| Backend | A Python server running FastAPI to handle the AI logic and database interactions. |
| AI Engine | Integrated Gemini 3.0 Flash for state-of-the-art transcription refinement and context restoration. |
| Real-time Logic | Implemented Socket.io to manage bi-directional communication between the user and the AI. |
| Retrieval System (RAG) | Built a pipeline that stores processed text in a Vector Database, enabling the AI to retrieve specific facts from a recording to answer user queries without hallucinating. |
Challenges we ran into
Challenge 1: API Quota Limits
One significant hurdle was the Gemini API quota limits, which we encountered during heavy testing. To solve this, we implemented a decentralized system allowing users to connect their own Gemini API keys, ensuring the app stays functional regardless of central quota bottlenecks.
Challenge 2: High Latency
We also struggled with high latency; initial API calls for transcription and processing took over a minute. We solved this by implementing parallel task execution in the backend. This allows the app to process data in a background queue so the user can continue using the Flutter frontend without the app feeling "frozen."
Accomplishments that we're proud of
We are particularly proud of implementing a fully functional Socket-based sessional memory. This ensures that the AI remembers the conversation context as long as the user is connected, but automatically clears the memory upon disconnection to keep the system lightweight and private.
Successfully moving from a Figma concept to a working RAG-based application within the hackathon timeline was also a major milestone for our team.
What we learned
This project was a masterclass in asynchronous programming. We learned how to manage long-running AI tasks without degrading the user experience. We also gained significant experience in:
- Socket-based state management
- Practical application of RAG pipelines
- Technical adaptability—knowing when to pivot our design based on the reality of API constraints and execution speed
What's next for NoteNest
The next phase for NoteNest involves expanding the "Nest" ecosystem. We plan to introduce:
| Feature | Description |
|---|---|
| Multi-Modal Nests | Allowing users to upload images of whiteboards or PDFs of research papers to merge with their audio notes. |
| Collaborative Nests | Real-time group study sessions where multiple users can chat with the same lecture bot simultaneously. |
| Edge Processing | Moving some of the transcription logic on-device to further reduce latency and improve privacy. |
Log in or sign up for Devpost to join the conversation.