notelm

"Notes that think with you", NoteLM is an AI-powered note-taking application that transforms how you work with information.

Comment

Inspiration

In a world saturated with information, the real challenge isn't access, but synthesis. We are constantly juggling research papers, articles, meeting notes, and video content. Traditional note-taking apps act as digital file cabinets—great for storage, but they fall short when it comes to helping us connect ideas and derive insights from our collected knowledge. The inspiration for NoteLM came from this challenge: to create a tool that doesn't just store notes but actively participates in the thinking process. We envisioned a notebook that could understand its own content, allowing users to have a conversation with their research.

What it does

NoteLM is an AI-powered note-taking application that transforms how you work with information. It allows you to:

  • Upload Anything: Add PDFs, YouTube videos, web articles, documents, and audio files directly into your notebooks.
  • Chat with Your Knowledge: Ask an AI assistant questions in natural language. The AI understands all your uploaded sources and provides context-aware answers, referencing specific documents.
  • Generate Smart Summaries: Automatically create comprehensive summaries that synthesize insights from your notes and all attached sources.
  • Write with an AI Assistant: The rich text editor is integrated with AI tools to help you connect ideas, suggest improvements, and generate content as you write.
  • Collaborate in Real-Time: All notes and sources sync instantly across devices and with other users, with live updates and presence indicators.

How we built it

NoteLM is a modern, full-stack application built with a focus on performance, user experience, and intelligent features.

  • Framework: Next.js 15 with the App Router and React 19, leveraging Server Components for speed.
  • Backend & Database: Supabase serves as the all-in-one backend, handling PostgreSQL, Authentication, - Storage, and Real-Time subscriptions for live collaboration.
  • AI & RAG: We use Langbase for the Retrieval-Augmented Generation (RAG) pipeline, which provides the - AI with context from user-uploaded documents. The language model itself is powered by Groq.
  • Editor: BlockNote provides the clean, block-based rich text editor.
  • Styling & UI: The interface is built with Tailwind CSS 4 and Framer Motion for a fluid, responsive, mobile-first design.
  • Runtime: The project uses Bun for fast package management and code execution.

Challenges we ran into

  1. Preventing AI Hallucinations: Ensuring the AI's responses were strictly based on the user's provided sources was critical. An untethered AI would be useless for research.
  2. Managing Real-Time State: Building a seamless, real-time collaborative experience without performance degradation was complex. We had to efficiently sync data across multiple clients and states.
  3. Handling Diverse Source Types: Each source type (PDF, YouTube, audio) requires a different method for content extraction and processing on the backend before it can be used by the AI.
  4. Designing an Intuitive AI UI: Creating a user interface that makes powerful AI features like source-specific chat and summary generation feel simple and intuitive was a major design challenge.

Accomplishments that we're proud of

  • A Truly Source-Aware AI: We successfully implemented a RAG pipeline that anchors the AI's responses to the user's content. It can accurately cite sources and will admit when it doesn't know the answer, which builds user trust.
  • Seamless Real-Time Collaboration: The application feels fast and alive. Notes, sources, and chat update instantly across all sessions, powered by Supabase's real-time capabilities.
  • A Unified, Multi-Source Workspace: We're proud of creating a single environment where a user can bring together disparate information formats (video, text, audio) and have them work together as a single, intelligent knowledge base.
  • Intuitive and Fluid UX: Despite the complexity under the hood, the UI is clean, responsive, and easy to navigate. Features like @mentioning a source in the chat make interacting with the AI feel natural.

What we learned

  • The Power of RAG: We learned how to build and implement a full Retrieval-Augmented Generation pipeline. This was a deep dive into vector embeddings, document chunking, and dynamic prompt engineering to give the LLM the precise context it needs.
  • Modern Full-Stack Architecture: We fully embraced the Next.js App Router paradigm, using Server-Actions and Server Components extensively. This simplified our architecture and dramatically improved performance by moving logic to the server.
  • Real-Time at Scale: We gained significant experience in designing and managing real-time systems, learning how to structure database listeners and client-side state to be both efficient and scalable.

What's next for notelm

  • Deeper AI Integration: We plan to introduce more proactive AI features, such as automatically detecting connections between different notes and sources and suggesting new insights to the user.
  • More Source Integrations: We aim to add support for more data sources, such as Notion, Google Drive, and other popular platforms, allowing users to connect their existing knowledge bases.
  • Enhanced Team Collaboration: We will build out more features for teams, including shared workspaces, permissions management, and collaborative AI tools designed for group projects.
  • Native Applications: To provide an even more seamless experience, we plan to explore building native desktop and mobile applications.

Built With

Share this project:

Updates