Moolai

Inspiration

I wanted a second brain I could talk to, and a way to show other people what my thinking looks like.

Ideas often get scattered across conversations, notes and browser tabs. When we share an idea, the questions, connections and reasoning that led to it can be difficult to explain. I wanted a place where those pieces could stay connected, making it easier to revisit my thinking and eventually share it with someone else.

That became Moolai: your second brain, in conversation.

What it does

Moolai is a personal thinking companion for brainstorming, research, project planning and learning. Talk or type naturally, and Moo captures your thoughts, organizes them into growing topic notes, and connects related ideas in a graph.

You can ask questions about your saved notes and receive answers with citations, research the web, or request help with what is on your screen. Screen capture happens only when you ask for it or accept an offer of help.

The original recorded text stays available alongside generated summaries, so you can revisit what you actually said. As an idea develops, its note can grow with it.

The current app focuses on building and exploring your own connected knowledge. The longer-term vision is to let people deliberately share parts of that thinking with others.

How I built it

The macOS app combines Electron, React and TypeScript, with Cytoscape.js for the graph and SQLite for local storage. A Swift companion handles microphone input, audio playback and screen capture. An independent Agent runtime accesses notes through a common storage interface.

A Python/FastAPI backend connects the app to its AI providers:

• NVIDIA Nemotron 3 Ultra, through Nebius Token Factory, plans tool use and produces cited answers and spoken summaries.
• NVIDIA Nemotron 3.5 Lightning, also through Token Factory, organizes thoughts into topic notes, titles and summaries in the background.
• TypeSafe Jev helps decide whether a turn is a thought to save or a request for assistance, assesses summaries, and evaluates connections between notes.
• MiniCPM-V interprets visual scenes, and Tavily supports web research.

Desktop speech recognition and speech output run locally using Whisper/whisper.cpp and Kokoro. Text and authorized screenshots are sent to cloud models for processing.

A browser demo reuses the Agent with temporary session storage and hosted speech, making the core experience accessible without installing the desktop app.

Challenges I faced

Understanding when to listen and when to respond. A spoken thought, a direct request and an interruption need different handling. Pauses and background sounds made this more complicated than a simple chat interface.

Organizing thoughts while they are still developing. Splitting notes at fixed time intervals can break an idea apart. I worked on growing topic notes that preserve the original text while allowing summaries to evolve.

Getting complete, usable model output. Some model calls used their output allowance on reasoning before returning the structured result the app needed. This required separate allowances for different tasks, validation of responses, and recovery paths that retain the source text.

What I learned

Building Moolai taught me that useful AI memory depends on preserving context, authorship and connections. Those details help someone understand how an idea developed and check the evidence behind an answer.

I also learned how much a conversational interface depends on timing: when to save, when to organize, when to speak and when to stop. Giving reasoning, organization and decision-making distinct roles made those responsibilities easier to inspect and improve.

What’s next

The next steps are improving everyday reliability, adding synchronization, and enabling controlled sharing of selected collections. I want someone to be able to explore the ideas and questions behind another person’s work, understand their perspective, and contribute their own thinking. Porting the project a an obsidian plugin.

Built With

• cloudflare-turnstile
• cytoscape.js
• docker
• electron
• fastapi
• glm
• kokoro
• minicpm-v
• mlx
• nebius-token-factory
• nvidia-nemotron
• openai
• postgresql
• python
• qwen3-asr
• react
• sqlite
• swift
• tavily
• typesafe-jev
• typescript
• upstash
• vercel
• vite
• whisper.cpp