Mental Galaxy

Inspiration

Your head is the loudest exactly when you have the least amount of time to deal with it: finals week, a hackathon, a Sunday night with a flight you haven't packed for. It all comes out as one run-on thought: "Stressed about tests, project due Friday, interview Saturday, flight Sunday -- and I'm not packed."

Every note app makes you stop and organize that yourself, which is the last thing you want to do mid-spiral. We wanted the opposite: just talk, and watch it sort itself out into something you can actually act on, not just another empty doc staring back at you.

What it does

Mental Galaxy is a voice-first thought-mapping app. You hold the mic and talk through your day. Your words stream in live, each thought gets classified into a task, emotion, or idea, and drops onto an animated, force-directed galaxy of bubbles with connections drawn between related thoughts.

From there, it goes further than a map:

Guidance: Tap any bubble and ask, "What should I do?" An agent pulls from your past sessions and suggests one concrete, grounded next step. It only runs when asked, so it never nags.
Action: Hit "Execute" on a task bubble, and specialized agents take over -- adding the event to your calendar or drafting an email. The bubble turns green when it's done.

How we built it

We split the work three ways -- one person on the voice pipeline, one on the canvas, one on agents/infra -- and converged at three milestones (the map, the guidance, the action) so we always had a demoable product instead of betting everything on the final hour.

Frontend & Voice: React + D3 for the force-directed canvas. Deepgram Nova-2 streams speech-to-text, piping raw PCM audio over a WebSocket using the AudioWorklet API for low latency.
Reasoning Orchestration: FastAPI orchestrates the pipeline. Claude (Anthropic) classifies each thought and finds connections, returning the structured JSON that drives the galaxy.
Memory: Redis + RedisVL for vector search over past sessions. This ensures the agent actually knows what you said last week, making suggestions feel grounded instead of generic.
Agents: Fetch.ai uAgents (registered on Agentverse) route each bubble to the right specialized agent. Google Calendar and Gmail APIs act as the agents' actual hands.
Observability: Arize logs every Claude classification decision to watch the model's behavior in real-time, while Sentry monitors the stack for errors.

Challenges we ran into

Getting an LLM to emit reliable, structured JSON under the pressure of messy, real-speech input took massive prompt iteration. For example, getting the model to correctly file "violin" and "cross country" under an existing "hobbies" bubble, rather than inventing three new bubbles, took significant work. We fixed this by reframing existing sections as "containers" rather than "peers," which completely changed how the model reasoned about placement.

Other hurdles included handling the backpressure of streaming audio over websockets, maintaining D3 force-simulation performance as the galaxy got crowded, and wiring up OAuth under serious time pressure.

What we learned

Reliable LLM outputs require constant refinement, and observability in a multi-stage AI pipeline isn't a nice-to-have. When something can break at Deepgram, Claude, Redis, or an agent, Arize and Sentry were the only reasons we could find the break fast.

But the biggest lesson wasn't technical. The best demo moment wasn't any specific feature we planned; it was the silence right after you stop talking, watching the galaxy automatically build and organize itself.

What's next

Richer connections: Adding causal and temporal links between bubbles, not just "related."
More execution agents: Expanding beyond calendar and email integrations.
Multiplayer: Shared galaxies so a team can map out a project together out loud.
Mobile App: So you can talk through your commute and watch your thoughts organize by the time you get home.