VoxAssist (by Void Stars)

Personal knowledge-base RAG for ask + plan — with optional voice.

Comment

Inspiration

Most "chat with your notes" tools search a pile of documents. We wanted something that actually knows the person behind the notes: their stack, preferences, and past decisions, since that context is already sitting in the markdown we (and our AI agents) write every day in tools like Obsidian.

The idea sharpened around a real hackathon problem: teammates constantly re-explain themselves to each other. What if you could just ask, "what's Rayan's stack, and how does he like to work?" and get a grounded, cited answer pulled from his own notes, scoped to what he's chosen to share? That became VoxAssist's core: not just "ask about yourself," but "ask about a friend," safely.

As Void Stars at cuHacking 2026, we also used the build to stretch outside our usual lanes, embedded systems and ML/retrieval, into production RAG, multi-user auth, and a live cloud deployment.

What it does

  • Ask: Ask a question about yourself (or a friend, with permission) and get an answer grounded only in real notes, with file and heading citations on every claim.
  • Plan: Describe a project idea and get a personalized brief, grounded in your real stack and past work, saved as a downloadable markdown file.
  • Ask a friend: The core differentiator: query only a teammate's notes marked shareable, enforced server-side, twice.
  • Upload : Drop in a new note on the spot; it's chunked, embedded, and searchable within seconds, with a per-file share toggle.
  • Voice (optional) – Push-to-talk speech-to-text and spoken read-aloud of answers.

It deliberately does not act as an agent that executes projects end to end. It answers and plans, then stops.

How we built it

Next.js 15 and TypeScript for the app and API, MongoDB Atlas Vector Search for storage, Voyage AI for embeddings, Gemini 2.5 Flash for fast ask synthesis, Claude Opus for the heavier plan generation, and DigitalOcean App Platform for deployment.

Notes are chunked on heading structure, not a fixed token window, so a retrieved chunk is always a coherent section. Each chunk is embedded and indexed for both vector and lexical search; at query time we fuse the two ranked lists with Reciprocal Rank Fusion, $\text{score}(d) = \sum_r \frac{1}{k + \text{rank}_r(d)}$, which catches exact names that pure semantic search tends to blur past. Retrieved chunks, with source metadata attached, are handed to the LLM with a strict grounding prompt: cite every claim, and say plainly when the notes don't cover something.

One of us (Momen) owned the frontend, ingestion, chunking, seed and CLI tooling, the other (Rayan) owned embeddings, database configuration, hybrid retrieval, and the generation prompts. We used Claude + V0 to build the Next.js frontend and wire it to the API routes, freeing both of us to focus on the retrieval and data layer. Locking a shared TypeScript contract for the chunk schema and AI module interfaces early let both halves be built and tested independently against mocks before wiring the real pieces together.

Challenges we ran into

  • $rankFusion wasn't actually available. Our Atlas cluster runs MongoDB 8.0.27; although introduced in 8.0, native $rankFusion needs 8.1+ for full compatibility. We implemented Reciprocal Rank Fusion ourselves in the application layer instead, keeping the hybrid retrieval story intact without depending on a server version we didn't control.
  • Two production bugs surfaced only after deploy. DigitalOcean was skipping devDependencies during build because NODE_ENV=production was set at build scope, silently breaking a required Tailwind package. Separately, our MongoDB client cached a rejected connection promise on first failure, poisoning every request until restart, triggered by an Atlas network rule blocking DigitalOcean's egress IPs. Both needed real production debugging, not just local fixes.
  • A branch merge nearly cost us a scaffold. Two of us scaffolded independently under time pressure and drifted into incompatible folder layouts. We reconciled by adopting one canonical structure and carefully porting the other's unique pieces back in.

Accomplishments that we're proud of

  • Shipping a real, live, multi-user deployment, not a local-only demo, in about a day and a half.
  • Getting the privacy model right: friend-scope queries are enforced server-side twice, and a note's owner can never be spoofed through frontmatter.
  • Watching plan mode work as intended: a project idea produced a genuinely personalized brief that correctly cited one persona's real past projects and steered around a stack they don't use.

What we learned

  • Cloud database features aren't guaranteed by "latest," they're gated by your cluster's version, worth checking before you design around a feature.
  • Locking a shared interface before writing implementation code was our highest-leverage decision, letting two very different specialties build in parallel and sync only at the seams.
  • Production-only bugs need a different debugging instinct than local development; tight smoke tests and fast redeploy loops matter more than they do day to day.

What's next for VoxAssist

  • Finer-grained sharing, per-section or per-tag instead of per-file.
  • Real incremental sync, so it becomes a living second brain instead of a one-time snapshot.
  • Full duplex voice, asking follow-ups without touching the keyboard.
  • Graph-aware retrieval using the [[wikilinks]] we already parse, pulling in connected notes, not just semantically similar ones.
  • A QNX voice-terminal head, a small embedded device that acts as a physical voice front end to the same backend. (Nice-to-have hardware layer + demonstration of real-time system and cloud integration)

Built With

Share this project:

Updates