zettelkasten

An immersive 3D spatial knowledge graph w/ AI-driven insights

Comment

Inspiration

We wanted a second brain that feels spatial and fast to use: notes and documents aren’t just a list: they’re a graph you can move through, and you should be able to ask questions out loud and get answers grounded in your knowledge and context. Existing 2D solutions feel flat; we've found that 3D spatial navigation is a necessity for knowledge graphs.

What it does

Zettelkasten is a web app for uploading, linking, and exploring notes and documents as a semantic knowledge graph, with optional 3D / spatial views (including WebSpatial-friendly flows for devices like PICO).

We have AI voice-driven insights and actions to add nodes and interact with the graph since integrated agents automatically have access to all connections between nodes and data points. This allows them to derive connections that would otherwise remain un-noticed.

How we built it

  • Next.js + TypeScript + React for the app shell, API routes, and auth (NextAuth).

  • PostgreSQL + Prisma for users, nodes, and structured graph data.

  • Microsoft Azure Blob for files; Azure AI Search + embeddings for retrieval, kNN machine learning clustering

  • Microsoft Azure AI Foundry (OpenAI-compatible) for grounded answers from your corpus.

  • ElevenLabs for speech-to-text; Azure Speech / ElevenLabs for TTS; client playback tuned for real devices.

  • Three.js + React Three Fiber and WebSpatial SDKs for spatial / XR UI where it matters.

  • Tanit XR models for cultural significance and to preserve global heritage sites

Challenges we ran into

  • 2D graph → meaningful 3D: embeddings are high-D; layout in 3D that humans can navigate (not random fog) needs new constraints: semantic axes, time, topic hierarchy, or learned camera paths.
  • Scale in space: thousands of nodes as instanced meshes / LOD / occlusion without killing WebGL on a headset browser.
  • Multimodal embeddings for pictures, models, texts, pdfs: basically an all-in-one embedding system that traverses across mediums

Accomplishments that we're proud of

The AI Agent allows for us to create reports, add new nodes with new data points, and even automatically cluster points using advanced machine learning algorithms. Seeing our idea come to life and solve our own problems when we used 2D knowledge graphs was one of the most awarding experiences I've had! Not only was this an awesome technical project to work on, but it gets me excited to brainstorm, learn, and step into unfamiliar domains!

What we learned

We learned more about the design and UI/UX side of things this part of the hackathon. We wanted the design of the experience to really shine and not only be functionable, but also pleasant to use. We believe that by implementing features like click-to-expand, and immersive graphs that really take advantage of spatial environments (aka not just being 2D panels!) we created a truly enriching experience.

What's next for zettelkasten

We are looking to take this to the next stage in the terms of the startup world! As students who have used 2D knowledge graphs in the past, we know the specific pain points, and believe with the advent of up and coming XR technology, we believe that not only will software like this be more common, but also widely commoditized! This project could very well be the first immersive, 3D spatial environment for knowledge graphs with AI-driven integrations, friendly to both students, learners, and researchers!

Built With

Share this project:

Updates