Inspiration

I've always wanted to write software that enhances human abilities. A major project of mind over the last decade has been a note-taking app that allows you to free your mind from mundane details and readily find them when you need them.

Mind Maps allow people to use the spatial part of their brain to aid memory. Unfortunately, in 2D, links other than a simple tree structure get tangled. In 3D, that's not a problem!

What it does

It allows you to create Mind Maps in mixed reality, so your spatial memory is fully engaged.

A collaborator can be useful, to surface other possibilities. Here, the collaborator is an AI. The structure of a Mind Map guides the AI (in ways a linear chat can't) toward the area you want to expand on, and keeps it on track.

How we built it

I used the Immersive Web SDK (IWSDK) for its support for hand interactions and the browser's IndexedDB to store large amounts of information.

Challenges we ran into

  1. The IWSDK has serious gaps in its documentation regarding text and dynamic user interfaces.
  2. There's not an active community for the IWSDK, which means you can't get help with problems.
  3. The IWSDK has bugs that I had to program around. (Not surprising, as it appears to have a single maintainer.)
  4. The Quest currently has a bug: the system keyboard doesn't initially send keystrokes to the DOM. Without a workaround, every new user will find the app unusable. And they don't come back.

Accomplishments that we're proud of

  1. Navigation through a room-sized space using only hand-rays and pinch.
  2. Using the structure of a Mind Map to focus the AI.

What we learned

The IWSDK makes interactions like dragging a node very easy (except for one bug), which is a lot of work using other WebXR frameworks. It has the potential to mature into a good choice for productivity apps.

I hadn't used AI via an API. There's a bewildering varieties of ways to access AIs from an app, but once I settled on a tack it was surprisingly straightforward.

What's next for MindGlobus

The next step is allowing users to export their Mind Maps. An important step after that will be loading common Mind Map formats. (The app already contains a dynamic simulation to arrange children; it will be trivial to expand 2D Mind Maps into 3D.)

Access to the AI costs money, and it's not clear what's the best way to recoup that cost. There's no mechanism for end-users to add credentials for an AI they're already paying for.

Built With

  • chat-completion
  • d3-force-3d
  • gsap
  • iwsdk
  • ollama
  • openai-api
  • papaparse
  • troika-three-text
  • webxr
Share this project:

Updates

Submission history