Spatial MindStudio AI

3D Spatial Knowledge Constellation Synthesizer using Google Gemini API & WebGL to convert complex architecture prompts into interactive 3D constellations for visionOS & WebXR.

Comment

Inspiration

What it does

How we built it

Challenges we ran into

Accomplishments that we're proud of

What we learned## 💡 Inspiration

Traditional 2D architecture diagrams, flowcharts, and flat mind maps quickly become cluttered when mapping complex software ecosystems or dense knowledge domains. With spatial computing and Apple Vision Pro leading the future of human-computer interaction, we created Spatial MindStudio AI to transform static text into living 3D spatial constellations—allowing developers and visionaries to step inside their ideas.

🚀 What it Does

Spatial MindStudio AI takes natural language prompts (e.g., "Build a 3D graph for Snapglow Photo Editor Architecture") and synthesizes a volumetric 3D knowledge graph in real time.

  • Gemini 3D Engine: Leverages Google Gemini API's structured JSON output mode to generate 3D coordinates [x, y, z], semantic node connections, glowing colors, and spatial camera targets.
  • Zero-Collision 3D Physics: Automatically normalizes incoming node coordinates across a 3D sphere radius to prevent node overlapping in 3D space.
  • Interactive Exploration: Users can rotate, pan, zoom, and inspect individual concept nodes with high-contrast HTML projected labels and volumetric light beams.
  • Spatial UI & Accessibility: Styled with futuristic dark glassmorphic overlays, 4K High-DPI typography scaling, and dynamic spatial presets.

🛠️ How We Built It

  • Backend: Python FastAPI server handling asynchronous API requests and strict Pydantic JSON schema validation.
  • AI Intelligence: Google Gemini API (gemini-2.5-flash) utilizing response_mime_type="application/json" with customized spatial system prompts for coordinate synthesis.
  • 3D Graphics Engine: Three.js (WebGL) running custom particle space dust, emissive sphere meshes, volumetric cylinder connectors, and OrbitControls.
  • Frontend Overlay: High-DPI responsive Glassmorphism UI with on-screen WebGL Raycasting for 3D node selection.

🏆 Alignment with Judging Criteria

Criteria Implementation Highlights
Interface Design Translucent glassmorphism spatial overlay, high-DPI font scaling, and Grandma UX accessibility principles.
Technical Implementation Seamless integration between FastAPI backend, Gemini structured output JSON, and Three.js 3D WebGL pipeline.
Storytelling Bridges the gap between flat 2D conceptual diagrams and multidimensional spatial knowledge exploration.
Problem-Solution Fit Eliminates visual clutter in multi-layered microservices, system architectures, and dense academic topics.
Novelty On-the-fly dynamic spatial coordinate generation using Large Language Models without hardcoded graph templates.
Fun & Interactivity Gamified 3D space exploration, glowing particle dust, dynamic theme presets, and fluid orbit controls.

🥽 Spatial Computing & Apple Vision Pro Readiness

Designed from the ground up to support spatial viewports. The WebGL canvas runs seamlessly inside visionOS Safari, leveraging WebXR head tracking and fallback hand-gesture pinch controls for spatial browsers.

⚙️ Challenges We Ran Into

  1. Spatial Coordinate Overlapping: Ensuring Gemini-generated spatial coordinates didn't stack on top of each other in 3D space. Resolved by implementing dynamic coordinate normalization and auto-framing bounding box math (THREE.Box3).
  2. High-DPI Readability: Scaling 3D screen-projected HTML text labels on high-resolution displays so they remain perfectly readable across screen sizes.

🔮 What's Next for Spatial MindStudio AI

  • Native visionOS Hand Gestures: Full 6DOF hand-tracking support for direct manipulation of 3D concept spheres in Apple Vision Pro space.
  • Multi-User Spatial Collaboration: Real-time WebRTC synchronization for team brainstorming inside shared 3D constellations.

What's next for Spatial MindStudio AI

Built With

Share this project:

Updates

Submission history