Neural illumination

Neural Illumination is an 3D learning platform that combines spatial computing, AI and Python. It showcases the power of Gemini 3 in creating next-generation educational experiences.

Comment

🌌 Neural Illumination (Gemini 3.1 pro preview model)

Reimagining How Humans Learn to Code

🚀 Inspiration — What If Learning Didn’t Feel Like the Past?

Programming education has barely evolved.

We still learn code through:

  • Static textbooks
  • Flat video tutorials
  • Silent IDEs that don’t see, hear, or understand us

But we asked a different question:

What if learning to code felt like stepping into the future?

Inspired by futuristic interfaces like Iron Man’s JARVIS and Star Trek’s Holodeck, we imagined a learning environment where:

  • An AI companion doesn’t just answer questions — it understands you
  • Code isn’t just text — it becomes living, interactive 3D objects
  • Your focus, posture, and emotions actively shape your learning experience

When Google announced Gemini 3, capable of unified text, vision, and code reasoning, we realized this vision was finally achievable.

“The best way to predict the future is to invent it.” — Alan Kay

That’s how Neural Illumination was born.

💡 What Is Neural Illumination?

Neural Illumination is an immersive 3D, AI-powered learning platform that redefines how students learn Python programming.

Instead of reading about code, students experience it — spatially, visually, and interactively.

This isn’t just a new UI. It’s a new learning paradigm.

🤖 Meet Luna — Your AI Learning Companion

Luna is not a chatbot.

She is an autonomous AI avatar that exists inside the 3D world.

She:

  • Walks, wanders, and follows you naturally
  • Listens to your voice, not just your keyboard
  • Tracks your learning progress and gives real-time feedback
  • Speaks back using natural neural voice synthesis

Powered by Gemini 3 and ElevenLabs, Luna feels less like software — and more like a mentor who’s present with you.

🔮 Voice-Controlled Holographic Programming

Say:

“Hey Luna, show me a sphere.”

And instantly:

  1. Gemini 3 generates Python code
  2. The code runs entirely in your browser via WebAssembly (Pyodide)
  3. A glowing 3D hologram materializes in front of you

Abstract code becomes tangible reality.

This single interaction bridges one of the biggest gaps in education:

Understanding how code becomes behavior.

👁️ AI Mentor Mode — Learning That Sees You

With Mentor Mode, Luna uses your webcam to become an AI learning coach:

  • Detects distraction (e.g., phone usage)
  • Notices poor posture or slouching
  • Encourages you when you’re focused and engaged

Instead of passive surveillance, Mentor Mode provides gentle, supportive guidance, powered by Gemini Vision analyzing frames in real time.

Learning becomes responsive, not rigid.

🐍 Adaptive, Gamified Python Curriculum

Neural Illumination isn’t just visually impressive — it’s educationally solid.

Students progress through:

  • 30+ structured challenges, from Hello World to recursion
  • AI-evaluated submissions with personalized feedback
  • XP, streaks, levels, and adaptive difficulty

This is not a demo or proof-of-concept. Students can genuinely learn Python end-to-end inside the platform.

😊 Mood-Responsive Learning Environments

Learning isn’t purely logical — it’s emotional.

Using facial expression analysis, Neural Illumination dynamically adapts the environment to your mood:

  • Cyber
  • Nature
  • Desert
  • Rainforest
  • Iceland

Your emotional state shapes the world around you, creating a calmer, more supportive learning experience.

🛠️ How We Built It

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                    Neural Illumination                       │
├─────────────────────────────────────────────────────────────┤
│ Frontend: React 19 + TypeScript + React Three Fiber          │
│ ├── 3D Rendering: Three.js + WebGL                           │
│ ├── Voice Input: Web Speech API                              │
│ └── Python Runtime: Pyodide (WebAssembly)                    │
├─────────────────────────────────────────────────────────────┤
│ AI Layer: Google Gemini 3 Pro Preview                        │
│ ├── Chat & Reasoning                                        │
│ ├── Vision (Mentor Mode & Mood Detection)                   │
│ ├── Code Generation                                         │
│ └── Submission Evaluation                                   │
├─────────────────────────────────────────────────────────────┤
│ Voice Output: ElevenLabs Neural TTS                           │
└─────────────────────────────────────────────────────────────┘

No backend servers. No installations. Everything runs directly in the browser.

🗓️ Development Journey

Week 1 — Foundations

  • Built the 3D spatial world using React Three Fiber
  • Designed the CyberRoom environment
  • Integrated Gemini for conversational AI

Week 2 — Luna Comes Alive

  • Created the 3D avatar
  • Implemented autonomous movement with a leash system
  • Added neural voice synthesis

Week 3 — Multimodal Magic

  • Integrated Gemini Vision
  • Built the voice-to-code hologram pipeline
  • Embedded Pyodide for Python execution

Week 4 — Polish & Learning Design

  • Added five dynamic environments
  • Implemented mood detection
  • Built the full Python curriculum
  • Added XP, streaks, and adaptive difficulty

⚙️ Key Technical Decisions

Choice Why It Mattered
React Three Fiber Declarative, scalable 3D with React
Pyodide True Python execution without servers
Gemini 3 Unified text, vision, and code reasoning
ElevenLabs Human-like voice for emotional presence

⚠️ Challenges & Solutions

Rate-Limiting (Vision APIs) → Exponential backoff + auto-pause analysis

Voice Recognition Errors → Phonetic correction and synonym mapping

Pyodide Cold Start → Lazy loading with progress feedback

Low-End Device Performance → Automatic quality tiers (Low / Medium / High)

Jittery Avatar Movement → Velocity dampening + snapping logic

🏆 What We’re Proud Of

🥇 A First-of-Its-Kind Integration

3D spatial computing + multimodal AI + in-browser Python + neural voice — all in one cohesive platform.

✨ The “Wow” Moment Is Real

“Hey Luna, show me a red torus.” → Code generated → Python executed → 3D object appears

It feels magical — and it works every time.

🧠 Luna Feels Alive

She walks, watches, reacts, and teaches. She feels like a companion, not a tool.

📚 Real Educational Impact

This isn’t a gimmick. It’s a complete, structured learning system.

📖 Lessons Learned

Technical

  • Multimodal AI simplifies entire system design
  • WebAssembly unlocks powerful browser experiences
  • Good 3D UX takes iteration and restraint

Product

  • Build the wow moment first
  • Constraints create better architecture
  • AI companions need personality, not just intelligence

🔮 What’s Next

Short-Term

  • Multiplayer collaborative learning
  • 3D data structure visualizers
  • Voice-to-code IDE
  • Mobile AR via WebXR

Long-Term

A spatial AI education platform for:

  • Mathematics
  • Chemistry
  • Physics
  • Beyond

🤝 Open Source Vision

We plan to open-source the core engine, enabling educators and developers to create their own AI companions, curricula, and immersive learning worlds.

🌟 Final Thought

Neural Illumination isn’t just about learning Python.

It’s about reimagining how humans learn — when AI doesn’t replace teachers, but stands beside students.

Built With

  • elevenlabs
  • eslint
  • gemini3pro
  • pyodide
  • react
  • react-three/drei
  • react-three/postprocessing
  • tailwind
  • three.js
  • typescript
  • vite
  • webspeech
Share this project:

Updates