EyeBridgeVr

• 

• 

• 

Inspiration

Millions of people regain sight after years of blindness — but learning to see again isn’t easy. The eyes can receive light, but the brain must relearn how to interpret depth, motion, color, and shape. This process can be slow, confusing, and emotionally draining.

EyeBridgeVR was inspired by the brain’s ability to adapt through neuroplasticity — the natural rewiring that occurs with new sensory input. We wanted to build a safe, supportive environment where newly sighted individuals can train their brain to see again using virtual reality.

By simulating real-world experiences and tracking eye activity, EyeBridgeVR bridges the gap between having sight and understanding sight.

What it does

EyeBridgeVR is a virtual neuroplasticity simulator that helps users rebuild visual understanding through multisensory VR training.

Multisensory Anchoring: Sight, sound, and touch are synchronized. When users identify objects, spatial audio and haptic feedback reinforce memory and recognition.

Progressive Visual Curriculum: Training starts with light and contrast, then progresses to shapes, objects, and complex spaces like staircases or busy streets.

AI Vision Coach: A Convai-powered assistant narrates the scene, describes object properties ("round," "tall," "moving"), and reacts in real time to the user’s gaze.

Therapist Dashboard: Tracks gaze patterns, recognition times, and depth-perception improvements, giving therapists valuable data for rehabilitation.

Safe Real-World Simulation: Users can practice navigating environments and identifying obstacles without physical risk.

How we built it

We developed EyeBridgeVR using Unity (URP) with the XR Interaction Toolkit for VR movement, teleportation, and interaction.

Integrated Convai SDK for the AI Vision Coach, enabling natural narration and dynamic conversations.

Simulated eye-tracking and gaze logging using Unity’s Input System to record user focus and timing.

Designed training stages with lighting, shaders, and dynamic difficulty, progressing from basic contrast detection to full spatial awareness.

Created data visualization dashboards for therapists to review gaze and depth-perception metrics.

Implemented accessibility settings such as contrast modes and adjustable proximity for patients with partial vision.

Challenges we ran into

Hardware Limitations: VR headset tracking and base-station calibration issues required repeated troubleshooting and driver updates.

Simulated Eye Tracking: Without real eye-tracking hardware, we built a custom script to approximate gaze data and smooth inputs.

AI Integration: Ensuring Convai responses matched what the user was looking at involved building new gaze-based event triggers.

Optimization: Keeping the experience smooth while maintaining realism and accessibility was a continuous balancing act.

Accomplishments that we're proud of

Built a working VR prototype that simulates visual rehabilitation for newly sighted patients.

Integrated an AI narrator that reacts to gaze in real time.

Designed training environments that adapt to user progress and difficulty.

Created a therapist dashboard that visualizes patient improvement over time.

What we learned

How neuroscience and VR can merge to create life-changing rehabilitation tools.

The importance of multisensory reinforcement for faster and more confident learning.

How to integrate Convai AI, Unity XR Toolkit, and data tracking to make responsive VR experiences.

That empathy-driven design can have just as much impact as technical skill in solving real-world challenges.

What's next for EyeBridgeVR

Integrate Real Eye-Tracking Hardware: Use Tobii or Vive Pro Eye to improve gaze precision and feedback accuracy.

Partner With Research Labs: Work with neuroscientists and therapists to validate the training curriculum clinically.

Adaptive AI Coaching: Train the AI to adjust the difficulty and style of feedback based on patient progress.

Expand Environments: Add scenarios like homes, streets, and classrooms for practical real-world training.

Mobile & Cloud Version: Develop a lighter version for Meta Quest or smartphones to make it accessible globally.

Built With

• convai
• steamvr