AetherLens

• 

  See

Inspiration

South Africa’s education system faces a profound crisis—highlighted by staggering national literacy statistics where the vast majority of young learners struggle with reading comprehension and sustained focus. At the same time, children and teens spend hours passively glued to social media feeds, short-form video loops, and digital distractions that erode attention spans.

We asked ourselves: Instead of fighting against digital media, why not weaponize it for good? Rather than letting screens pull youth away from learning, AetherLens transforms the mediums they already love—movies, stories, and cinematic media—into an active, high-engagement educational powerhouse designed to rebuild cognitive focus and academic literacy. What it does

AetherLens is an immersive spatial co-watching tutor built for Meta Quest. Users step into a virtual theater environment and attach any video or film alongside their chosen subjects (ranging from mathematics and science to history and literature). Powered by real-time multimodal AI, AetherLens watches the film alongside you, dynamically abstracting the narrative and visual cues on screen to deliver live voice lessons, contextual insights, and interactive breakdowns tailored precisely to your custom curriculum. How we built it

Engine & XR Tools: Built using Unity configured for Meta Quest, leveraging the Meta XR Core SDK and Interaction SDK for seamless, controller-free hands-first navigation.

Spatial UI: Designed an intuitive floating virtual screen featuring our signature + attachment mechanism, allowing users to interact with the environment entirely through natural hand pinches and gestures.

AI & Multimodal Stream Pipeline: Integrated real-time video frame capture and audio routing connected to a multimodal AI backend, utilizing custom system prompts to ensure the tutor intelligently tracks content across multiple academic disciplines without breaking immersion.

Challenges we ran into

Latency & Real-Time Sync: Balancing the performance of streaming visual frame buffers from a virtual screen while maintaining low-latency, natural voice conversation required careful optimization of the audio-video pipeline.

Contextual Relevance: Fine-tuning the AI prompt engineering to ensure it successfully filters out irrelevant dialogue while dynamically locking onto moments that match abstract, cross-disciplinary subjects (like mapping cinematic scenes to real-world educational concepts).

Accomplishments that we're proud of

Cross-Disciplinary Versatility: Successfully moving beyond rigid coding tools to create a universal engine that can adapt to any subject in the multiverse of knowledge.

Immersive Hands-First Design: Delivering a fluid spatial interface that adheres strictly to Meta's hands-first interaction guidelines, making the co-watching experience feel natural and futuristic.

Reclaiming Digital Attention: Designing a tool that directly targets the modern attention crisis by converting passive screen habits into active, deep-focus learning sessions.

What we learned

The Power of Spatial Context: We learned how profoundly spatial computing changes learning—combining visual narrative memory with real-time conversational AI dramatically improves information retention and engagement compared to traditional flat screens.

Edge-Case Stream Management: Gained deep technical insight into managing lightweight stream pipelines inside Unity for standalone VR headsets.

What's next for AetherLens

Expanded Syllabus Importing: Adding direct support for uploading PDF syllabi, textbooks, or custom curricula so the AI can automatically tailor movie scenes to a student's exact coursework.

Multi-User Co-Watching Rooms: Expanding into shared spatial rooms where friends or study groups can watch films together alongside a synchronized AI tutor.

Public Beta Release: Polishing the standalone Quest build for a wider release on App Lab / Horizon Store.

Built With

• android
• c#
• coresdk
• gemini
• geminiapi
• metaxr
• multimodalai
• spatialcomputing
• unity
• websockets