VidyaSetu 3D: Vernacular WebAR Lab

Inspiration

In India, over 70% of government and affordable private schools in Tier-2/3 cities and rural areas lack functional science laboratories. Millions of students preparing for Board exams (CBSE/ICSE/State Boards) and competitive exams (JEE/NEET) are forced to memorize flat 2D diagrams without ever handling laboratory equipment.

Existing virtual solutions like OLabs rely on dated 2012 Flash/2D animations with zero interactivity, while Western VR labs (e.g., Labster) cost thousands of dollars and demand expensive VR headsets.

We built VidyaSetu 3D to democratize science education across Bharat by turning any ₹8,000 Android smartphone or basic laptop into a fully interactive 3D and WebAR laboratory with a real-time Socratic voice mentor in Indian regional languages at ₹0 hardware cost.

What it does

VidyaSetu 3D is an immersive, accessible, zero-app-install WebAR laboratory aligned with NEP 2020 and the NCERT Class 9–12 Practical Curriculum:

• 🥽 Zero-Install WebAR (Desk Tracking): Uses the device camera directly in the browser to place burettes, prisms, circuit boards, and anatomical organs onto a student’s physical study desk.
• 🧙‍♂️ Vernacular Socratic AI Voice Mentor ("Guru AI"): An interactive voice tutor supporting Hindi (हिन्दी), Hinglish, Tamil (தமிழ்), Telugu (తెలుగు), Marathi (मराठी), Bengali (বাংলা), and English. Rather than giving away answers, Guru AI guides students through inquiry ("Look at the color! Why did the indicator turn pink at 25 mL? Think about the ions!").
• 🧪 4 Full NCERT 3D Laboratories:
  1. Chemistry (Acid-Base Titration): Burette drop physics, pH indicator color shifts (Phenolphthalein/Methyl Orange), and molarity calculation.
  2. Physics (Prism Optics): Snell's Law refraction, angle of minimum deviation, and VIBGYOR rainbow dispersion.
  3. Physics (Ohm's Law Circuit): DC power supply, variable rheostat, glowing filament ($P = VI$), and live linear $V-I$ graph plotting.
  4. Biology (3D Heart Anatomy & ECG): Cardiac cycle pulsation, clinical Lead-II ECG waveform monitor, and Web Audio synthesized Lub-Dub stethoscope sounds.
• 📊 Digital Practical Notebook: Auto-records experimental trials, computes concordant values, and generates dynamic scientific graphs.
• 👨‍🏫 Institutional & Teacher Dashboard: Real-time class accuracy telemetry, AI pedagogical misconception heatmaps, and one-click CSV gradebook export.
• 🏅 NEP 2020 Practical Skills Certification: Oral viva assessment leading to a verified, printable Certificate of Practical Competency.

How we built it

• 3D Graphics & WebAR Engine: Built using Three.js and WebGL, implementing PBR shaders, Cauchy optical dispersion equations, and device camera stream projection.
• Frontend Architecture: React 19, TypeScript, and Tailwind CSS with a responsive glassmorphic design system.
• Socratic Voice AI: Web Speech Recognition & Synthesis APIs integrated with an intelligent Socratic reasoning prompt engine for instant regional language interaction.
• Procedural Physical Audio: Web Audio API synthesizer generating real-time liquid drop splashes, glass clinks, circuit switch clicks, and ECG telemetry beeps.
• Gamification & Certification: Dynamic SVG graph plotters, Canvas Confetti, and printable DOM canvas certifications.

Challenges we ran into

1. Zero-Latency WebAR on Budget Mobile Browsers: Optimizing Three.js rendering pipelines to maintain a smooth 60 FPS on low-cost Android smartphones without dedicated GPUs.
2. Realistic Optical Ray Tracing: Implementing wavelength-dependent Cauchy refractive indices ($1.514 \le n \le 1.538$) to accurately simulate the physical fanning of the VIBGYOR rainbow spectrum through a 3D glass prism.
3. Multilingual Socratic Pedagogical Flow: Ensuring the AI mentor consistently encourages inquiry across 7 Indian languages without resorting to spoon-feeding direct answers.

Accomplishments that we're proud of

• 100% Zero-Hardware Barrier: Runs entirely in a standard web browser with no heavy APK downloads and no VR headsets required.
• Authentic NEP 2020 Alignment: Bridges the experiential learning gap for millions of underprivileged students.
• Native Vernacular Inclusivity: Full voice and UI support for Hindi, Tamil, Telugu, Marathi, Bengali, and Hinglish.

What we learned

• How WebXR and WebAR can replace millions of dollars worth of physical lab infrastructure in emerging markets.
• Techniques for synthesizing realistic physics-based audio entirely with the browser's Web Audio API.
• The power of Socratic dialogue in AI-guided STEM education.

What's next for VidyaSetu 3D: Vernacular WebAR Lab

• Expanded NCERT Syllabus: Adding Class 11 & 12 Organic Chemistry reaction mechanisms and Ray Optics telescope simulations.
• State Board Curriculum Localization: Direct integration with DIKSHA and State Educational Boards across India.
• Offline PWA Mode: Offline-first caching for schools with intermittent rural internet connectivity.

Built With

• artificial-intelligence
• canvas-confetti
• edtech
• gemini-api
• ncert
• nep-2020
• react
• tailwindcss
• three.js
• typescript
• vite
• web-audio-api
• web-speech-api
• webar
• webgl