Quran Orion AR — Sacred Spatial Prompter

Spatial Quran prompter for Meta Orion & Horizon OS. Hands-free phonetic speech tracking, sub-25mm WebXR micro-pinch, and zero-touch gaze-dwell for prayers and memorization.

Comment

Inspiration

For over 1,400 years, memorizing and reciting the Holy Quran has been preserved by millions of memorizers (Huffadh) and congregational prayer leaders (Imams) worldwide. However, leading congregational prayers (Salah) strictly forbids holding a phone or physical book—excessive extraneous physical motion (Harakah Kathirah) and gaze deviation invalidate the prayer under traditional legal consensus across major Islamic schools of jurisprudence. At the same time, bulky immersive VR headsets are completely unsuitable because they isolate the worshipper from the physical congregation, block the view of the prayer mat and floor (Mawdi' al-Sujud), and disrupt spiritual decorum.

With Meta Orion and optical see-through smart glasses running on Meta Horizon OS, spatial computing finally provides the lightweight form factor (~98g) and true optical transparency required to solve this dilemma while honoring sacred tradition.

What it does

Quran Orion AR transforms optical smart glasses into a discrete, hands-free spiritual prompter:

  1. Optical Waveguide Additive Light Display: True black (#000000) renders as 100% transparent glass on optical waveguides, keeping the physical prayer hall and congregation fully visible while verses float as radiant gold and cyan holograms.
  2. Real-time Phonetic Syllable Tracking & Two-Tone Feedback: A millisecond-precision speech verification engine aligns spoken syllables with classical 7th-century script. Words articulated into the microphone render dynamically in a live transcript box—correctly verified words illuminate in radiant Electric Blue (#38BDF8), while misrecognitions or ambient acoustic noise appear in Muted Red (#F87171) with a real-time RMS microphone VU level meter.
  3. 1.5-Second Adaptive Hesitation Engine: Listens to the reciter's natural breathing cadence; if a pause exceeds 1.5 seconds, the engine reveals only the next single-word hint to gently restore reciter flow without visual clutter.
  4. Takbeerat Al-Intiqāl & Optical Khushū' Dimming: During congregational prayer (Imam Salah), the engine acoustically detects transition phrases («اللَّهُ أَكْبَرُ», «سَمِعَ اللَّهُ لِمَنْ حَمِدَهُ», «رَبَّنَا وَلَكَ الْحَمْدُ»). Upon bowing (Rukū') and prostrating (Sujūd), it automatically dims the waveguide display to 35% opacity with subtle spatial blur, eliminating downward optical glare onto the prayer mat. When the Imam rises to the second Rak'ah and articulates «الْحَمْدُ لِلَّهِ رَبِّ الْعَالَمِينَ», the display returns to 100% brightness and smoothly resumes tracking.
  5. Tokenized Sliding-Window Fuzzy Search Engine: An ultra-fast in-memory search engine scanning all 6,236 verses in ~45ms with zero network calls. It tolerates acoustic speech mismatches, dropped/added Hamzas (أ/إ/آ ↔ ا), open-Taa (رحمت ↔ رحمة), dagger Alifs (رحمن ↔ رحمان), and attached particles (و/ف/ل/ب/ال). Matched search tokens illuminate in radiant Electric Cyan (#38BDF8) while preserving 100% of authentic Uthmani Tashkeel (diacritics).
  6. Multi-Surah Result Selection & The "Ar-Rahman Rule": If a verse occurs across multiple Surahs, each Surah is displayed as a distinct interactive card. Repeating refrains within the same chapter (e.g. «فَبِأَيِّ آلَاءِ رَبِّكُمَا تُكَذِّبَانِ» repeating 31 times in Surah Ar-Rahman) are consolidated into a single card with an amber badge (🔄 Repeats 31x) and dual action buttons: 🎯 Direct Jump or 🕌 Salah Mode (opens Fatihah first).
  7. Bilateral Sub-25mm WebXR Micro-Pinch: Uses raw WebXR joint tracking across 25 hand bones to detect subtle micro-pinches (under 25mm between thumb and index finger) executed discretely with hands resting naturally in prayer posture (standing with hands folded).
  8. Hands-Free Prostration Gaze-Dwell: Prayer leaders naturally look downward toward the floor prostration spot (Mawdi' al-Sujud); a 1.2-second downward gaze alignment triggers hands-free word hints with zero physical movement.
  9. 4 Specialized Spatial Context Modes:
    • 🎯 Judge & Demo Mode (Active by Default): Built specifically for instant evaluation with authentic audio recitation recordings (Sheikh Mahmoud Khalil Al-Husary) and English phonetics pre-loaded.
    • 📖 Hifz Study Mode: Word masking, audio repetition, hands-free voice commands, and transliteration overlays for students.
    • 🕌 Imam Salah Mode: 100% silent, zero-touch operation with Sirr (silent) Basmalah compliance and Takbeer optical Khushū' dimming.
    • 👥 Imam Follower Mode: Synchronized congregation HUD with automatic pacing lock for worshippers standing behind the prayer leader.

How we built it

  • Spatial & WebXR Layer: Engineered using Chromium WebXR APIs measuring 3D Euclidean distances between thumb and index finger joints for discrete micro-pinch navigation, complemented by an optical additive shader (#000000 = true transparent glass).
  • Phonetic & Normalization Engine: Built a custom linguistic normalizer handling classical 7th-century Quranic orthography (archaic letter forms, inverted vowels, dagger alifs, open-Taa رحمت ↔ رحمة, and disjointed prefixes) paired with dynamic acoustic phonetic tolerance and two-tone speech feedback.
  • Acoustic Liturgical State Machine (Takbeerat Al-Intiqāl): Engineered real-time acoustic detection of prayer transition phrases («اللَّهُ أَكْبَرُ», «سَمِعَ اللَّهُ لِمَنْ حَمِدَهُ», «رَبَّنَا وَلَكَ الْحَمْدُ») governing an Optical Khushū' Dimming engine (opacity: 0.35, blur 0.6px) that eliminates waveguide reflection during bowing (Rukū') and prostration (Sujūd), with automatic Rak'ah 2 Fatihah resumption.
  • Tokenized Sliding-Window Search Engine: Built an ultra-fast in-memory fuzzy search engine executing across all 114 Surahs (6,236 verses) in ~45ms with zero network dependency. Features a dual-buffer character projection index that highlights matched phrases in radiant Electric Cyan while preserving 100% of authentic Uthmani Tashkeel (diacritics).
  • Dual-Layer Audio HAL & Far-Field DSP: Implemented parallel Web Audio capture with non-blocking DSP (GainNode + DynamicsCompressorNode) forcing Android/Pixel hardware HAL out of telephony VoIP mode into Media recording mode for crystal-clear distant audio capture.
  • Packaging & Horizon OS Distribution: Deployed as an edge-accelerated WebXR PWA on Cloudflare Pages (https://quran-orion.pages.dev/?view=ar) and packaged into a standalone native Meta Horizon OS application (com.jnolabs.quranorion) with minSdkVersion 29 and targetSdkVersion 32, cryptographically signed via APK Signature Scheme v3 and uploaded to the Meta Quest Alpha Release Channel.
  • Offline-First Dataset: Embedded an in-memory 114-chapter local search and storage engine backed by Service Worker caching for zero-network stone environments.

🛠️ Key Meta Horizon Technologies & Architecture Highlights

  • Immersive Web SDK (IWSDK / WebXR): Architected natively on Chromium WebXR Device APIs and Meta's Immersive Web SDK. Delivered as an instant, zero-install, edge-accelerated Progressive Web App hosted on Cloudflare Pages with full 6DoF spatial tracking, sub-25mm joint detection, and optical additive waveguide rendering (#000000 = 100% transparent glass).
  • Sub-25mm Neural Hand Interactions: Custom Bilateral Sub-25mm Euclidean Micro-Pinch Engine tracking 25 hand bones in 3D space, engineered specifically for prayer posture (Qiyām) without raising arms or using controllers.
  • Standalone Meta Horizon OS Android Architecture: Packaged native Android application (com.jnolabs.quranorion) targeting Android 12L / Horizon OS, signed with release keys using APK Signature Scheme v3 and verified live in the Meta Quest Alpha Release Channel.

Challenges we ran into

  • Classical Script vs. Modern Speech Recognition: Modern speech engines transcribe standard contemporary spoken language, whereas the Quran uses 7th-century classical script with archaic orthography and strict Tajweed phonetics. We built an offline audit tool running across 82,456 tokens in 170ms to ensure 100% phonetic reconciliation without false skips.
  • Waveguide Optical Glare During Prostration (Sujūd): In optical see-through AR, bright text reflecting off waveguides directly onto a dark prayer mat breaks the reciter's reverence (Khushū'). Rather than requiring manual button presses (which invalidate prayer), we solved this by acoustically tracking transition Takbeers («الله أكبر») to automatically drop display brightness to 35% with soft spatial blur during prostration, returning to 100% upon rising for Rak'ah 2.
  • Cross-Surah Prefix Collisions & Repetition Floods: Reciters transitioning from Al-Fatihah to other chapters sharing identical opening phrases (e.g. Al-Kahf 18:1 «الْحَمْدُ لِلَّهِ») originally risked causing the HUD to bounce back. We engineered strict Surah context-gating during prayer. Additionally, for repeating refrains across 6,236 verses, we instituted the "Ar-Rahman Rule"—deduplicating intra-Surah repetitions into consolidated cards with repetition badges (Repeats 31x) while preserving multi-Surah candidate selection.
  • Diacritic Preservation During Text Highlighting: Highlighting search tokens inside classical Arabic text usually strips diacritics, corrupting sacred scripture. We solved this with dual-buffer offset mapping that projects normalized search spans back onto raw diacritized Uthmani strings.

Accomplishments that we're proud of

  • Sub-Millisecond Phonetic Tracking & Standalone Execution: Instantaneous syllable verification and two-tone visual feedback on standalone mobile and spatial devices with zero cloud latency.
  • First Zero-Touch Liturgical AR Prompter: Acoustically detects bodily prayer transitions (Takbeerat Al-Intiqāl) and adjusts waveguide optics automatically without a single physical touch.
  • In-Memory 45ms Full-Quran Fuzzy Search: Instantaneous cross-Surah fuzzy searching across all 6,236 verses with speech-to-text mismatch resilience, repetition deduplication, and diacritic-preserving cyan highlights.
  • Zero-Friction Judge Experience: Anyone, anywhere can evaluate the complete live speech recognition engine, hesitation countdown, and spatial controls in 30 seconds via desktop browser simulation or natively in Quest 3/Orion.
  • Full Offline Operation: Operates 100% offline across all 114 chapters with 0ms verse loading and complete Service Worker caching.

What we learned

Optical See-Through (OST) waveguides demand a complete inversion of UI thinking: black is transparent, peripheral vision must be respected, and micro-gestures must be subtle enough to feel imperceptible to surrounding observers. Furthermore, building for sacred traditions requires deep respect for liturgical jurisprudence (Fiqh) where UI interruptions directly impact spiritual validity.

What's next for Quran Orion AR

  • On-device embedded WebAssembly / TFLite Quranic acoustic models.
  • Personalized voice calibration profiles.
  • Native Meta Neural Wristband (sEMG) integration for subtle finger-twitch gesture input.

🧭 Instructions for Judges & Reviewers

⚡ 30-Second Quick Review (Zero Hardware Required)

  1. Open Live App: Navigate to https://quran-orion.pages.dev/?view=ar in any standard browser.
  2. Judge Mode Active by Default: The app opens directly in 🎯 Judge mode with Sheikh Al-Husary sample recitation and English phonetic transliteration pre-loaded and ready.
  3. Test Real-Time Speech Verification (Sample Reciter): In the bottom-left prompter controls, click "▶ Play Sample Reciter (Sheikh Al-Husary)" to watch real-time word verification illuminate in radiant gold.
  4. Test Adaptive Hesitation Assistance (Live Mic Mode): Click "⏸ Pause" (or press D) to stop automated playback, then press Space to activate your Live Microphone. Recite the first 1–2 words (e.g. «الْحَمْدُ لِلَّهِ»), then intentionally pause / hesitate. Notice the circular countdown ring activate; after 1.5 seconds without speech, Orion automatically illuminates only the next single-word hint («رَبِّ») to gently restore reciter flow.
  5. Test Manual Controls & Shortcuts:
    • Press P: Simulates under-25mm Euclidean micro-pinch (unmasks next hint).
    • Double-press P: Advances forward to the next verse.
    • Press S: Opens the spatial Surah & Ayah picker modal with tokenized fuzzy search, multi-surah cards, and cyan highlights.
    • Press C: Toggles webcam passthrough (simulates see-through optical background).
    • Hover over bottom border: Simulates downward floor prostration gaze-dwell trigger (Mawdi' al-Sujud).

👓 Eye-Gaze Selection & Neural Micro-Pinch

  • Eye-Gaze Targeting: Look directly at any floating UI element (such as the Surah Badge or Mode Selector). Orion's eye tracking focuses on it.
  • Neural Micro-Pinch: Tap index finger and thumb together (👌) with hands resting in your lap or prayer posture (hands folded). Orion's neural EMG wristband registers the micro-motor nerve signals as an instant click without lifting arms.

⌨️ Meta Orion Hardware Controls & Hotkeys Table

Action Meta Orion Glasses Hardware / Gesture Simulator / Keyboard Shortcut HUD Function / Behavior
Select Surah & Ayah Gaze at Surah Badge + Micro-Pinch S or click Surah badge Opens spatial modal with tokenized fuzzy search, multi-surah cards & cyan highlights
Context Mode Dropdown Gaze at Context Dropdown + Micro-Pinch Click Context dropdown Switch context: Hifz Study, Imam Salah, Imam Follower (with auto-locked pacing), or Judge
Cycle Prompter Mode Gaze at Mode Badge + Micro-Pinch M or click Mode badge Cycles between visual prompter modes: Safety-Net, Continuous Flow, and Hafiz Blind
Takbeer / Rukū' Dimming Speak "الله أكبر" in Imam Salah Voice / Automatic in Prayer Dims HUD to 35% opacity & blur for prostration reverence; auto-brightens on Rak'ah 2 Fatihah
Open Settings Drawer Gaze at "Scene" + Micro-Pinch Click "Scene" button Opens drawer to adjust Speech Pacing (Strict vs Smart Assist), Far-Field Boost & Background
Far-Field Audio Boost — Settings Drawer toggle / auto-active Studio HAL profile (+6dB Gain + Dynamics Compressor) for distant phone / TV room recitation
Reveal Next Hint Single-Tap on right frame temple arm H or P Unmasks next word hint discretely
Next Verse Double-Tap on right temple arm or double-pinch → or double P Advances forward to next verse
Previous Verse Swipe Backward on right temple arm ← or long-pinch Returns to previous verse
Back 1 Word Left-hand single micro-pinch O Re-masks current word and steps back one word
Rewind Current Verse Left-hand double micro-pinch Double O Rewinds prompter to start of current verse
Gaze-Dwell Hint Look down at floor prostration line (1.2s) Hover over bottom border line Triggers hands-free prostration hint without touch
Toggle Microphone Voice command / automatic in prayer Spacebar Toggles live audio speech recognition with real-time VU meter & blue/red live transcript
Sample Reciter — D or click "Play Sample Reciter" Automated demo of Sheikh Al-Husary recitation with auto-sync
Webcam Passthrough — C Toggles live webcam passthrough background (simulates see-through lenses)
WebXR AR Mode Gaze at XR Badge + Micro-Pinch W Launches/exits WebXR immersive 6DoF session
Foldable / Mobile Mode Auto-detected (Pixel Fold, Galaxy Fold) Browser Fold Simulator (844×1080) Centered optical waveguide, docked drawer, and adaptive 2.15rem typography
Close Modal / Exit HUD — ESC Closes active modal, settings drawer, or exits AR HUD

📖 Public Documentation & Downloadable Booklets

Built With

Share this project:

Updates

Submission history