Real Eyes

Title Screen
Camera Permissions
Dark Truth Tags
Smartphone Tag
Light Truth Mode
Thumb Tag for Manual Scan
Health and Food tag
Truth Revealing Effect
Workspace and Screens
Thumb Tab
Thumb swipe

Overview

Updated video demonstration can be seen here! https://youtu.be/IkuomDehPvw

Instructions

Before testing, please ensure you have an up-to-date scan of your spatial environment (space setup). Please allow spatial and camera permissions in the app's prompt once you start. An internet connection is required.

Real Eyes is a Mixed Reality application and experience for the Meta Quest 3/3S that reveals the hidden truths behind everyday objects, brands, and behaviors. Using on-device passthrough, spatial anchors, multimodal vision models, Real Eyes transforms the world into an interactive lens — exposing environmental, labor, health, privacy, and systemic impacts through dynamic “Truth Tags.”

Real Eyes turns awareness into agency, helping people see the systems shaping their everyday lives in a way that feels immersive, intuitive, and impossible to ignore.

What Inspired It

We live in a world full of polished surfaces — ads, packaging, apps, branding — each designed to influence how we think and behave. But behind every object is a deeper story: • how it was made • who it affects • what systems it feeds • and the consequences we don’t see

Films like “They Live” the rise of AI-driven persuasion, and today’s opaque algorithms motivated me to build a tool that restores transparency instead of exploiting attention. Real Eyes is designed as an anti-manipulation technology — a counter-force to the invisible systems shaping our choices.

What It Does

When the user looks around their environment, Real Eyes:

Analyzes real-world objects through passthrough MR Anchors spatial Truth Tags into physical space Reveals two complementary perspectives:

Dark Truth: systemic harms (environment, labor, health, privacy, corruption) Light Truth: nuance, progress, and actionable alternatives

User sees an Impact Level Score of the overall object detected as well as the levels of severity for each category.

Provides behavior-level warnings (e.g., doomscrolling, screen fatigue, fast-food overconsumption)

Lets the player toggle perspectives with micro-gestures or tap interactions

A full Truth Mode system (Dark ↔ Light) that updates all tags in real time

Persistent anchors allow truths to remain in the environment across sessions

The experience is designed to be clear, respectful, and empowering — never accusatory or personal.

How It Works (Technical Breakdown)

Passthrough Camera Pipeline

Real Eyes uses the Meta Passthrough Camera Access API to capture high-resolution environment frames. A hybrid system ensures performance:

Frame fingerprinting avoids redundant model calls Head-stabilization detection ensures clean, sharp analysis frames Gesture-based “instant scan” for user-driven scanning

Vision LLM Analysis

I built a strict JSON schema for the AI to return:

object category or brand bounding box + normalized coordinates dual truth variants (dark + light) severity scores (0–1 across five categories) optional source links behavior-level warnings All formatting is validated by Unity’s JSON parsing system.

Spatial Placement & Tag Stability

Each Truth Tag uses:

Physics raycasts from the capture-time camera Environment depth (MR Utility Kit) for real-world accurate placement Tag separation logic to avoid overlap Adaptive offsets for objects near the user (like phones) OVRSpatialAnchor for persistence Tags feel rooted in the real world — they don’t float randomly or drift.

Interaction & Usability

Real Eyes supports both left and right hand interactions: Thumb Tab for manual scan Ray-based pointing and selecting (Distance Input) Physical tap triggers like a touchpad/ipad on buttons, panels (Direct Input) Microgesture Thumb tap interactions Dynamic highlighting Micro-gesture swipes handle Dark ↔ Light mode switching.

Ethical Design

Real Eyes avoids:

Personalization Persuasion Emotional manipulation Tracking identity Collecting any user data All analysis happens only on request and is not stored. No personal attributes are inferred. The app’s About section explains exactly how and why it operates.

Why It Matters

Systems surround us, but we rarely see what shapes our health, our environment , our labor conditions , our digital privacy , or our culture and beliefs .

Real Eyes gives users a transparent layer of insight in a medium (MR) uniquely suited for understanding context. It’s not just educational — it’s transformative. A tool like this ideally becomes a personal empowerment layer in emerging XR ecosystems.