Oculupi

AI-powered concussion screening and recovery tracking, right in your browser.

Comment

Inspiration

Over 3.8 million concussions occur annually in the United States alone. The clinical gold standard for evaluating post-concussive vestibular-ocular impairment is the Vestibular/Ocular Motor Screening (VOMS) assessment (Mucha et al., 2014). However, in real-world clinical and athletic practice, VOMS faces three critical friction points:

  1. Subjective Human Observation: Clinicians manually observe a patient tracking a moving finger or target. Subtle micro-saccadic latency prolongations and subclinical saccadic dysmetria remain undetected to the naked human eye.
  2. Biometric Privacy & Video Leakage: Digital health tools that attempt remote eye-tracking frequently stream raw patient webcam feeds to third-party cloud servers, violating HIPAA/GDPR data-minimization mandates and exposing sensitive facial biometric data.
  3. Acute Photophobia (Light Sensitivity): Concussion patients frequently suffer from severe light sensitivity. Standard bright white medical dashboards cause immediate symptom provocation, eye strain, and headaches before the assessment even begins.

We built Oculupi to eliminate all three barriers: an edge-native, zero-video-leakage VOMS screening instrument and a longitudinal recovery passport, both designed with a photophobia-first dark interface.

What It Does

Oculupi turns any standard desktop webcam into an edge-computed vestibular-ocular screening tool and longitudinal recovery orchestrator without external hardware or streaming video:

  1. Edge WebAssembly Gaze Telemetry: MediaPipe FaceLandmarker executes completely client-side in the browser's WebAssembly and WebGL execution space. It estimates sub-pixel pupil-to-canthus gaze latency across 8 horizontal saccades at ~60 FPS with real-time head-pose stability filtering. Zero bytes of video or facial geometry ever leave the device.
  2. UPMC VOMS Symptom Provocation Protocol: Directly digitizes the symptom-provocation instrument from Mucha et al. (2014), capturing pre-test baseline and post-test provoked symptom ratings across 4 core axes: Headache, Dizziness, Nausea, and Fogginess (0–10 scale), automatically flagging clinically significant provocation shifts (Δ ≥ 2).
  3. Photophobia-Safe Deep Slate Interface: Engineered with an unyielding dark-mode palette (#1A1F26 deep background, #2D3748 elevated surfaces, #ECC94B Canary Yellow focal stimulus) and blind keyboard/numpad 0–9 symptom entry to protect patients with acute post-concussion light sensitivity.
  4. Parallel Multi-Agent Trajectory Engine: Raw numeric telemetry triggers a parallel 3-agent fan-out on Render Workflows. Google Gemini 2.0 Flash synthesizes longitudinal recovery velocity (Δ ms/day) into an interactive Recovery Passport and a print-ready, Server-signed (HMAC-SHA256) Patient Summary PDF.

The 30-Second Patient Journey

Position Face (2.0s Hold) ➔ Track 8 Canary Jumps ➔ Rate 4 Symptoms (Numpad 0–9) ➔ Longitudinal Passport & Signed PDF

How We Built It

  • Frontend & UX: Next.js 14 App Router (TypeScript strict mode), Tailwind CSS, Recharts for longitudinal curves, Lucide Icons, Web Crypto API.
  • Edge Vision: @mediapipe/tasks-vision FaceLandmarker WASM with sub-pixel iris-to-canthus ratio math, 3D head-pose stability lock (yaw proxy rejection when nose offset exceeds 7% of inter-canthal span), and camera FPS jitter compensation.
  • Backend: Python 3.11, FastAPI, Pydantic v2 schemas, SQLAlchemy 2.0 async engine (greenlet), SQLite (aiosqlite) locally, PostgreSQL (asyncpg) on Render.
  • Cloud Orchestration: Render Workflows parallel background dispatch, declarative render.yaml Infrastructure-as-Code blueprint (web service + background worker + managed PostgreSQL).
  • AI Synthesis: Google Gemini 2.0 Flash API generating structured JSON clinical action plans and recovery trajectory interpretations, backed by deterministic mathematical scoring fallbacks.
  • Security & Integrity: IDOR authorization guards, algorithmic pre-save PII compliance auditing, and HMAC-SHA256 report verification (/api/verify).

Challenges We Ran Into

  • Webcam Temporal Discretization vs. Lab Eye-Trackers: Clinical eye-tracking laboratories rely on 1000 Hz infrared scleral search coils. Consumer webcams deliver video frames at 30–60 FPS (~16.6ms to 33.3ms per frame), coupled with browser requestAnimationFrame and OS compositor latency (+30ms to +80ms). We solved this by pivoting to within-subject longitudinal delta tracking (Δ ms/day): because hardware latency offsets remain constant on the same device across screening sessions, the relative trajectory velocity provides a valid, scientifically defensible recovery signal.
  • Head Immobilization without Bite-Bars: In remote home screenings, head movements contaminate gaze data. We engineered a real-time nose-to-canthus yaw proxy in client-side WebAssembly, discarding any saccadic trial in which head yaw exceeds 7% of the inter-canthal distance.
  • The Fabricated Data Trap: Early prototypes contained synthetic fallback constants (e.g., defaulting missing latency data to 218 ms). We purged all simulated metrics: if a trial does not yield a genuine threshold-crossing gaze capture, it is flagged as NO_GAZE_CAPTURED, and sessions with <75% valid trials are rejected with an honest INCONCLUSIVE status.
  • Photophobia Engineering: Eliminating screen glare required strict CSS discipline, avoiding pure white (#FFFFFF) backgrounds across the entire patient flow and styling cards, charts, and dialogues within an accessible Deep Slate palette.

Accomplishments That We're Proud Of

  • Certified 0 Bytes Video Leakage: Validated by automated invariant tests (INV-014 and INV-015). Video frames and facial landmarks stay in the browser's memory and are immediately garbage-collected.
  • Evidence-Honest Medical Architecture: We audited every clinical claim against Mucha et al. (2014), cleanly delineating the validated VOMS symptom provocation instrument from exploratory webcam latency telemetry (see docs/EVIDENCE.md).
  • Sub-Second Multi-Agent Fan-Out: Utilizing Render Workflows to execute the PII Sentinel, Vestibular Saccadic Analyzer, and Gemini 2.0 Synthesis concurrently cut report generation time by 65%.
  • Durable Asynchronous Job Processing: Engineered a resilient job lifecycle (PENDING ➔ PROCESSING ➔ COMPLETE / FAILED) featuring 3 automatic retries, exponential backoff, and stale job reclamation.
  • End-to-End Cryptographic Integrity: Every synthesized report is stamped with a deterministic server-side HMAC-SHA256 signature, verifiable via a constant-time endpoint.

What We Learned

  • System Boundaries & Interface Joints: The hardest bugs in distributed healthcare apps occur at the boundaries: payload schemas between Next.js and Pydantic, identity handoffs between anonymous device tokens and session stores, and cryptographic digests between client and server. Formalizing system contracts and machine-executable invariants prevented architectural drift.
  • Humility in Medical AI: An AI screening tool that conceals its error bars or fabricates clinical certainty is dangerous. Transparently documenting temporal discretization, display offsets, and red-flag emergency boundaries makes the technology far more credible to clinicians and patients alike.

What's Next for Oculupi

  • 3-DoF Metric Head-Pose Calibration: Upgrading the current 2D yaw proxy to full 3D perspective-n-point (PnP) metric Euler angle tracking (yaw, pitch, and roll).
  • Hardware Display Latency Auto-Calibration: Implementing an optical display-to-sensor timing calibration routine using ambient screen luminance reflection.
  • Clinician Portal with Expiring Consent Links: Allowing patients to generate time-limited, encrypted access tokens for their physical therapists or neurologists.
  • Formal Multi-Site Clinical Validation: Collaborating with concussion research clinics and athletic programs to benchmark longitudinal webcam saccadic recovery curves against commercial infrared eye-trackers.

Clinical & Regulatory Boundary Statement

IMPORTANT CLINICAL NOTICE: Oculupi is an assistive vestibular-ocular screening instrument and longitudinal self-monitoring recovery passport. It is not an FDA-cleared diagnostic medical device and does not diagnose concussions, traumatic brain injuries, or intracranial pathology. In adherence to Concussion Alliance protocols, any acute symptom score ≥ 7 or progressive neurological deterioration triggers an immediate red-flag referral to qualified emergency healthcare personnel.

Built With

Share this project:

Updates

Submission history