Eye Test Result
Filtering On

OptiX: Software-Based Vision Correction

Inspiration

Eye conditions may change, but changing lenses can be expensive and impractical. Using glasses for long periods can also be tiring, and sometimes we lose them altogether.
So, we thought: what if we could make vision correction entirely free and software-based, a digital layer that lives on your desktop and corrects your vision in real time?

What It Does

OptiX is a two-part end-to-end optometry software solution:

A. AI-Automated Eye Test (Powered by ElevenLabs + xAI)

When users don’t know their eye condition, OptiX runs an AI-driven eye test similar to what you’d experience at an optometrist’s office.
Using an ElevenLabs voice agent, the app guides the user through visual tests — asking them to identify letters or focus on targets, while xAI analyzes responses in real time to adjust the next question dynamically.
This conversational, adaptive approach lets users test their vision without needing an in-person appointment.

B. Software-Based Pre-Correction (Backed by Research)

Once the user’s condition (e.g., myopia, astigmatism) is identified, OptiX applies a screen pre-correction layer that applies algorithms of pre-corrections on each pixel that directly convolutes and modifies pixels directly on the display.
Based on formulas from vision correction research, this filter compensates for visual aberrations — letting users see their screens clearly without physical glasses.

How We Built It

OptiX consists of two main modules:

A. AI-Automated Eye Test

Tech Stack: ElevenLabs VoiceCall Agents + xAI API
Functionality: AI optometrist conducts the exam conversationally.
Storage: Patient and test data stored in SQL.

B. Software-Based Pre-Correction

Framework: Electron.js for desktop app framework, React for frontend UI.
Overlay Rendering: Implemented using WebGL and Electron transparent layers.
Computation: Real-time filter calculation based on the user’s sphere, cylinder, and axis.
Interface: Adjustable sliders for eye parameters; overlay toggled via keyboard shortcuts.

Challenges We Ran Into

Implementing real-time pre-correction through WebGPU and WebGL initially failed — each method recursively stacked screenshot layers, causing an infinite capture loop.
Attempting GPU driver hooks (DXGI device) was too intrusive and inconsistent.
Translating research-grade optical algorithms into real-world computation was difficult — it required manual verification (literally taking off our glasses) to test the visual effects.
Synchronizing ElevenLabs Agent SDK with xAI’s adaptive workflow was complex; both systems had to remain tightly in sync.

Accomplishments That We're Proud Of

Built the first functional prototype of a software-based optometric system that bridges AI-assisted diagnosis and display-level vision correction.
Successfully implemented real-time optical parameter adjustments based on peer-reviewed research.
Addressed a meaningful accessibility problem, helping people with vision impairments interact with screens more comfortably and affordably.

What We Learned

Real-time image processing on standard GPUs is harder than expected, performance and latency are real constraints.
Academic research often assumes ideal hardware; implementing these techniques in consumer settings exposes practical limits of hardware and rendering pipelines.

What’s Next for OptiX

**Integrate directly on hardware device driver level with DXGI hooks for both Windows and Macbook, so that the pre-correction for the screen is done without screen capture. This will significantly reduce latency, making it more fit for gaming.
Implement eye-tracking and gesture control, enabling actions like zooming or panning via eye movement, potentially benefiting users with motor disabilities.
Conduct our own research to adapt optical correction algorithms for consumer displays.
Continue pioneering software-based optometric technology as a new accessibility frontier.

References

Xu, Feng and Dayang Li. “Software Based Visual Aberration Correction for HMDs.” 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (2018): 246–250.
Fu-Chung Huang, Gordon Wetzstein, Brian A. Barsky, and Ramesh Raskar. “Eyeglasses-free display: towards correcting visual aberrations with computational light field displays.” ACM Transactions on Graphics, Vol. 33, No. 4, Article 59 (2014). https://doi.org/10.1145/2601097.2601122