VisualEd

"Turn any textbook page into AI-generated visuals. Learn by seeing, not just reading."

Inspiration

Textbooks haven't changed in decades. Dense paragraphs, maybe one diagram, and students are expected to just get it. We knew there had to be a better way. Visual learning dramatically improves retention, yet most students don't have access to rich visual explainers for every topic they study.

We built VisualEd to close that gap. Point your VR headset at any textbook page and watch the concept come to life as an AI-generated image and video, instantly.

What it does

VisualEd is a VR-assisted AI learning tool. Here's the flow:

1. Capture: Use your VR headset camera to photograph a textbook passage
2. Extract: OCR pulls the raw text from the image
3. Understand: An LLM interprets the concept and generates a rich visual prompt
4. Visualize: An image generation model produces an illustrative diagram

How we built it

We chained four systems into one seamless pipeline:

1. VR Headset: as the capture device (camera input)
2. OCR Engine: to extract clean text from the captured image
3. LLM: to interpret the concept and write a precise visual prompt
4. Visual Generation Models: to produce the final educational visuals

Everything is connected through a custom integration layer that moves data from capture to text to image to video automatically.

Challenges we ran into

OCR accuracy was our biggest early hurdle. Textbook layouts with small fonts, multi-column formatting, and embedded figures made clean text extraction harder than expected. We also spent significant time on prompt engineering to ensure the AI generated educationally accurate visuals, not just aesthetically similar ones. Chaining four models together introduced latency we had to optimize, and VR camera lighting conditions affected OCR reliability in ways we didn't anticipate.

Accomplishments that we're proud of

We successfully built a working end-to-end pipeline from a physical textbook page to a visual in a single hackathon session. We also found a genuinely novel use for VR hardware: not as a gaming device, but as an intelligent reading companion.

What we learned

We learned how to design multi-model AI pipelines where each step feeds the next. Prompt engineering turned out to be as important as the models themselves. The quality of the visual prompt determines everything downstream. We also learned that OCR preprocessing (contrast adjustment, cropping, denoising) makes a massive difference in text extraction accuracy.

Why Spectacles 2026 will make VisualEd even more powerful

Specs 2026 will read text better because of three stacked improvements working together:

1. Vision-language AI models (GPT-4o, Gemini) replacing traditional OCR, handling real-world conditions like glare, angles, and faded print far better. Source
2. On-device Snapdragon XR processing for near-instant recognition without cloud round-trips. Source
3. Improved hardware with a lighter form factor and anticipated camera upgrades over the 226g dev kit. Source

What's next for VisualEd

Real-time processing: reduce latency for instant in-headset feedback Subject-specific tuning: optimized pipelines for science, math, history, and medicine Math support: render equations and formulas as 3D visualizations Mobile version: extend beyond VR to smartphone cameras for accessibility LMS integration: connect with Canvas or Google Classroom for classroom use

Built With

• typescript