LOOB: Immersive AI Reading Companion

Blink to Flip a Page
CAD
CAD + BOOK
Dev Debug
OCR Demo
Reading Assistance
OCR Camera Setup
Application Software
Base

Inspiration

We wanted to make reading physical books feel more accessible, immersive, and hands-free.

LOOB combines the experience of reading a real book with:

Hands-free page turning
Contextual help for difficult words
Voice narration
Immersive audio that matches the story’s mood

What It Does

LOOB uses an eye-tracking camera to detect three intentional blinks. This gesture activates the book-flipping hardware and turns the page.

After the page turns, a second camera scans the new page using optical character recognition (OCR). The scanned text enables AI-powered reading assistance, such as explaining the meaning of a word within the context of the current passage.

LOOB also uses ElevenLabs for voice narration and immersive audio. For example:

A suspenseful passage can use a more fitting narration style and subtle ambient audio.
Specific events, such as a creaking door, can trigger matching sound effects.

How We Built It

We designed and 3D printed a custom mount and book-flipping mechanism. We then connected it to an STM32 microcontroller through a USB-to-TTL UART connection.

On the software side, we built:

An OpenCV and MediaPipe eye tracker for blink and gaze detection
A two-camera workflow: one camera for eye tracking and another for OCR
A timed camera handoff that releases the eye camera before opening the OCR camera
A FastAPI backend to coordinate scans, OCR jobs, and camera state
A reading interface with dedicated Reader and Developer views
OpenAI-powered contextual reading assistance
ElevenLabs narration and immersive sound support

Challenges We Ran Into

Handling Different Physical Pages

Physical pages vary significantly in weight, thickness, texture, and stiffness. A mechanism that reliably turns the page of one book may struggle with another.