Gem Reader

Not everyone reads the same way. Gem Reader uses Gemini to turn any document, in any language, into a natural-voice narration — so no book goes unread

Comment

Inspiration

Like most of the planet, there was a time I had to stay home for a long while. That's when, past 30, I picked up a habit I had always wanted: reading. For the first time in my life I finished an entire book. Then came another, and another, until that year I was reading almost two books a month. I can say today it was one of the years I grew the most in my whole life. Something shifted in how I thought, in how I saw things, and I'm sure it was because of the books.

Then, little by little, everything went back to normal: more going out, more work, more things to do, and above all, more distractions. Since then, no matter how much I've wanted to and how many books I've bought, I haven't been able to pick the habit back up.

That's where the idea for Gem Reader came from. Looking for a way to get back into reading, I bought the audiobook version of a book I also had in print, and started listening and reading at the same time. I was stunned. I'd seen actors narrate their own books in special sessions in movies, but I didn't know the experience of reading and listening at once was this powerful. It forces you to truly focus, without distractions.

I remember thinking it would be great to find an app where you could take a photo of a book or any text and have it narrated to you while you read. I looked for alternatives: they exist, some with very good voices, but none fully convinced me. To this day I haven't found an app that natively lets you photograph a text and have it narrated directly; you always have to run the photos through OCR, convert them to PDF, and only then import them. I thought maybe Gemini could solve exactly that, but I also noticed that every alternative out there, including that first idea, was still limited.

That's the exact point where I thought: why not build something that doesn't just help bring back the habit of reading, but helps everyone who can't read for one reason or another, physical, time, or language?

That's how Gem Reader was born: an app that aims to remove those barriers. Being able to listen to any document, any story, any text, in any language, narrated with natural voices you choose yourself. It came from my own path back to reading, but it ended up becoming, above all, a project so that everyone gets the chance to read, no excuses.

What it does

Gem Reader turns any document (a PDF, an EPUB, a photo of a scanned page, a public-domain book) into a natural listening and reading experience, in the reader's own language, with a voice that understands the shape of the text (headings, notes, quotes) instead of reading characters in a row.

It's not a generic text reader. It's a digital publishing operation run almost entirely by AI agents: Gemini enriches every book that enters the catalog (synopsis, BISAC categories, publication year), decides what gets published and what gets rejected, detects language, answers support questions in production, and Google Cloud TTS narrates the text with natural voices. All with English, Spanish, and Portuguese as interface languages from day one.

Category: Education & Human Potential. The thesis: access to a book shouldn't depend on how, when, or in what language you can read it.

How we built it

  • Backend: Node.js + Express, a simple, direct architecture built to iterate fast during the 90 days of the hackathon.
  • Frontend: React 19 + TypeScript, installable PWA, full i18n across 3 languages.
  • AI: Google Gemini (OCR, catalog enrichment, reading chat, support agent) + Google Cloud Text-to-Speech for narration.
  • Data: Firebase Auth + Firestore + Cloud Storage.
  • A piece that's fundamental to the project: a custom telemetry layer (telemetry.js) that logs every Gemini call with its real cost, plus a separate "operations decisions" collection (opsEvents) that records the why behind every autonomous decision an agent makes. Not just that the AI was called, but what it decided and with what confidence.

Challenges we ran into

  • Getting narration to play without any gaps. The player went through several iterations (live streaming while the audio is still being generated, then stable persistence in the cloud with session resume) before reaching reliable, chapter-by-chapter playback. None of the business side of this project means anything if the audio doesn't sound right.
  • Designing for agents that can be audited, not just that work. Anyone can call an LLM; the real challenge was making every call log itself as a traceable business decision, with its reasoning, its confidence, and its cost, so it could be shown, not just so it would work in the demo.
  • Costing things for real, not estimating them. While building our unit economics numbers we found we were measuring the real cost wrong: voice synthesis (Google Cloud TTS), likely the heaviest expense in the whole project, wasn't being logged at all. We had to instrument it with official pricing before we could trust a single number on our own pricing page.
  • Being honest with ourselves about traction. We decided not to chase revenue from friends or family in the final days just to show a number. We'd rather build a real operation, even if that means reporting zero revenue.
  • i18n with no leaks. With three languages and a backend that originally thought in Spanish, we found and fixed server error messages that were leaking untranslated into the English interface, exactly the kind of detail that breaks the impression of a polished product in front of a judge.

Accomplishments that we're proud of

  • Above all, having built this entire application in such a short time and arriving with a real product, running in production, ready to be demoed. Not a mockup.
  • Having run into serious technical problems along the way and, with patience, having worked through every one of them, with no shortcuts left half-done.
  • 545 autonomous AI decisions logged in production, not in a test environment, on the real server.
  • 99% of those decisions resolved without human intervention (escalation rate to a human: 1%).
  • Real, measured (not estimated) cost of every business decision: $0.00123 per decision on average, with a total accumulated spend measured in production of $3.61 across Gemini ($1.69) and Google Cloud TTS ($1.92).
  • Full interface parity across 3 languages (English, Spanish, Portuguese), with no untranslated strings, audited line by line.
  • An operations panel (/admin/operations) where anyone can read, decision by decision, the reasoning the agent used. Not a black box.
  • A custom audio streaming setup (Media Source Extensions) that starts playing while Gemini and Cloud TTS are still generating the rest of the chapter, instead of waiting for the full file.
  • Built, tested, and deployed to production within the hackathon window (first commit: June 7, 2026).

What we learned

  • That Gemini blocks narration when it detects it's reproducing protected text nearly word for word (a copyright safeguard built into the model itself, not a limit of our own infrastructure). We had to design the narration pipeline around that limit from the start, not patch it in after something broke in production.
  • That building on a model that evolves as fast as Gemini demands real resilience: at one point, Google silently retired an entire model family and broke four features at once without warning. We learned not to tie any feature to a single model, but to configurable fallback model lists instead.
  • That gapless narration (streaming audio in real time while Gemini and Cloud TTS are still generating it) is, in practice, one of the hardest engineering problems in the whole project, harder than any prompt.
  • That "AI in the product" and "AI running the business" are completely different things, and that the difference can only be shown with logs, not with talk.
  • That business numbers without real instrumentation are just opinion dressed up as data, and that it's worth pausing feature work to fix the measurement before presenting a figure.
  • That being honest about what doesn't exist yet (revenue, paying users, our own domain) is more defensible in front of a jury than any attempt to dress it up.

What's next for Gem Reader

Short term: validate and stabilize what already exists.

  • Validate the pricing model we've already published with the first real users from the waitlist.
  • Close out the Education plan with per-institution licensing.
  • Our own domain, and moving out of the open beta phase.
  • Expand automated quality control across the catalog.
  • Keep hardening the autonomous operations layer: more business decisions delegated to agents, held to the same auditability standard we've required from day one.

Longer-term vision: once the core product is validated and stable.

  • From "pick a voice" to "create a voice." Today the user selects from predefined voices. The next step is using Gemini's TTS stack so anyone can create a voice from a prompt: an accent from their own country, a specific narrator tone, a particular storytelling style. We believe this is the natural path for Gem Reader: giving the user full control over how the voice that narrates their story sounds, not just which one they pick.
  • A creator suite powered by Gemini. Taking Gem Reader beyond being an editorial content platform: building, inside the app itself, the tools for anyone to write, create, and sell written content, later narrated by Gemini or by our own voices. The idea is for Gem Reader to stop being just the place where a story is heard, and become the tool with which the next ones are written.

Built With

Share this project:

Updates