The Book Reads you

A storybook that reads you first: it scans your face and mood, then writes a story starring you.

Comment

Inspiration

The inspiration came from a joke a friend and I made a few years ago about books. I came up with the idea of a book that reads you, and thought it would be interesting and funny to actually build it. "Flip the Script" was the perfect excuse.

What it does

You look into your webcam and the book reads you: your glasses, hair, clothes, small details like an earring or a logo, and your mood. Then it writes and reads aloud a short story where you're the hero, every detail matters to the plot, and your mood sets the tone. One photo, used once, never stored.

How we built it

  • Frontend: React + Vite, with MediaPipe Face Landmarker for face tracking, auto-capture and measuring expression.
  • Backend: Node + Express.
  • AI on Nebius: Kimi K3 reads the photo, Gemma 3 and MiniCPM-V double-check it, and Qwen3 writes the story.

Challenges we ran into

  • The AI invented things (a chain the person wasn't wearing), so a second model now has to agree before a detail is kept.
  • It once described skin colour despite being told not to, so we added a hard filter in code.
  • Small details were lost at low resolution, so we send a 1080p close-up of the face.

Accomplishments that we're proud of

  • It notices tiny details and implements them into the story.
  • A sad face gets a gentle, kind story, not a cheerful one.

What we learned

  • Don't trust one AI model's eyes. Make two agree.
  • Important rules belong in code, not just prompts.

What's next for The Book Reads you

  • Two-hero stories with a friend.
  • Illustrations of your character.

Built With

  • express.js
  • gemma
  • kimi-k3
  • minicpm
  • quen
  • vite
  • web-speech-api
Share this project:

Updates

Submission history