Ballad AI

People have stopped reading in 2025. Ballad AI uses AI to generate dynamic soundtracks for any book, making stories as engaging as possible. We're bringing reading back by giving it a soundtrack.

Comment

Inspiration

Reading is dying in 2025. With attention spans shortened by TikTok and Netflix, books feel static and boring compared to multimedia entertainment. As avid readers ourselves, we wished for a method to make book more immersive. We asked: what if every story had its own dynamic soundtrack that adapted to the narrative?

What it does

Ballad AI analyzes any book using natural language processing to understand emotional beats, pacing, and atmosphere. It then generates personalized musical accompaniments that sync with the reading experience—building tension during climactic scenes, adding mystery to detective stories, or creating wonder during fantasy adventures. Readers get an immersive, multimedia experience that makes books better.

How we built it

We combined several AI technologies:

  • Text Analysis: Language models to extract emotional sentiment, mood, and narrative pacing from literary text.
  • Music Generation: Generates dynamic soundscapes based on textual analysis
  • Real-time Sync: Built a reading app that reads the book aloud and adjusts music in real-time
  • Web Interface: Created a responsive web application using React for seamless user experience

Challenges we ran into

  • Fine-grained Music Control: We originally envisioned layering many music tracks so that we can dynamically shift the mood of the music in real time. However, current available APIs only offer text-conditioned generation with a maximum of 30s, so we had to compensate with a fading and looping effect.
  • Processing Time: Current music generation is slow, so it's difficult to generate music in real time.

Accomplishments that we're proud of

  • Successfully generated coherent, emotionally-appropriate soundtracks for diverse genres
  • Created smooth transitions between musical themes that enhance rather than distract from reading
  • Built a working prototype that demonstrates the concept's viability

What we learned

  • Our frontend was partially developed by AI, but we found AI generated code to be difficult to maintain.
  • Most of our members had to pick at least 1 unknown framework such as FastAPI, Next JS, etc.

What's next for Ballad AI

  • Implementing user voice recognition so we can allow for the user to read the book aloud
  • Eye tracking, generating visual elements to accompany the music and narrative
  • Better musical control with Lyria Realtime API

Built With

Share this project:

Updates