Voice Studio

Turn any document into a faithful, affordable audiobook.

Comment

Inspiration

Voice Studio began as a small local experiment. I wanted to turn documents from my ongoing master’s degree into something I could listen to during spare moments: while driving, relaxing, or even taking a bath. I built a one-off prototype with GPT-5.6 Sol and a local Chatterbox setup to see whether the result could sound good enough for hours of listening.

What it does

Voice Studio turns text-based PDFs, DOCX files, Markdown, and plain text into navigable audiobooks. It cleans distracting formatting, proposes chapter markers, preserves the original meaning, and lets users review the structure before generating audio. Users can choose a built-in voice or provide their own reference. The system supports Spanish and English, with a focus on natural long-form narration rather than short voice demos.

How we built it

I used Codex with GPT-5.6 throughout the build for implementation, debugging, benchmarking, and documentation. Delegated agents helped with document cleanup, benchmark runs, asset discovery, and security audits.

The frontend uses Next.js, while the API and orchestration layer use Node.js and TypeScript. A separate Python/CUDA worker runs Chatterbox for voice synthesis.

Documents are processed in bounded segments. The formatting pipeline validates every result, retries recoverable failures, and stores checkpoints so interrupted jobs can resume. MongoDB stores workflow state, while S3-compatible storage keeps source files, audio segments, and final artifacts durable.

Challenges we ran into

The first challenge was quality. Lightweight voices were affordable but sounded too robotic for long listening sessions, so we tested several options and chose Chatterbox as the quality-first engine.

The second challenge was document structure. Models sometimes returned conversational answers, malformed tool calls, or unreliable chapter titles. We introduced a simpler text protocol, deterministic outline detection, bounded segments, validation, and explicit progress reporting.

Cloud reliability created another set of problems. We had to handle cold starts, signed URL differences, private networking, missing storage checkpoints, and workers restarting halfway through an audiobook. These failures led us to make storage and recovery part of the core design.

More challenges and solutions are well described in the project's repository.

Accomplishments that we're proud of

We are proud that Voice Studio grew from a local one-use prototype into a working local and cloud system without losing its original purpose.

The project now has:

  • A complete document-to-audiobook workflow.
  • Natural Spanish and English voice options.
  • Reviewable and editable chapter titles.
  • Resumable document formatting and audio generation.
  • Durable cloud storage for intermediate and final artifacts.
  • Local Docker Compose and Tilt workflows for a good development experience.
  • A deployed application and static landing page.
  • Benchmarks that guided model and outline decisions instead of relying on a single successful run.
  • Optimized for very low-cost, high-quality audio output.

Most importantly, the output is designed to preserve the source rather than summarize or rewrite it.

What we learned

We learned that model quality is only one part of a reliable AI product. Clear contracts, bounded inputs, validation, retries, observability, and persistence are just as important. We also learned that the best workflow is often hybrid. Deterministic code is better at preserving document structure and tracking state, while models are better at improving readability and pronunciation within clearly defined boundaries. Finally, we learned to keep the user experience honest. A cold GPU worker or a long document may take time, but clear progress, useful messages, and resumability make that wait feel understandable rather than broken.

What's next for Voice Studio

Next, we want to make the public demo easier to explore, expand the voice and language catalog, and continue lowering the cost of generation.

We also would like to improve provenance tracking for reusable voice assets, offer more control over narration preferences, and eventually allow users to edit the cleaned text itself while preserving a clear record of what changed.

The long-term goal is simple: make more of the things people need to read available as faithful, affordable audio.

Built With

Share this project:

Updates