PollyGlott

Live translation with conversational intelligence, helping people understand not just the words, but what matters and what to say next.

Comment

Inspiration

Travel and cross-language conversations still feel mediated by technology. Most translation apps make people stop, record, press buttons, wait for a translation, and then repeat the process in the other direction.

PollyGlott started with a simpler question:

What would a translator look like if the technology almost disappeared from the conversation?

The goal is not merely to translate sentences. It is to help two people who do not share a language communicate as naturally as possible.

What it does

PollyGlott is a live conversation translator designed around continuous understanding rather than turn-by-turn translation.

When another person speaks, PollyGlott can:

  • continuously listen to speech
  • show an ongoing translation in the user's home language
  • preserve the original speech alongside it
  • maintain a compact Live Intelligence layer with:
    • what is happening now
    • important names, numbers, commitments, ambiguities, intent, register, and cultural cues
    • useful things the user may want to say next

Suggested replies are never spoken automatically.

The user selects a suggestion, reviews or edits the translated response, and explicitly presses Approve & Speak before PollyGlott speaks to the other person.

The objective is to get as close as possible to a non-mediated conversation.

PollyGlott is being designed around two operating models:

Model N is the networked full-intelligence experience, combining live speech recognition, translation, and AI-powered conversational intelligence.

Model C is the local-first model for environments where cloud services may not be appropriate or reliably available. The interaction model remains the same while the underlying speech and translation engines change.

Current Model N language packs include Spanish, Korean, Mandarin, Cantonese, French, German, Hindi, Bangla, and Japanese, with additional languages planned.

How we built it

The mobile application is built with React Native, Expo, and TypeScript.

The current iOS live-recognition implementation uses Apple's speech-recognition APIs to obtain partial speech results while a person is still speaking.

A Cloudflare Worker provides the networked application backend and mediates translation and Live Intelligence services. OpenAI powers the current Model N intelligence layer.

The architecture also includes:

  • language-pack based translation behavior
  • RevenueCat subscription entitlements
  • Stripe-based Travel Pass support
  • signed offline entitlement leases
  • secure device storage
  • separate networked and local-first operating profiles

The user interface was deliberately separated into:

  • a sparse live conversation Home screen
  • Language Packs
  • Sessions
  • Settings
  • a Review Reply flow

The Settings experience also supports System, Light, and Dark appearance modes.

A core implementation rule is approve before speak. AI-generated suggestions cannot independently speak on the user's behalf. The user must see, optionally edit, and explicitly approve the translated response first.

Challenges we ran into

The hardest problem was not basic translation.

An early version used a conventional:

record → stop → translate

workflow.

It technically worked, but it failed the actual product test. The user still had to think about when to stop recording, when to translate, and what button to press next.

That led us to redesign PollyGlott around continuous partial speech recognition and live translation.

Other challenges included:

  • keeping translation latency low enough for natural conversation
  • making partial translations useful before an utterance is complete
  • preventing AI-generated replies from speaking without approval
  • handling interruption and barge-in naturally
  • supporting mixed-language conversations
  • keeping Live Intelligence useful without overwhelming the screen
  • designing cloud and local-first models without creating two separate products
  • validating device-level speech support across languages, especially Cantonese
  • separating commerce, diagnostics, and language management from the live conversation experience

Accomplishments that we're proud of

We are proud that PollyGlott evolved from a working translator into a much more ambitious live-conversation system.

Some of the milestones we reached include:

  • continuous iOS speech recognition with partial results
  • live translation while the other person is still speaking
  • a rolling Live Intelligence layer with Now, Important, and Say next
  • explicit Review Reply → Approve & Speak control
  • multiple language packs in a common architecture
  • a sparse conversation-first interface
  • persistent System / Light / Dark appearance settings
  • a networked Model N architecture
  • a separate local-first Model C architecture
  • Cloudflare Worker backend services
  • RevenueCat and Stripe commerce architecture
  • signed offline entitlement leases for travel scenarios

Most importantly, the product direction became clearer through testing: if the user has to think about operating the translator, the experience is not finished.

What we learned

The biggest lesson was that translation quality alone is not enough.

A conversation can still feel broken even when every sentence is translated correctly.

Latency, interruption behavior, visual simplicity, context, reply assistance, and user control all matter.

We also learned that a good AI interface often requires showing less, not more.

As PollyGlott became more intelligent, the main conversation interface became simpler.

The other important lesson was that AI should assist communication without taking control of it. That is why approve-before-speak became a hard product invariant.

What's next for PollyGlott

The next milestones are:

  • automatic multilingual and mixed-language recognition
  • more natural Mandarin / Cantonese / English switching
  • improved barge-in so PollyGlott stops speaking and resumes listening when the other person interrupts
  • lower partial-translation latency
  • better long-session stability
  • broader language support
  • stronger Model C local-first capabilities
  • more device-level language validation
  • continued refinement of Live Intelligence without adding visual clutter

The long-term goal is simple:

Put the phone between two people who do not share a language and let them forget that the phone is there.

Built With

Share this project:

Updates

posted an update —

PollyGlott is now submitted to RevenueCat Ship-a-ton 2026. The current build focuses on continuous live translation, Live Intelligence, and explicit approve-before-speak control. Next up: lower latency, stronger mixed-language recognition, and deeper local-first support.

Log in or sign up for Devpost to join the conversation.

Submission history