TeLeo: Voice Transcription

Remembering what was said in a conversation, class, meeting is hard. Worse if you're deaf or have ADHD. TeLeo fixes that: Record w/ live captions, summarize, chat, improve or translate transcription.

Comment

Inspiration

This one is personal, I got a little sister who is deaf and inspired this app based on Apple's llm on-device tools and advancements in llms overall.

She is a Bolivian Sign Language Teacher in an English speaking school. Where she is not allowed to put on her hearing aids and must solely communicate in sign language with the students or staff. Problem is she does not speak English and does not hear at all without her hearing aids, which makes it really hard to comprehend what is going on in different circumstances.

What it does

You can import or record an audio (with live captions), then one can summarize the transcription or interact with it asking questions in a chat. Besides that, the audio might be in different languages, or poorly transcribed (noise in the background, far away, etc.). Therefore with Pro one can improve the transcription all together and have a better presented transcription to the user. Lastly, one can translate into other languages! Say we are recording a language we do not know about, then we can simply translate the transcription to your mother tongue and continue the conversation. "Type" allows Deaf people who are communicating with someone who does not know Sign Language, by type what it wants to converse about.

How we built it

Native iOS app, with Apple's llm on-device. Uses min iOS version 26.2. RevenueCat ofc for monetization. Always. Backend I used fastapi python.

Challenges we ran into

Some technical challenges as:

  • Handling correctly the on-device llms.
  • Keeping recording, playback, transcription, translation, summary, and chat lifecycle states consistent.
  • Handling interruptions, cancellation, background audio, network loss, and unfinished recordings safely.
  • Pip was a bit annoying too. But beyond technical challenges, I had to comprehend if this was truly solving a problem for my little sister. I currently live in Canada, and she lives in Bolivia. Therefore the distance and time difference did not help much.

Accomplishments that we're proud of

Be able to help my sister and others in need. Already the free features are enough to make a big impact based on continuous feedback received on the usage of recordings. (e.g. usually the need is 15mins of recording, mainly summarize (they don't read all the poor transcription), and perhaps do one or 2 follow up questions - all this can be done as a free user)

What we learned

So much about llm transcriptions + backend. I actually was not sold on this app based on the quality of llms and kept a rough version for my little sister. Until recently when latest llms really contribute a big help for her.

I think this can be better expanded. Which motivates me to get my hands dirty and learn more about RAG to see how embedding models could possibly contribute efficient recollection of data to assist people.

What's next for TeLeo

I want to keep serving people who are often left out, helping them follow and understand what's happening around them. That goes beyond the deaf and hard-of-hearing community to include older adults, people with ADHD, and others.

Built With

Share this project:

Updates

Submission history