Inspiration People with alexithymia find it hard to identify and express emotions, both their own and other people’s. A friend says “I’m fine” and they can’t tell whether it was sincere, sarcastic or upset. They can also come across as cold or flat when they don’t mean to. Many socially awkward people face the same problem. Practising with real people is high-pressure, and getting it wrong has real social costs. We wanted a judgement-free place to practise, where the voices sound real.

What it does InterpretTime has two modes: Quickfire: an expressive AI voice says a line, and you tap the emotion you heard. It starts with the basics (happy, sad, angry, afraid, calm). Once you’re ready, it moves on to deeper, subtler emotions: sarcastic, frustrated, disappointed, nervous, excited, embarrassed, bored and relieved. Conversation: a spoken role-play in a real-life scenario. A narrator sets the scene, a character talks to you in an emotional tone, and you reply out loud. Scenarios come in Quick, Medium or Full lengths. Hints are there if you’re stuck, and a gentle nudge appears if you go quiet.

After every reply, InterpretTime checks two things: What you said: whether your words fit how the other person was feeling. How you said it: your pitch, volume and pace compared with your normal speaking voice, to see if your tone matched the moment.

If your words were right but your tone was off, InterpretTime plays your exact reply back in a clone of your own voice, delivered the right way. If you miss twice in a row, it steps in. It says a good reply in your voice, and you copy it until both the words and the tone match.

At the end, you get a review of each reply and a progress panel. The progress panel shows which emotions you recognise reliably and which you’re still learning. It always links new emotions back to ones you already know: “sarcasm often sounds like happy words said in a flat, annoyed way.”

How we built it: Next.js 16, React 19, TypeScript and Tailwind, hosted on Vercel behind a password. ElevenLabs text-to-speech (eleven_v4) with audio tags for emotional delivery. Separate narrator and character voices make scenes feel like real conversations. ElevenLabs speech-to-text (Scribe) transcribes spoken replies. ElevenLabs Instant Voice Cloning makes a copy of the user’s voice from a short reading passage, used only with their consent and deletable at any time. Claude writes the scenarios, judges replies (words first, then tone), gives hints and feedback, and decides when to step in. It returns structured output that we validate with Zod. In-browser voice analysis: a YIN pitch tracker plus loudness, pauses and speaking-rate measurement using the Web Audio API. Results are compared with a baseline recording of the user’s normal voice, so we judge tone relative to them, not to an average person. Mock mode lets the whole app run with scripted responses and no API keys, so we could build and test without spending credit.

Challenges we ran into: Making emotions subtle. Our first sarcastic lines were cartoonish (“Oh great…”). Real sarcasm is quieter, so we reworked the prompts and voice tags to sound like real people. Measuring tone fairly. Everyone’s voice is different, so a fixed “angry = loud” rule doesn’t work. Comparing against each user’s own baseline solved this. Voice clone quality. Early clones didn’t sound much like the user. A longer reading passage and higher similarity settings made a big difference. API limits. We hit voice-library restrictions and concurrency limits on the free plan, and had to make multi-voice scenes request audio one voice at a time. Feedback that helps rather than judges. Our audience is often anxious about getting emotions “wrong”, so the tone of the feedback mattered as much as its accuracy. Accomplishments that we’re proud of Hearing your own voice say your reply the right way is a powerful moment. You can’t learn that from a textbook. The step-in loop turns repeated mistakes into guided practice instead of frustration. We built a full working product, from spoken conversations to progress tracking, in a short time, and it’s live.

What we learned: Emotion lives as much in how something is said as in what is said, and both can be taught. Expressive TTS has become realistic enough to train emotional listening. Designing for neurodivergent users means being clear, patient and kind in every message.

What’s next for InterpretTime: Reading facial expressions alongside tone of voice. More scenarios (work, school, dating, family) and user-requested situations. Long-term progress tracking across sessions, plus accounts. Working with therapists and alexithymia communities to test how well it helps. A mobile app for practising on the go.

Built With

Share this project:

Updates

Submission history