Room to Speak

• 

• 

  Take one photo of his room, or try an example room.

• 

  AI rings the things he'd want to say, each with a phrase.

• 

  He taps the walker. The room dims and the device speaks for him.

• 

  Setup needs the internet once. Everything else runs on the device.

• 

  One photo, rings, one tap.

Inspiration

After a stroke, many people with aphasia understand everything but can't find the words. Hospital picture cards (EAT, DRINK, TOILET) look nothing like their life, so they ignore them. A board made from their own room works better, but a therapist has to build it by hand. (Add your own reason here, one line.)

What it does

• The caregiver takes one photo of his room.
• AI puts rings on the things he'd want to talk about, each with a short phrase like "Aku mau kopi" (I'd like coffee).
• The caregiver can hear, edit, remove or add spots.
• He taps an object, and the device speaks for him.
• Yes / No / Help / Pain / Toilet are always on screen.
• Everything stays on the device. No account.
• No photo? Tap one of four example rooms.

How we built it

• Planned with the Devpost Learn Skill Pack: scope → PRD → spec → build checklist.
• React + Vite + TypeScript.
• One Vercel function finds the objects: Gemini first, DeepSeek as backup.
• The phone's own voice speaks the phrases.

Challenges we ran into

• Gemini was sometimes busy or slow, so I added a backup chain to DeepSeek.
• Rings overlapped on small screens, so they get nudged apart.
• A resting hand could trigger speech, so a tap only counts when the finger lifts.

Accomplishments that we're proud of

• It works end to end, live, in Indonesian and English.
• Anyone can try it in seconds with the example rooms.
• Big targets, high contrast, keyboard support.

What we learned

• Planning first made the build fast and calm.
• Testing against the real AI caught bugs that mock tests missed.

What's next for Room of Speak

• A recorded family voice.
• More rooms.
• A tablet app for the bedside.
• Testing with real families and speech therapists.

Built With

• deepseek
• gemini
• playwright
• react
• typescript
• vercel
• vite
• vitest
• web-speech-api