Speech Therapy AI

AI tools vibecoded for a speech therapist - echolalia-friendly choice generators, custom visual exercises, and real-time transcription solving actual clinical challenges.

Comment

Speech Therapy AI

Inspiration

My wife is a speech and language therapist working with intellectually disabled adults, and I wanted to create AI tools to streamline her practice and better serve her patients. Each feature was developed for super specific purposes based on real therapeutic needs - from communicating with an intellectually disabled deaf person to teach her sign language, to creating custom exercises for people who wouldn't otherwise pay enough attention in sessions.

What it does

Real-time Transcription Tool: Provides live subtitles for communicating with deaf patients who don't use sign language - essentially real-life captions for face-to-face conversations, enabling therapists to bridge communication gaps with intellectually disabled deaf individuals.

Custom Coloring Pages: Converts any image into therapeutic coloring sheets with customizable color schemes, allowing therapists to create personalized activities that capture attention and maintain engagement for patients who struggle with focus during traditional therapy sessions.

Choice Generator for Echolalia: Creates visual choice presentations that work around echolalia (when patients repeat the last thing they hear), ensuring fair decision-making by presenting options simultaneously rather than sequentially, so patients with this condition can make genuine choices instead of automatically selecting the last option heard.

How we built it

  • Frontend: Bolt.new for rapid development and prototyping
  • Speech Recognition: Deepgram API for real-time transcription
  • Image Processing: Replicate API with Flux Kontext Pro for custom coloring page generation
  • Visual Communication: Arasaac open source API for standard pictograms (essential for representing abstract concepts like emotions or verbs that photos cannot properly describe)
  • Image Resources: Unsplash API for fetching high-quality images to visually represent choice options
  • Deployment: Netlify hosting with Ionos domain
  • Development Approach: "Vibecoding" - building tools based on immediate therapeutic needs rather than following traditional development patterns

Challenges we ran into

  • API integration issues and credit management across multiple services
  • Version control mishaps with Bolt auto-commits nearly breaking the app
  • Learning GitHub recovery to restore previous versions
  • Code cleanup while maintaining functionality
  • Designing interfaces that work for patients with varying cognitive abilities and communication challenges

Accomplishments that we're proud of

Built multiple working AI tools with zero coding experience, creating genuine value for speech therapy practice with intellectually disabled adults. Each tool addresses real, specific therapeutic challenges that existing solutions don't handle, from echolalia-friendly interfaces to attention-grabbing visual exercises.

What we learned

How to integrate multiple APIs, handle version control, and build functional AI applications from scratch. More importantly, learned to develop tools that solve highly specific accessibility and communication challenges in therapeutic settings.

What's next for Speech Therapy AI

Expanding the toolkit with additional specialized features: progress tracking systems, pronunciation guides adapted for intellectual disabilities, assessment utilities, and more communication aids tailored to the unique needs of each patient population served.

Built With

  • arasaac
  • bolt.new
  • deepgram
  • github
  • ionos
  • netlify
  • replicate
  • unsplash
Share this project:

Updates