Ozzy

Voice Assistance for Enhanced Communication Ozzy helps users with speech difficulties communicate more effectively using advanced AI technology to clarify spoken words.

Comment

Inspiration

Our team was inspired by the stories of individuals with speech difficulties who struggle to be understood in daily conversations. We recognize that millions of people worldwide live with conditions like Parkinson's, stuttering, apraxia, or have suffered from strokes that affect speech clarity. When we learned about the hackathon's focus on assistive technology for disabilities, we knew we wanted to create a solution that could help bridge this communication gap and improve quality of life through accessible technology.

What it does

Ozzy serves as a real-time communication assistant that helps individuals with speech difficulties be better understood. The app:

  1. Records speech through the device's microphone
  2. Processes and clarifies the speech using OpenAI's Whisper API, which is specially trained to understand diverse speech patterns
  3. Displays the recognized text on the screen for verification
  4. Provides clear, audible playback using text-to-speech technology
  5. Offers translation capabilities for multilingual communication

This creates a seamless bridge between spoken words and clear communication, empowering users to express themselves confidently in various settings like doctor's appointments, ordering at restaurants, or casual conversations.

How we built it

We built Ozzy as a cross-platform mobile application using:

  • React Native and Expo framework for rapid development and cross-platform compatibility
  • TypeScript for type-safe code that reduces potential bugs
  • OpenAI's Whisper API for advanced speech recognition that works well with diverse speech patterns
  • Expo AV for audio recording and playback capabilities
  • Expo Speech for high-quality text-to-speech synthesis
  • React Navigation for a seamless user experience between screens
  • Context API for theme management (light/dark mode)
  • Environment variables for secure API key management

We prioritized an accessibility-first design approach, ensuring the interface is intuitive and usable for people with various abilities.

Challenges we ran into

Throughout development, we faced several challenges:

  • Optimizing audio recording quality to improve speech recognition accuracy
  • Handling OpenAI API rate limits and ensuring efficient usage
  • Resolving JSX compatibility issues in Expo's third-party libraries
  • Creating an interface that's accessible to users with various disabilities
  • Balancing real-time processing with quality of results
  • Managing the application's performance while processing audio files

A particularly difficult challenge was patching several Expo and React Native libraries to fix JSX syntax errors that were preventing the app from running on web and mobile platforms.

Accomplishments that we're proud of

Despite the challenges, we're proud of:

  • Creating a fully functional application that addresses a real accessibility need
  • Developing an intuitive interface that considers various user abilities
  • Successfully integrating advanced AI technology in a user-friendly way
  • Building a cross-platform solution that works on both iOS and Android
  • Implementing real-time processing with minimal latency
  • Creating a product with potential for real-world impact on people's lives

What we learned

This project was a significant learning experience for our team:

  • We deepened our understanding of accessibility requirements and design principles
  • We gained expertise in working with audio processing in mobile applications
  • We learned to efficiently integrate AI APIs into front-end applications
  • We improved our troubleshooting skills when facing complex library compatibility issues
  • We gained insights into the challenges faced by people with speech disabilities
  • We learned how to balance technical complexity with user experience

What's next for Ozzy

We see great potential for Ozzy's future development:

  • Building a personalized speech model that improves over time as it learns a user's specific speech patterns
  • Adding more languages and dialects to support a broader user base
  • Implementing offline mode for basic functionality without an internet connection
  • Creating custom voice options for text-to-speech playback
  • Developing a companion app for two-way conversations
  • Partnering with speech therapists to incorporate therapeutic exercises
  • Exploring integration with smart home devices and voice assistants

We're committed to continuing the development of Ozzy beyond this hackathon, with the goal of making communication more accessible for everyone. Visit us at www.useozzy.com

Built With

  • babel
  • expo-av
  • expo-speech
  • expo.io
  • framer-motion
  • jsx/tsx
  • lucide-icons
  • next.js
  • openai-whisper-api
  • react-native
  • react-native-paper
  • react-navigation
  • tailwind-css
  • typescript
Share this project:

Updates