The inspiration behind creating a "SOUNDMATE : Smart Device for the Hearing Impaired" concept came from witnessing one of our team members’ grandfather face challenges with hearing due to age. Observing his struggles with daily communication and reliance on others for safety motivated us to develop technology that could support greater independence for those with hearing impairments. While the smart watch itself remains part of our future development scope, we focused on building the foundational components, including an audio classification model and speech-to-text functionality. These elements will eventually integrate into a wearable device that can detect critical sounds, such as car horns and sirens, and alert the user through vibrations and visual notifications on a mobile app.
The Smart Wearable for the Hearing Impaired is an innovative device that will be designed to enhance safety and situational awareness for individuals with hearing impairments. It will detect and classify important environmental sounds, such as car horns and sirens, alerting users through vibrations when these critical sounds are identified. The device will also provide real-time speech-to-text translations via a mobile app, ensuring users remain aware of their surroundings. Additionally, it will indicate the relative direction of detected sounds, helping users navigate their environment more effectively. The wearable will notify users when their registered name is called out, ensuring they never miss important interactions. Visual representations of audio classifications will be displayed on the mobile interface, providing an intuitive and interactive way for users to understand their environment.
We designed the audio classification model. The input (which can be the recording audio file of surrounding noise) is processed by calculating Mel Frequency Cepstral Coefficients (MFCC) for feature extraction and classification. Our model can identify important sounds and trigger notifications, enabling users to recognize specific alerts. Additionally, we implemented a speech-to-text feature on a mobile app, allowing users to receive real-time visual representations of sounds, further supporting communication and awareness.
The most significant challenge was the lack of available audio capturing devices during development. This limitation prevented us from implementing real-time audio monitoring features for our ML model, requiring us to work with pre-recorded audio files for testing and validation.
The project successfully achieved an impressive 88% accuracy in the audio classification model, demonstrating its ability to accurately identify critical environmental sounds. This achievement highlights the effectiveness of the solution in addressing a significant need for individuals with hearing impairments, providing them with reliable and timely alerts to enhance their safety and awareness. Additionally, the team developed a user-friendly mobile interface, designed to offer clear and intuitive visual feedback, allowing users to easily interpret the detected sounds and their classifications. This practical solution is poised to make a meaningful impact on the daily lives of users, offering both functionality and ease of use.
Through this project, we gained valuable experience in audio processing, edge computing, and the potential of adaptive technology to enhance independence and safety for individuals with hearing impairments. This work has inspired us to continue exploring assistive technologies and paves the way for future development of the envisioned smart watch, further enhancing its practical impact on users' daily lives.
We have outlined several exciting future developments for the Smart Device for the Hearing Impaired. To enhance user privacy and data security, SOUNDMATE plans to implement blockchain technology as a crucial component of its infrastructure. By deploying the application on a blockchain network, we can ensure that all user data, including audio processing history, personalized settings, and speech-to-text translations, are stored in a decentralized and encrypted manner. This implementation will give users complete control over their sensitive data through smart contracts, preventing unauthorized access and potential misuse. The blockchain integration will create an immutable record of system interactions while maintaining user anonymity, which is particularly important for healthcare-related applications. Smart contracts can also enable secure sharing of usage patterns with healthcare providers when authorized by users, potentially improving the quality of hearing care services. Additionally, blockchain's transparent nature will help build trust among users while ensuring their privacy rights are protected. This technological advancement aligns with SOUNDMATE's mission to provide not just an assistive device, but a secure and trustworthy platform that respects user privacy in the digital age.
Log in or sign up for Devpost to join the conversation.