Inspiration
People who are deaf or hard of hearing can easily miss important everyday sounds such as smoke alarms, doorbells, knocking, or a ringing phone. While assistive technologies exist, many are expensive or require dedicated hardware. We wanted to explore how modern web technologies could provide an accessible, low-cost alternative that works directly in the browser. Our goal was to create a solution that improves safety, increases independence, and demonstrates how technology can make everyday environments more inclusive.
What it does
SoundSense is a web application that monitors environmental sounds and converts them into clear visual alerts. When a sound is detected, the app displays a large full-screen notification, provides optional vibration feedback, and logs the event in an alert history.
Users can:
- Detect sounds such as smoke alarms, doorbells, knocking, phone ringing, and loud noises.
- View real-time visual notifications.
- Customize which sounds are monitored.
- Adjust detection sensitivity.
- Enable browser notifications, vibration, and high-contrast mode.
- View analytics and a history of previous detections.
- Use a built-in demo mode to showcase the application's functionality.
How we built it
We built SoundSense using React, Vite, JavaScript, HTML, and CSS. The application uses the Web Audio API to access microphone input and analyze audio in real time. Prototype detection logic processes characteristics such as volume and frequency to identify different categories of environmental sounds. React powers the interactive user interface, while responsive design ensures the application works across desktop and mobile devices. Local browser storage is used to save user preferences and alert history.
Challenges we ran into
One of the biggest challenges was reliably distinguishing between different sounds using only browser-based audio processing. Environmental noise, microphone quality, and varying room conditions all affected detection accuracy. We also needed to balance sensitivity to reduce false positives while still detecting important sounds. Designing an interface that communicated urgent information quickly while remaining simple, intuitive, and accessible was another significant challenge.
Accomplishments that we're proud of
- Built a complete accessibility-focused web application from scratch.
- Developed a modern, responsive, and user-friendly interface.
- Implemented real-time microphone monitoring and visual alerts.
- Added customizable accessibility features including vibration, browser notifications, and high-contrast mode.
- Created analytics, alert history, onboarding, and demo functionality to deliver a polished user experience.
- Demonstrated how browser-based technologies can improve accessibility without requiring specialized hardware.
What we learned
Throughout this project, we learned how to process live audio using the Web Audio API, design accessible user interfaces, and build responsive React applications. We also gained a deeper appreciation for user-centered design and the importance of considering accessibility from the very beginning of the development process.
What's next for SoundSense
Our next step is integrating a machine learning audio classification model to improve detection accuracy and recognize a wider variety of sounds, including emergency sirens, glass breaking, babies crying, and household appliances. We also plan to add smartwatch support, cloud synchronization, customizable sound training, multilingual notifications, emergency contact integration, and smart home connectivity. Our long-term vision is for SoundSense to become an intelligent accessibility companion that helps people stay aware of the sounds that matter most in their daily lives.
Log in or sign up for Devpost to join the conversation.