Tenor

• 

  Tenor base station and sound sensor

Inspiration

We were inspired by H&R Block's sponsored track to create something that could benefit several members in our community in their day-to-day, and we came up with Tenor. Technology greatly increases our quality of life, but it should also be used to the fullest extent for those who need it the most, such as the deaf/hard-of-hearing (HoH) community. We decided to leverage several different hardware platforms to create a home alert system for this community.

What it does

This product is a highly-scalible home alert system that helps those who are deaf/HoH feel more comfortable in their homes. Our custom hardware is able to detect auditory anomalies and report them to the base station, where a touchscreen can capture the attention of the user and point them in the right direction of this event. Auditory anomalies can include a baby crying, glass breaking, and more. This product allows users to feel much more in-tune with their surroundings in a cost-effective way.

How we built it

The base station's touchscreen is powered by a Raspberry Pi 5 running a QT framework to display relevent information to the user. The Pi 5 communicates with the sound sensor module via WiFi using the MQTT protocol. This custom sound sensor is powered by an ESP32-S3 and uses a simple microphone module to capture and record audio waveforms. The sound sensing module leverages machine learning to pinpoint auditory anomolies using anomaly detection algorithms from EdgeImpulse. This tool is quite helpful as it generates a custom C++ file that can be used to handle anomalies as they occur on embedded hardware.

Challenges we ran into

This project was difficult in many ways, but it mainly all falls back to hardware compatibility. It is very difficult to manage so many different platforms together at once to create something truly cohesive. We even had to make an emergency trip to MicroCenter to get the parts we needed. We were also completely new to using ML, so using it for the first time for something as simple as this was quite challenging for us.

Accomplishments that we're proud of

We are very proud of the custom hardware solution we were able to implement. We are confident that this solution is almost as cost-effective as it gets, and a lot went into making it look and feel the way that it does. We are also super proud of what we were able to accomplish in such a short amount of time, as well as given our very limited hardware constaints.

What we learned

We learned that tackling a hardware project like this at a hackathon is extremely difficult due to the limit hardware available. This made a lot of the ideas we originally came up with unfeasible, and makes developing the idea we finally landed on even more difficult as we ended up using all of our own personal hardware for this event.

What's next for Tenor

If we were given more time, we are interested in implementing more advanced anomaly-detection algorithms that could detect specific sounds and display them for the user. Due to the high-scalability of our design, we could easily add more sound detectors to detect sound in larger spaces.

Built With

• c++
• edgeimpulse
• esp32
• ml
• mosquitto
• platformio
• python
• qt
• raspberry-pi