Sound-ificator-inator

Sound-ificator-inator is meant to block out sounds for those with sensory issues and instead replace them with an environment of their choosing. It utilizes real-time data to create spatial audio.

Comment 1

TEAMMATES

Inspiration

The original idea came from one of our members struggling with sensory overload in crowds. We were inspired by the concept of what echo3D's software could do, even if we did not end up using it in the final product.

What it does

Sound-ificator-inator uses the raspberry pi sense hat to emulate a person turning their head. We do not have earbuds or headgear with gyro, so we decided to emulate it with the stuff we did have. Turning the pi (turning your head) will cause differences in the ambiance noises you hear. Imagine hearing a sound behind you and then turning so your ear is facing it; it emulates that effect. This is meant to mimic real life as closely as possible to help users feel immersed in their soundscape environment. Turning your head quickly to notice the changes in sound could be a stim for some! Users can also select a soundscape from three different environments (dream, water, forest).

Further, there is a green cube to help visualize the angle and orientation of the raspberry pi. You can better see this in the video.

How we built it

We started by separating features into smaller goals/tasks. From there, we each started picking ones that played to our strengths and sat in a large circle for several hours. We would pair program too! After some troubleshooting, food poisoning (we are serious; one of us had to go home very early), and brainstorming workarounds, our final tech stack was:

  • JS
  • Express.js (for the API server)
  • Three.js (for the 3D rendering)
  • HTML/CSS
  • Python (for the raspberry pi scripts)
  • Vite (for building three.js code)
  • NPM
  • Render (for deploying and hosting the API server)

Challenges we ran into

DEPLOYMENT. We needed a special compiler (vite) for the 3D graphics to be rendered in-browser and making a functional deployment workflow was our biggest challenge. Three professors and several hours later, we managed to scrape by.

Accomplishments that we're proud of

My teammates are taking a well deserved rest, so I will just speak on my own opinion. I am most proud of the overall product. I can't believe we all managed to make something this cool in under 24 hours. It made me so happy; I'm not sure how else I can put it.

If you want specific accomplishments, I'm proud of how flexible I was. My team and I bounced questions off of each other a lot and I felt like I had answers for most, despite feeling lost in some parts of the development process. I even managed to set up my own API server—which I have never done before—and help my team implement it into their parts of the project. It was INCREDIBLY fun to work with all of them and I couldn't have asked for better teammates.

Also, Blair said he was really proud of how the site looked. He did most of our front end. I can't speak for Michael or Zach but they should be proud of everything they did; everyone did a LOT of heavy lifting in various areas. --Blade

What we learned

  • An uncomfortable amount about vite and how it decides to compile files. We got really good at debugging vite issues from just the built file.
  • A lot about three.js! It's a library that none of us have ever used before and it turns out you can make really cool stuff with it. We'll definitely keep it in mind for the future.
  • We managed to hook up our pi to our website without ssh! There was an issue with the OS on my pi that prevented it from using ssh, so we brainstormed some workarounds (bluetooth, APIs) and actually figured out how to connect a pi to an API.
  • Grabbing PI oriantation data in Python. New to us; we did it in C like 3 years ago once.
  • Render can be used for really quick and easy stuff.
  • Coding for 20 hours without sleeping is not nearly as exhausting as you would think it is. We honestly weren't that tired. It was too much fun building something cool.

What's next for Sound-ificator-inator

The 'calibrate orientation' button should reset the pi gyro orientation to whatever it started as when the session began. We did not get to implement this functionality due to time constraint. Further, we accidentally stole the 'henhacks.github.io' URL and this could be used for the real HenHacks staff, so we will be changing the name ASAP.

Share this project:

Updates